Job Description
Role Summary
\n
Lead efforts to implement new Data Quality standards, emphasizing on modernizing existing ETL pipelines with reusable engineering patterns for implementing Data quality in a "shift left" approach.
\n
\n
Required Skills
\n
Strong ETL/ELT and data engineering experience (Datastage, DBT, Informatica, Snowflake)
\n
Strong SQL + Python skills
\n
Hands-on Data Quality engineering experience
\n
Experience embedding automated DQ checks into pipelines
\n
Strong CI/CD and automated testing knowledge
\n
Understanding of shift-left/inline DQ architecture
\n
Experience designing reusable, scalable pipeline/DQ patterns
\n
Technical leadership and ability to drive adoption across engineering teams
\n
\n
Key Deliverables
\n
DQ-Enabled ETL Pipelines
\n
Implement DQ checks across priority ETL pipelines
\n
Incorporate completeness, validity, accuracy, consistency, uniqueness, and reconciliation checks
\n
Determine appropriate DQ checkpoints throughout the pipeline
\n
Automated DQ Testing & CI/CD Integration
\n
Integrate DQ validation into automated pipeline testing and CI/CD processes
\n
Establish quality gates and thresholds for pipeline promotion
\n
Automate execution and reporting of DQ tests
\n
DQ Failure & Exception Handling
\n
Implement standardized approaches for handling DQ failures
\n
Define reject, quarantine, warning, and pipeline-stop patterns
\n
Enable automated alerting and issue routing
\n
DQ Monitoring & Reporting
\n
Implement monitoring of DQ results within ETL pipelines
\n
Provide visibility into DQ failures, trends, and recurring issues
\n
Capture DQ results and metadata for downstream reporting and governance