Job Description
Job DescriptionSr. Data Engineer
Remote
Our customer is a growing health benefits company. The company is hiring a Sr Data Engineer to build the data foundation behind its cost prediction and pricing models. This is the beginning of an overarching technology modernization for a rapidly growing organization. The ideal ideal candidate has experience designing and building data pipelines and models in AWS for claims data.
Responsibilities
- Design, build, and maintain fully serverless data pipelines on AWS (Lambda, Glue, Athena, Step Functions, S3) supporting analytics, reporting, and AI use cases
- Build versioned data models and feature stores with clear semantics, reproducible backfills, and safeguards against leakage or inconsistent definitions
- Develop batch and event-driven ingestion pipelines (Kafka, Kinesis, or similar) for near-real-time scoring and decisioning
- Implement automated data quality controls, monitoring, and lineage, including schema change, reconciliation, and data maturity checks
- Partner with AI and Analytics teams on MLOps: orchestrating training and scoring workflows and capturing model and data versions to close the learning loop
- Optimize cost and performance through Athena query tuning, columnar formats (Parquet, Iceberg), and efficient S3 partitioning
- Enforce engineering best practices, including code review, CI/CD for data pipelines, and Infrastructure as Code with AWS CDK
Requirements
- 5+ years of data engineering with ownership of production pipelines and analytics platforms; hands-on ML/AI support strongly preferred
- Advanced Python and SQL, plus production experience with AWS serverless data services; PySpark strongly preferred
- Infrastructure as Code with AWS CDK (TypeScript)
- Experience with streaming or event-driven pipelines (Kafka preferred) and open table formats such as Iceberg
- Data governance and security for sensitive, regulated data (PHI, financial); healthcare claims or insurance experience strongly preferred