Sr. Data Engineer

Job Description

Description

May sit hybrid onsite in Wilmington, DE, McLean, VA, Richmond, VA or NYC\n
\n
\nOur client seeks a Data Engineer professional with strong ability to support a large-scale customer migration and data load initiative. The role will build and optimize AWS-based data pipelines using Java, Scala, and Spark, with PySpark and AWS Glue across multi-stage workflows. The engagement will emphasize event-driven patterns, data quality, and operational resilience.\n
\n
Due to client requirements, applicants must be willing and able to work on a w2 basis. For our w2 consultants, we offer a great benefits package that includes Medical, Dental, and Vision benefits, 401k with company matching, and life insurance.

Rate: $70.00 to $80.00/hr. w2\n
\n
Responsibilities

\n

\n
• Design, build, and maintain AWS data pipelines using S3, Glue, Step Functions, and Lambda.

\n

• Develop back-end data processing services in Java and Scala with Spark and PySpark.

\n

• Implement event-based architectures leveraging EventBridge and S3 event notifications.

\n

• Configure and scale AWS Glue jobs, addressing version and table format considerations.

\n

• Establish data validation, security checks, schema drift handling, and backfill strategies.

\n

• Integrate external API calls within data pipelines where required.

\n

• Contribute to CI/CD pipelines and Spring Boot based repositories.

\n

• Support production runbooks, reruns from event triggers, and downstream job synchronization.
  
  

\n

Experience Requirements

\n

\n
• Hands-on Java with data engineering background. Python considered, Java preferred.

\n

• Proficiency with Scala, Spark, and PySpark.

\n

• Strong AWS experience: S3, Glue, Step Functions, Lambda, and EventBridge.

\n

• Knowledge of state machine scheduling and multi-stage pipeline orchestration.

\n

• Experience with data quality controls, schema evolution, and backfill management.

\n

• CI/CD pipeline implementation and operations.

\n

• Spring Boot repository familiarity.

\n

• Plus: external API integration in pipelines and AI automation exposure.

\n

• Prior experience supporting financial services data migrations preferred. Prior Capital One experience preferred.
  
  

\n

Education Requirements