Job Description
Position : Senior Data Engineer
\n
Location : Charlotte, NC / Wilmington, DE / Columbus, OH
\n
Day 1 Onsite (Hybrid)
\n
\n
Experienced Senior Data Engineer (12+ Years) to support large‑scale data platform modernization initiatives within a regulated banking environment.
\n
The role focuses on designing and building enterprise-grade in-house frameworks, supporting high-volume batch and CDC-based incremental processing using Cloudera platform, and enabling ongoing Google Cloud Platform (GCP) modernization efforts.
\n
\n
Technology & Skill Requirements
\n
- \n
- Apache Spark (PySpark and/or Scala) in large-scale production environments
- Cloudera Hadoop ecosystem (HDFS, Hive, YARN, Spark on Cloudera)
- Strong SQL expertise with complex transformations, performance tuning, and reconciliation logic
- Enterprise RDBMS experience with Oracle and MS SQL Server
- Batch ingestion, incremental ingestion, and CDC processing patterns
- CDC concepts and tooling (tool-agnostic: GoldenGate, Debezium, or equivalent)
- Data merge, deduplication, watermarking, checkpointing, and SCD handling
- Google Cloud Platform services including Dataproc , Composer and Dataplex
- Hybrid on‑prem to cloud data architecture and migration patterns
- Metadata-driven framework development and data quality validation techniques
- CI/CD pipeline implementation using enterprise tooling (GitHub Actions, Jenkins, DevOps)
- Git-based development workflows, code reviews, and automated testing practices
- Experience using Copilot or similar AI-assisted development tools safely and effectively in enterprise environments
- Logging, monitoring, alerting, and operational readiness practices
- Secure coding, access control, and compliance-aware development
- Documentation of design artifacts, runbooks, and operational procedures
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n