Job Description
Job DescriptionAbout the Role
We are seeking an experienced Senior Data Engineer with deep expertise in Databricks to design, develop, and optimize enterprise-scale data pipelines. You'll play a key role in building scalable Lakehouse solutions while ensuring reliable, high-quality data for analytics, reporting, and downstream applications.
Key Responsibilities
- Design and build scalable ETL/ELT pipelines using Databricks
- Develop PySpark and Spark SQL transformations
- Build Bronze, Silver, and Gold data layers using Medallion Architecture
- Implement Delta Live Tables, Workflows, and Unity Catalog
- Optimize Spark jobs for scalability and performance
- Design dimensional data models for analytical workloads
- Implement data quality validations and monitoring
- Collaborate with analytics, engineering, and architecture teams
- Build reusable frameworks and automation
- Support CI/CD deployment pipelines and Git-based development
Required Qualifications
- 7–10+ years of Data Engineering experience
- Extensive hands-on Databricks experience
- Strong PySpark programming
- Advanced Spark SQL skills
- Experience with Delta Lake
- Experience implementing Medallion Architecture
- Hands-on Delta Live Tables
- Unity Catalog experience
- Strong SQL optimization and performance tuning
- Experience with AWS, Azure, or GCP
- CI/CD and Git experience
- Strong understanding of distributed data systems
- Experience designing enterprise data models
Preferred Qualifications
- Databricks certification
- Experience with streaming data
- Experience building reusable data frameworks
- Experience with enterprise governance
- Exposure to AI-assisted development tools including Databricks AI Genie
#LI-WJ1
#PSI2