Job Description
Job Description Role: Data Engineer - Databricks, Scala
Skills: Digital: Scala, Digital: Databricks Experience Required: 8-10 Years
Databricks, Scala- Data Engineer
Must have atleast 8 years of extensive Data engineering experience in Databricks, ETL/ELT using data pipelines, SQL/Procesurr experience
Must have 8 years of hands on development proficiency in implementing Databricks solutions using scala and spark with data frames and notebooks/sql
Must have 8 years of hands on development and performance tuning/ enhancement experience in scala/pyspark and sql
Must have strong experience with Databricks, including developing and optimizing spark jobs, data transformations, and data processing workflows
Must have experience and a good understanding on the below topics
Databricks delta lake storage
Unity catalog
Autoloader
Lakehouse architecture
Medallion architecture
Databricks serverless options
Dataset and data frame
Databricks clusters configuration and sizing
Databricks job orchestration
Spark structured streaming