Job Description
Job Description
Data Engineer
Experience
2–5 years
Job Type
Full-time
Job Summary
We are looking for a skilled Data Engineer to design, develop, and maintain scalable data pipelines and data processing solutions. The ideal candidate should have strong experience in SQL, Python, ETL/ELT processes, cloud data platforms, and data warehousing.
Key Responsibilities
- Design, develop, and maintain reliable data pipelines for batch and real-time data processing.
- Build and optimize ETL/ELT workflows from multiple data sources.
- Develop complex SQL queries, stored procedures, and data transformations.
- Work with data warehouses, data lakes, and cloud-based data platforms.
- Perform data cleansing, transformation, validation, and integration.
- Monitor data pipelines and troubleshoot failures, performance issues, and data-quality problems.
- Optimize data processing jobs and database queries for performance and scalability.
- Implement data-quality checks and ensure data accuracy and consistency.
- Collaborate with Data Scientists, BI Developers, Analysts, and Software Engineers.
- Participate in data architecture and pipeline design discussions.
- Maintain technical documentation for data pipelines, workflows, and data models.
- Follow security, governance, and best practices for handling enterprise data.
Required Skills
- Strong proficiency in SQL.
- Strong programming experience in Python.
- Hands-on experience with ETL/ELT pipelines.
- Experience with data warehousing concepts and dimensional data modeling.
- Experience with databases such as SQL Server, PostgreSQL, MySQL, or similar.
- Experience working with cloud platforms such as Microsoft Azure, AWS, or GCP.
- Understanding of data lakes and distributed data processing.
- Experience with version control systems such as Git.
- Strong understanding of data quality, data validation, and error handling.
Preferred Skills
- Experience with Microsoft Fabric, Azure Data Factory, Azure Synapse, or Databricks.
- Experience with Apache Spark/PySpark.
- Knowledge of Power BI and BI data models.
- Experience with streaming technologies such as Kafka.
- Knowledge of CI/CD and DevOps practices.
- Experience working with REST APIs and integrating data from external systems.
- Familiarity with data governance, security, and compliance.
Education
- Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field.
Soft Skills
- Strong analytical and problem-solving skills.
- Good communication and collaboration skills.
- Ability to work independently and as part of a team.
- Strong attention to detail.
- Ability to troubleshoot complex data issues.
Key Qualifications
- 2–5 years of hands-on experience in Data Engineering.
- Strong SQL and Python expertise.
- Practical experience developing and supporting production data pipelines.
- Good understanding of cloud data engineering and modern data architecture.