Search

Senior Data Engineer

PublishedPublished: 6/14/2022
Technology

Job Description

Senior Data Engineer — Contract

\n

Location: Hybrid — 3 days/week onsite in Newark, NJ

\n


\n

About the Role

\n

Join a major investment firm as a Senior AWS Data Engineer supporting enterprise data platform initiatives. This role focuses on building scalable data pipelines, CI/CD automation, and end-to-end data solutions across the AWS ecosystem. Experience with Informatica (data catalogue, marketplace, data lineage, data quality), Denodo, and/or Ataccama is a strong plus.

\n


\n

Qualifications

\n

    \n
  • Bachelor's degree in Computer Science, Software Engineering, MIS, or equivalent combination of education and experience
  • \n

  • 8+ years of experience as a Data Engineer on the AWS stack, including DevOps tooling
  • \n

  • AWS Solutions Architect or AWS Developer Certification required
  • \n

  • Strong hands-on experience with AWS services, including:
  • \n

  • CloudFormation, S3, Athena, Glue, Glue DataBrew, EMR/Spark
  • \n

  • RDS, Redshift, DynamoDB, Lambda, Step Functions
  • \n

  • IAM, KMS, SM, EventBridge, EC2, SQS, SNS
  • \n

  • LakeFormation, CloudWatch, CloudTrail, DataSync, DMS
  • \n

  • Experience implementing high-velocity streaming solutions using Amazon Kinesis, AWS Managed Airflow, and AWS Managed Kafka (preferred)
  • \n

  • Proven experience building AWS data lake/data warehouse solutions
  • \n

  • Skilled in designing, developing, and implementing data ingestion pipelines on AWS
  • \n

  • Knowledge of ETL/ELT implementation for data solutions
  • \n

  • Experience delivering end-to-end data solutions: ingest, storage, integration, processing, and access
  • \n

  • Familiarity implementing RBAC strategies using AWS IAM and Redshift RBAC model
  • \n

  • Experience building and implementing CI/CD pipelines using CloudFormation and Jenkins
  • \n

  • Strong programming skills in Python, Shell scripting, and SQL
  • \n

  • Experience with SQL stored procedures for data analysis
  • \n

  • Experience building automated pipelines to ingest data from relational databases, file systems, and NAS shares into AWS RDS, Aurora, and Redshift
  • \n

  • Experience building automated pipelines to ingest data from REST APIs into AWS S3 and relational databases
  • \n

  • Solid DevOps background, including:
  • \n

  • Bitbucket (source code management)
  • \n

  • Python, Shell, Groovy scripting
  • \n

  • API deployment for tooling integration
  • \n

  • Jenkins and CloudBees (Pipeline as Code, Shared Libraries)
  • \n

  • SonarQube, Artifactory, and similar DevOps tools
  • \n

  • Maven, MS Build, Gradle
  • \n

  • Docker
  • \n

  • Jira and Confluence
  • \n

  • Infrastructure as Code via CloudFormation
  • \n

  • Experience creating Jenkins CI pipelines integrating Sonar/security scans and test automation
  • \n

  • Ability to accurately document exceptions, issues, action plans, meeting minutes, and lessons learned
  • \n

\n


\n

Responsibilities

\n

    \n
  • Design, build, and maintain efficient, reusable, and reliable code
  • \n

  • Ensure optimal performance and quality of high-scale data applications and services
  • \n

  • Participate in system design discussions
  • \n

  • Independently perform hands-on development and unit testing
  • \n

  • Collaborate with the development team to integrate components into the enterprise data platform
  • \n

  • Work cross-functionally with product, QE/QA, and other teams throughout the full software development lifecycle
  • \n

  • Identify and resolve performance issues
  • \n

  • Stay current with emerging technologies and implementation practices
  • \n

  • Participate in code reviews to ensure standards and best practices are met
  • \n

  • Take ownership of estimating, planning, and managing tasks in an Agile environment
  • \n

  • Take shared responsibility for software quality rather than relying on separate QA teams
  • \n

  • Collaborate closely with all team members toward shared goals
  • \n

  • Engage with users as needed to clarify requirements
  • \n

\n


Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...