Search

Lead Python Engineer - Big Data & Risk Analytics

PublishedPublished: 6/14/2022
Technology
|

Job Description

Job Title: Lead Python Engineer – Big Data & Risk Analytics

\n

Location: Jersey City, NJ

\n

Duration: Contract - 6 months (C2H potential)

\n


\n

Job Description

\n

We are seeking a Lead Engineer, Big Data Risk Aggregation to join our dynamic team. The ideal candidate will have strong experience in large-scale data engineering, Apache Spark, and risk aggregation within Capital Markets and a proven ability to design and operate high-performance, auditable aggregation platforms that deliver reportable risk metrics.

\n

Responsibilities:

\n

    \n
  • Architect and build a high-performance, metadata-driven aggregation platform using Apache Spark.
  • \n

  • Design and implement algorithms to reduce computational complexity for high-dimensional risk aggregations.
  • \n

  • Create and maintain a semantic layer and business-friendly data model to power BI and AI-driven queries.
  • \n

  • Optimize Spark internals including partitioning, shuffle strategies, and memory for extreme performance.
  • \n

  • Build a self-service framework that enables quants and risk managers to define new aggregations without new code.
  • \n

  • Deliver high-performance APIs and curated datasets for UIs and Python Notebooks to support analysis and replication.
  • \n

  • Load aggregated metrics into OLAP stores such as Apache Pinot or Apache Druid to support analytics and UI layers.
  • \n

\n

Required Skills & Qualifications:

\n

    \n
  • 10+ years of hands-on large-scale data engineering experience, preferably in Capital Markets or Risk technology.
  • \n

  • Expert-level Apache Spark knowledge including architecture, performance tuning, shuffle optimization, and partitioning.
  • \n

  • Deep understanding of risk aggregation for VaR, CCAR, Stress Testing, and FRTB.
  • \n

  • Strong programming skills in Java and/or Python with expert-level SQL.
  • \n

  • Experience building semantic layers and data models enabling BI and AI applications.
  • \n

  • Proven delivery with high-performance OLAP databases such as Apache Pinot or Apache Druid.
  • \n

  • Degree in Computer Science, Financial Mathematics, Financial Engineering, or a related quantitative field.
  • \n

\n

Preferred Skills:

\n

    \n
  • Experience leading data engineering teams and establishing best practices for big data processing.
  • \n

  • Demonstrated strength in performance diagnostics across CPU, memory, and network I/O.
  • \n

\n


Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...