Applied ML Scientist, Audience Behavior Platform

Job Description

Job Description

About DDM Group

DDM Group is a digital media company with three properties: Atlanta Black Star, The Shadow League and Finurah. Over 14 years, DDM grew its audience to more than 20 million monthly users at peak entirely through organic distribution, without paid traffic. DDM has turned what it learned into a proprietary model of what captures an audience's attention and what moves people to change their behavior. We are now testing that model rigorously, and building it into an AI-powered service.

The role

You will own the evaluation of the model from raw data to verdict. You will build the dataset from 14 years of content and analytics (nearly 100,000 published articles on Atlanta Black Star alone, each with Google Analytics history), have AI models score the full archive, validate those scores against trained human raters on a sample, test whether those scores predict real audience behavior, and run live experiments on our properties. Your findings decide what we build and where we invest. We want an honest answer, including if the answer is no. If the model holds up, you will turn the evaluation pipeline into the core of the product: the scoring engine, the generator that proposes signals and plans, and the experiment and learning loop.

This is a remote contract role starting immediately. You will report to Neil Nelson, CEO, who owns the model, and work closely with our editorial team, whose producers serve as raters.

What you will do

• Build the evaluation dataset: export story text and metadata from our WordPress sites, pull Google Analytics 4 data by traffic source, and recover older data from other sources where needed.

• Design a balanced, representative sample, including stories that failed, not only the hits.

• Run the scoring: set up blind rating, train and calibrate human raters with the editorial lead, measure how well raters agree, and write and validate the prompts for AI raters against human judgment.

• Test the model: fit weights on one part of the data and validate them on data held back; compare alternative ways of combining scores; control for distribution effects such as platform algorithm changes and traffic source.

• Test early-warning indicators on historical audience data.

• Design, size and analyze live A/B tests on our properties.

• Keep a clean, reproducible record of every dataset, decision, score, prediction and result.

• Report findings plainly to the CEO, against success criteria agreed before the analysis begins.