Senior AI R&D Engineer – Pre-training Data (f/m/d)

Stellenbeschreibung:

At Aleph Alpha, we are one of the few companies in Europe working on foundation model pre‑training. Our customers in finance, manufacturing, public administration need models that understand German, meet European regulatory requirements, and work reliably in high‑stakes settings. We’re building that in Heidelberg.

Team culture

We foster a culture built on ownership, autonomy, and empowerment. Teams and individual contributors are trusted to take responsibility for their work and drive meaningful impact. We maintain a flat organizational structure with efficient, supportive management that enables quick decision‑making, open communication, and a strong sense of shared purpose.

About the Role

As a Senior AI Engineer in Pre‑training Data, you will work across the full stack of data preparation – from sourcing and acquisition to processing, filtering, and mixture design. You’ll deep‑dive into data quality analysis, evaluate what makes a corpus valuable, optimize large‑scale pipelines, build tooling that gives the team visibility into what our models train on, and read the latest research on pre‑training data methods.

We approach data work evidence‑based. Decisions about filtering strategies, data mixtures, and quality thresholds are backed by ablations – you’ll design and run targeted experiments to validate that your data choices improve model outcomes.

We look for someone who combines significant research experience (in industry or academia) with high engineering competence. Your work sits at high leverage: data you source, curate, and synthesize directly determines what the models learn, how well they perform, and where they fall short. You’ll have direct influence on the models we ship.

Your Responsibilities

Co‑Own data pipelines end‑to‑end: Design, build, and maintain the infrastructure that sources, processes, deduplicates, filters, and prepares pre‑training corpora at scale. Own the conversion from curated corpora to training‑ready streaming formats.
Curate and compose data mixtures: Define and iterate on the data blends used for pre‑training – balancing domains, languages, quality tiers, and licensing requirements to maximise model capability.
Build data quality tooling: Develop classifiers, heuristics, and analysis frameworks that measure and enforce data quality across terabyte‑scale corpora. Monitor pipeline health and data quality metrics at scale.
Close data gaps: Work with evaluation and post‑training teams to identify where model weaknesses trace back to data coverage, then source or generate the data needed to address them.
Collaborate with post‑training: Partner closely with the post‑training team to ensure pre‑training data decisions support downstream fine‑tuning, alignment, and deployment goals – data choices upstream shape what’s possible downstream.
Co‑Own German‑language data: Ensure deep, high‑quality coverage of German‑language corpora – this is core to our value proposition, not an afterthought.
Establish data‑to‑performance signal: Design and run ablation studies to validate data choices – measuring how changes in composition, filtering, or sourcing affect pre‑training evaluation metrics and downstream capabilities.
Take data transparency seriously: Maintain data lineage and provenance so the team knows exactly what went into each training run.

Your Profile

Basic Qualifications

Track record of shipping impactful technical work – research, infrastructure, or both.
Strong Python skills and comfort with data engineering and ML infrastructure, including experience with deep learning frameworks, workflow orchestration, object storage, columnar data formats, and distributed processing.
Ability to reason about what a dataset contributes to model training and whether it matters – not just process data, but understand it.
Ownership mentality: see problems through from diagnosis to solution to deployment.
Willingness to relocate to Heidelberg or travel at least fortnightly.

Preferred Qualifications

Experience with large‑scale data processing for ML – corpus sourcing, curation, cleaning, deduplication, and filtering.
Familiarity with data quality methods: classifier‑based filtering, heuristic scoring, perplexity‑based selection, and decontamination.
Understanding of foundation model training – how data composition, scale, and mixing ratios affect capabilities.
Experience with web‑scale data sourcing and crawl processing (Common Crawl, WARC pipelines).
Rust proficiency – parts of our data pipeline are performance‑critical.
Infrastructure knowledge – experience with Kubernetes, container orchestration, or cloud‑native ML infrastructure.
PhD in machine learning, NLP, data engineering, or a related field (valued but not required).
Bonus: German language proficiency can be helpful for curating and assessing German‑language data.