For our client, a globally active asset management firm with a strong presence across institutional and private client segments, we are currently looking for an Data Engineer - Databricks (gn) to strengthen the team.

Purpose of job

The role is responsible for designing, building and operating data products and reusable data preparation components on a Databricks-based Data and AI Platform, while acting as a technical enabler for internal platform users through expert guidance and advanced-level support. The position ensures adherence to security, privacy and regulatory requirements via a compliance‑by‑design approach, maintains alignment with established best practices for data pipelines and data quality, and drives continuous platform improvement through the integration of new features and standardized, reusable pipelines.

Tasks

Develop and maintain a library of modular, reusable pipeline components — covering data intake, processing, verification and enrichment — to produce consistently structured, AI/ML‑ready datasets from both structured and unstructured sources.
Architect and run dependable data workflows on Databricks, pulling from a wide range of internal and external origins to produce clean, validated data assets available for AI, ML and reporting purposes.
Govern the lifecycle of layered data assets across maturity tiers, upholding quality and timeliness standards while maintaining purpose‑specific, analytics‑ and model‑ready output datasets.
Build and maintain transformation workflows that derive meaningful predictive attributes from raw data, alongside a centrally managed attribute repository with clear versioning, ownership and service‑level commitments.
Work alongside AI and ML engineers to establish and maintain data supply chains for retrieval‑augmented generation systems, covering content segmentation, vector representation updates and index synchronization.
Embed governance controls — covering permissions, traceability, data lifecycle management, encryption and audit trails — into platform design to meet both internal policies and external regulatory requirements.

Required

At least 3 years of hands‑on experience designing and running large‑scale data pipelines and data products on Databricks — batch and/or streaming — preferably in regulated or governance‑heavy environments (ideally in the Financial Services industry)
Advanced proficiency in Databricks, Spark, Python and SQL, complemented by sound software engineering practices such as CI/CD workflows and infrastructure‑as‑code tooling (e.g., Terraform) on Azure.
Solid grasp of data engineering principles relevant to AI/ML contexts, including data modelling, quality assurance, feature collaboration and reproducibility, as well as an understanding of how data characteristics influence model behaviour.
Strong command of modern data architecture patterns — including Lakehouse principles, layered data organisation and data product thinking — and the ability to put them into practice within a governed enterprise setting.
Degree in computer science or a comparable discipline.
Strong analytical mindset paired with structured planning, clear documentation and the ability to effectively transfer knowledge to colleagues.

Preferred

Professional background in the financial sector, ideally within asset management, combined with international work experience and relevant Databricks certifications.

#J-18808-Ljbffr

BLACKBULL INTERNATIONAL GmbH