Senior Data Platform Lead

JobtailorRemote

Stellenbeschreibung:

Own the technical strategy, architecture, and roadmap for the organization's data platform, with Databricks as the primary compute and analytics engine

Define platform options and roles and responsibilities in the federated environment

Lead platform reliability, performance, scalability, SLA compliance, and operational risk reduction

Establish and enforce engineering standards, including CI/CD, infrastructure-as-code, and testing practices

Oversee Databricks cluster configuration, cost management, and resource optimization

Drive adoption of best practices across Delta Lake, Unity Catalog, MLflow, and related components

Create a business- and consumer-oriented data products shopping experience

Technically realize data governance and data quality frameworks with the Enterprise Data Governance team

Triage and prioritize platform demand from business units, analytics teams, and product owners

Translate stakeholder requirements into platform solutions while balancing delivery speed, architectural integrity, and roadmap priorities

Serve as senior technical contributor and platform authority on enterprise data transformation programs

Collaborate with data architects, data product owners, and data governance leads

Provide technical due diligence and risk assessment for tools, vendors, and integration patterns

Contribute to data strategy documentation, business cases, and executive presentations

Lead or oversee batch ingestion, event streaming, and API-based data movement patterns

Ensure connectivity between Databricks and upstream/downstream systems

Evaluate and manage integration tooling such as ADF, dbt, Fivetran, and Kafka

Define standards for data contracts, schema management, and pipeline observability

Directly manage and develop 4-7 consultants who are data and platform engineers

Foster engineering excellence, continuous improvement, and collaborative problem-solving

Manage third‑party vendor relationships

Support financial planning up to a three-year horizon

Work in a hybrid model with office attendance aligned with local regulations and regular travel

Requirements

Around 10+ years of experience leading data platform and/or data engineering teams in a complex enterprise environment

Deep hands-on expertise with Databricks, including Spark, Delta Lake, Databricks SQL, Workflows, and Unity Catalog

Strong understanding of lakehouse architecture and cloud-native data infrastructure (Azure preferred / AWS / GCP)

Experience owning a technical platform end-to-end, including reliability, security, cost, and roadmap

Track record of working across business and technology stakeholders to manage demand and deliver outcomes

Previous experience leading a team

Familiarity with data governance, data cataloguing, and metadata management practices

Fluency in English

Background in data integration, including pipeline design, event streaming, or ETL/ELT tooling (highly desirable)

Experience contributing to large-scale strategic data programs (highly desirable)

Exposure to data mesh, data product thinking, or domain-oriented data ownership models (highly desirable)

Familiarity with MLflow, Feature Store, or Databricks Model Serving (highly desirable)

Executive presence and ability to influence without authority

Growth mindset and commitment to personal and team development

Reference to the Leadership Framework is checked during the recruitment process

Core Competencies

Demonstrates expertise in leading data platform strategies and architectures, particularly with Databricks and cloud-native data infrastructures. Proven ability to manage cross-functional teams, drive data governance, and deliver high-quality data products while ensuring operational excellence.