Lead Data Engineer - Data Migration & Lakehouse

Description

You thrive on turning decades of legacy complexity into clean, modern data architecture. You feel at home navigating undocumented mainframes, tangled schemas, and ambiguous specs — and you know how to translate that chaos into a clear technical path forward. If you love owning complex migrations end-to-end, building trust with stakeholders through clarity and honesty, and shaping how data moves from legacy into the lakehouse era — this is for you.

Your tasks

• Migration Ownership: Lead large-scale migration projects from legacy systems (ranging from SQL over ERP to mainframes) into modern cloud/lakehouse targets, taking full technical responsibility.
• Databricks Architecture: Design and implement robust pipelines using Spark, Delta Lake, Unity Catalog, Lakeflow Declarative Pipelines, and Databricks Asset Bundles for CI/CD-style deployment.
• Data Modeling & Reconciliation: Own data modeling, schema mapping, and reconciliation across heterogeneous systems, resolving structural mismatches independently.
• Discovery & Assessment Leadership: Reviewing source documentation, interviewing SMEs, producing gap analyses, and translating ambiguous or outdated specs into concrete technical requirements.
• Data Quality & Validation: Establish data quality frameworks and automated validation as team standards.
• Stakeholder Communication: Proactively surface risks — documentation gaps, structural mismatches — clearly and early to non-technical stakeholders, representing SQUER with confidence.

Your qualifications

• Must-Haves
  • Migration Track Record: 5+ years in data engineering, with several large-scale legacy-to-cloud/lakehouse migrations led independently.
  • Databricks Depth: Hands‑on mastery of the Databricks ecosystem — Spark, Delta Lake, Unity Catalog, Lakeflow Declarative Pipelines, and Asset Bundles for CI/CD-style deployment.
  • Data Modeling & Reconciliation: Strong command of data modeling and schema mapping across heterogeneous systems, from mainframes to legacy ERPs.
  • Discovery Experience: Experience in discovery/assessment phases end-to-end — interviewing SMEs, identifying gaps between legacy systems and target architecture, turning ambiguity into concrete requirements.
  • Data Quality Rigor: Practical experience with data quality frameworks and automated validation (e.g., Great Expectations, dbt tests, custom reconciliation tooling).
  • Communication & Risk Ownership: Ability to break down complex topics and surface risks early and clearly to both technical and non-technical stakeholders.
  • AI-Assisted Workflows: Comfort using AI/LLM‑assisted tooling in a data pipeline context.

• Nice‑to‑Haves
  • Experience with legacy system archaeology (undocumented COBOL/mainframe systems, EDI, proprietary ERPs)
  • Exposure to agentic AI
  • Metadata management or data catalog tooling beyond Unity Catalog
  • Experience building or evaluating LLM‑based mapping/matching tools (schema matching, entity resolution)

Your Benefits

Why You’ll Love Working at SQUER:

• Ownership & Responsibility: at SQUER, we believe in empowering our team. In our flat hierarchies, your ideas and actions matter more than titles. Take the reins, make an impact and grow with us.
• Learn & Grow: Tap into a € 5.000 annual training budget to sharpen your skills and take advantage of our in‑house Leadership Academy. Join and be part of our active tech community through meetups, hackathons and training sessions. Get up to 40h/year additional time off for your personal education.
• Exciting Challenges & Real Impact : Dive into diverse projects where your work truly makes a difference – for our clients and for SQUER.
• Flexibility for Performance: Benefit from hybrid work options, flexible hours, workations within the EU, and additional vacation days – giving you the freedom to perform at your best.
• Tailored Tools for Success : Choose your preferred hardware configuration and leverage our suite of AI tools and licenses (Anthropic, OpenAI, and more) to suit your working style.
• We Value You : Beyond perks and benefits, we offer a competitive annual salary to recognize your exceptional contributions.
• Be Part of a Thriving Team Culture: At SQUER, collaboration and connection are at the heart of what we do. Through open communication, shared successes, and meaningful team events – including our annual SQUER offsite – we cultivate a culture where everyone feels valued and inspired to grow together.
• A Perks‑Packed Workplace : Work in a central office near Ostbahnhof and enjoy meal vouchers, EGYM Wellpass membership, a free Deutschland‑Ticket, JobRad, referral bonuses, Corporate Benefits, and Pluxee employee gifts for special occasions.

We believe in transparent and fair compensation. For this role, the annual gross salary ranges between € 81.000 and € 103.000 – the exact placement depends on the seniority level we assess together during the process.