Data Engineering Lead

Direct message the job poster from Eagle Search

Eagle Search is delighted to be partnered with a fast-growing, European energy technology firm using AI and data-driven automation to transform how large-scale energy assets are operated and traded. Backed by strong leadership and recent investment, the company is building an AI‑first data platform at the core of its business.

The Role

We are looking for a Data Engineering Lead to manage our client's data team, and play a key role in designing and scaling their data platform - powering everything from analytics to real‑time trading decisions. This is a hands‑on leadership role, combining architectural vision, team management, and technical delivery to ensure reliable, high‑quality data for ML and trading systems.

Key Responsibilities

• Design and maintain scalable, cost‑efficient data models, storage, and processing systems (BigQuery, Cloud Storage, etc.).
• Communicate with external and internal data providers and ensure smooth data onboarding.
• Ensure data accessibility and usability across the organisation via well‑structured repositories and APIs.
• Develop, maintain, and optimise automated pipelines from multiple data sources (APIs, sFTP, web scraping, etc.).
• Implement robust ETL/ELT workflows for both real‑time and batch processing using Airflow or similar tools.
• Ensure data freshness, reliability, and completeness across the full data lifecycle.

Data Quality & Observability

• Build monitoring, validation, and alerting systems to ensure high data quality and integrity.
• Develop dashboards and internal data products that empower analytics, ML, and trading teams.
• Continuously analyse data to identify inconsistencies, anomalies, and optimisation potential.
• Partner with ML and trading teams to deliver reliable datasets and feature pipelines for predictive and decision models.
• Collaborate on deployment, model feedback loops, and real‑time data interfaces.
• Contribute to the design and scaling of the cloud‑based data infrastructure.
• Lead and mentor a team of data engineers, fostering a high‑performance and learning‑oriented culture.
• Define and execute the technical roadmap for the data platform, aligning with company and product strategy.
• Oversee day‑to‑day operations, ensuring delivery reliability, cost efficiency, and long‑term maintainability.

Required Technical Skills

• Expert in Python (pandas, dask) and SQL; strong background in data engineering and architecture.
• Experience building data infrastructure on Google Cloud Platform (BigQuery, Cloud Storage, Pub/Sub, etc.).
• Knowledge of ETL/ELT pipelines, workflow orchestration, and DataOps/CI‑CD practices.
• Practical experience with SQL and NoSQL databases (e.g., PostgreSQL, MongoDB), including schema design and optimisation.
• Familiarity with CI/CD pipelines, infrastructure as code (Terraform, GitHub Actions), and modern data ops practices.
• Proficiency with Git and collaborative development workflows.
• Strong grasp of data quality, observability, and cost optimisation principles in a production context.

Required Experience:

• You hold a degree in Computer Science, Information Systems Management (Wirtschaftsinformatik), Environmental Informatics, Renewable Energy, Mathematics, Industrial Engineering, or a related field - or bring equivalent practical experience.
• You have proven experience in data engineering or data infrastructure, ideally with exposure to production‑grade, high‑availability systems.
• You combine software engineering excellence with a strong sense for scalable, maintainable architecture.
• You enjoy leading by example - mentoring others, improving processes, and setting technical direction while staying hands‑on.
• You are structured and self‑driven, with the ability to balance strategic priorities and day‑to‑day execution.
• You are a collaborative communicator who thrives in cross‑functional teams (trading, ML, product) and values clarity and transparency.
• You are motivated by impact and purpose, and want to contribute to the decarbonisation of the energy system.

Seniority level

Mid‑Senior level

Employment type

Full-time

Job function

Services for Renewable Energy

Referrals increase your chances of interviewing at Eagle Search by 2x

Get notified about new Senior Data Engineer jobs in Berlin, Germany .

#J-18808-Ljbffr