data engineer for enterprise data platforms

Описание

Exadel is an AI-first global technology company with over 25 years of engineering leadership, more than 2,000 team members, and over 500 active projects for Fortune 500 clients. It builds AI platforms and supports enterprise digital transformation.

Задачи

• Develop and maintain Bronze, Silver, and Gold data pipelines using SQL-based ELT patterns;
• Implement data ingestion from SFTP, APIs, and databases into BigQuery;
• Build and optimize transformation logic for the curated Silver Mart layer using dimensional modeling principles;
• Create fact and dimension tables, including SCD handling and surrogate key logic;
• Implement data quality checks and integrate them with pipeline execution;
• Support Airflow DAG development for pipeline orchestration, scheduling, and monitoring;
• Enable reprocessing and idempotency so pipelines can be reliably rerun from Bronze;
• Collaborate on schema evolution and deployments using CI/CD tools;
• Optimize BigQuery performance and cost through partitioning, clustering, and query tuning;
• Implement and maintain data integrations with downstream systems such as ERP and reporting tools;
• Contribute to metadata, lineage, and documentation in the data catalog;
• Support testing, reconciliation, and parallel-run validation during migration from legacy systems;
• Troubleshoot pipeline failures and support monitoring and alerting processes.

Требования

• Strong experience with Google BigQuery and GCP;
• Hands-on expertise with Apache Airflow, especially Cloud Composer;
• Experience with BigQuery, AlloyDB / PostgreSQL, and Teradata;
• Proven experience with SQL-based ELT, SFTP, and API integrations, including NetSuite and PSPs;
• Knowledge of data governance frameworks and OpenMetadata, including lineage and catalog;
• Understanding of IAM/RBAC;
• Strong understanding of CI/CD and DevOps practices using GitHub, Jenkins, Flyway, and Terraform;
• Experience building and maintaining metadata-driven data quality frameworks with BigQuery and Airflow;
• Upper-Intermediate English.

Условия

• In-office, hybrid, or remote flexibility;
• Medical healthcare;
• Recognition program;
• Ongoing learning and reimbursement;
• Well-being program;
• Team events and local benefits;
• Sports compensation;
• Referral bonuses;
• Top-tier equipment provision;
• International projects.