Senior Data Engineer – Agents Systems

Responsibilities

• Own and evolve streaming data pipelines that power live inference and real-time model serving across Kraken's AI infrastructure
• Design and build feature stores that serve low-latency, high-reliability features to production ML models
• Implement and maintain streaming systems using RisingWave, Apache Flink, or Kafka Streams, selecting the right tool for the workload
• Partner with ML engineers and AI infra teams to define data contracts, feature schemas, and pipeline SLAs
• Drive pipelines toward real-time where batch exists today reducing latency from hours to seconds
• Ensure data quality, observability, and auditability across all streaming and feature engineering systems
• Contribute to inference pipeline tooling where data engineering and model serving intersect
• Evaluate emerging streaming and feature store technologies and shape the team's technical roadmap

Requirements

• 5+ years in data engineering with at least 2 years focused on streaming systems in production
• Hands‑on experience with RisingWave, Apache Flink, Kafka Streams, or comparable stream processing frameworks
• Strong understanding of feature store design — online/offline consistency, point‑in‑time correctness, low‑latency serving
• Experience building data pipelines that feed production ML models or inference systems
• Proficiency in Python and/or Scala; SQL fluency required
• Familiarity with data quality frameworks, pipeline observability, and SLA ownership
• Comfortable operating in a fast‑moving, ambiguous environment embedded within an AI‑focused team