Senior Machine Learning Engineer (IntelliJ AI)

Overview

Join JetBrains as a Senior Machine Learning Engineer (IntelliJ AI). We develop AI features that accelerate developers inside JetBrains IDEs.

Responsibilities

• Design and drive evaluation frameworks for AI features, including metrics, experiments, and trace analysis.
• Diagnose model performance issues (prompt drift, context mismatches, latency/quality trade-offs) and translate findings into actionable improvements.
• Experiment with lightweight models to continuously develop our ML system.
• Act as the ML liaison for product teams across JetBrains, adapting and scaling AI capabilities to their needs.
• Build and maintain small helper models (re‑rankers, classifiers, embedding models) to support domain‑specific tasks.
• Collaborate with colleagues in ML, product, engineering, and analytics teams to deliver improvements and monitor their impact in production.
• Stay up to date with research in LLMs, agents, and evaluation, bringing best practices into our workflows.
• Mentor junior engineers and help shape team culture, processes, and tooling around experimentation and evaluation.

Qualifications

• 5+ years of experience as an ML Engineer with production‑grade ML systems (especially LLMs and agent architectures).
• Experience with LLM evaluation methods and frameworks.
• Design and run end‑to‑end experiments – hypotheses, metrics, data collection, analysis, and decision making.
• Skilled in context‑aware pipelines or conversational/agent systems.
• Strong Python programming skills.
• Hands‑on experience fine‑tuning or training smaller models (domain‑specific fine‑tuning, lightweight customizations).
• Communicate clearly and effectively across teams, translating ML/AI insights into product features.
• Prior mentorship experience with ML/evaluation engineers.
• Thrive in a cross‑functional, fast‑moving environment, taking ownership, iterating quickly, and delivering results.
• Familiarity with agent‑based systems and orchestrating multi‑step reasoning agents (optional).
• Experience with the Kotlin programming language (optional).

Seniority Level

Mid‑Senior level

Employment Type

Full‑time

Job Function

Engineering and Information Technology