AI Quality Engineer (f/m/d)

Über Uns

Do you want to take ownership from day one and actively shape the future in a growing company? VoiceLine is a fast‑growing, VC‑backed startup based in Munich. We build an AI‑powered solution that not only helps field sales and service teams turn conversations into structured data efficiently, but also helps them be smarter on the road. This enables customer information to be documented, analyzed, and seamlessly integrated into existing systems. Together with leading enterprise customers, we are working to make field sales future ready.

The Role

As our first AI Quality Engineer, you will own the feedback loop that tells us how well our AI is actually performing. That means building the systems to detect, measure, and surface failures, across report quality, assistant behaviour, and pipeline reliability, so the rest of the team can act on them with confidence.

What You Will Work On

• Build automated assessment pipelines that score generated reports for completeness and accuracy, combining output diffs against user submissions, and LLM-based scoring
• Instrument the pipeline end-to-end: LLM gateways, traces, dashboards, and observability tools that surface regressions in specific stages
• Analyse assistant conversations to identify and cluster recurring failure modes, missed intent, wrong field extraction, off‑track dialogue
• Build and maintain E2E test harnesses covering the full voice to report flow
• contribute to feature work when quality systems are stable

Dein Profil

• Solid experience serving LLM-powered products in production
• Hands‑on with observability tooling, Langfuse, LiteLLM, or similar
• Analytical instinct: you spot patterns in failures and translate them into actionable metrics
• Self‑directed, comfortable defining your own scope in a small, fast‑moving team
• German language skills - reviewing German-language conversations - is a plus

Deine Benefits

• A dynamic environment with flexible working hours and a strong team culture
• Competitive and fair compensation package, including virtual equity participation
• Modern office in the heart of Munich and the flexibility to work remotely
• Access to Wellpass and attractive sports and wellness offerings to support your well-being
• Support for public transport (Deutschlandticket)