Manager, LLM Accuracy Evaluation

Overview

We are looking for a visionary Manager to lead a team of extraordinary engineers pioneering new methodologies for evaluating the capabilities of next-generation AI models—spanning LLMs, RAG systems, agents, and vision models. As a manager, you will play a critical role in driving the development and deployment of the latest flagship models from our community and partners—such as Nemotron, Llama-4, DeepSeek, GPT-4o, and Gemini. This leadership role offers a rare opportunity to help shape the future of AI at a company operating at the very forefront of the AI revolution. You will guide your team to deliver state-of-the-art evaluations and optimized deployments with lightning-fast inference, working on the world’s most powerful GPU clusters and gaining early access to unreleased hardware. Your leadership will directly influence NVIDIA’s roadmap and the broader AI ecosystem, making a lasting industry impact.

Responsibilities

• Lead and mentor a team of highly skilled engineers, fostering their growth while solving the most ambitious challenges in AI evaluation.
• Drive the accuracy evaluation of flagship AI models, coordinating efforts across internal teams and external partners to ensure timely, high-quality results.
• Collaborate with stakeholders across NVIDIA to balance speed of delivery with rigorous engineering practices.
• Develop and implement new methodologies for evaluating LLMs, multimodal systems, and agent frameworks at scale.
• Build a culture of innovation and excellence, encouraging continuous improvement and adoption of best practices in AI evaluation and deployment.

Qualifications

• BS, MS, or PhD in Computer Science, AI, Applied Math, or related field, or equivalent experience, with 7+ years of industry experience, including 3+ years in leadership.
• Proven success leading engineering teams and delivering complex AI/deep learning projects.
• Deep understanding of modern AI technologies—LLMs, multimodal models, retrieval-augmented generation, and agent frameworks—with the ability to guide technical strategy.
• Outstanding communication skills and the ability to partner effectively across organizations and with external collaborators.
• Demonstrated ability to mentor and grow engineering talent, fostering collaboration and technical excellence.

Standout Qualifications / Desired Experience

• Experience managing teams that shipped AI products or services using LLMs, RAG, or multimodal/agent models.
• Hands-on expertise in deploying and optimizing AI models in production, with platforms such as TensorRT, Triton, or ONNX.
• Strong background in MLOps/DevOps, with a focus on scaling deep learning workloads.
• Proven ability to manage large-scale AI evaluations and training workloads on HPC clusters, ensuring efficiency and reproducibility.
• Deep understanding of cloud infrastructure, containerization (Docker), and orchestration (Kubernetes), with an emphasis on scalability and reliability.

NVIDIA is committed to fostering a diverse work environment and proud to be an equal opportunity employer. We do not discriminate on the basis of race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status or any other characteristic protected by law.