Lead AI OPS Engineer

We are building a self-serve Enterprise AI Gateway that standardizes how every team uses large language models with strong governance. As a Lead AI OPS Engineer , you will design automation-first onboarding, controls, and observability for access, cost, and logging across the platform. Join a small, fast-moving team.

Responsibilities

• Design self-service onboarding for teams and agents, from initial request through to functional access
• Automate key provisioning, rotation, and permission updates
• Tune rate limits and quotas as adoption and usage increase
• Deploy and configure models including Vertex AI endpoints, vendor fallbacks, and Model Armor
• Operate MCP servers behind the gateway under the same governance rules as models
• Develop Python extensions that integrate the gateway with other enterprise systems
• Maintain the full platform as code using Terraform, Helm, and Jenkins
• Monitor and protect production health across availability, latency, and cost with alerting
• Assist teams using the platform and convert recurring questions into automation

Requirements

• Proven 5+ years of experience using Python for automation, extensions, and integrations
• Solid 5+ years of SRE experience with a track record of keeping production systems reliable
• Deep expertise in Kubernetes, preferably on GKE
• Hands-on experience with Google Vertex AI, especially endpoints for model serving and Model Armor
• Practical proficiency with Terraform, Helm, and CI/CD using Jenkins
• Working knowledge of GenAI/Agentic AI concepts (patterns, frameworks, protocols)
• Production exposure to an AI gateway (LiteLLM, EPAM DIAL or similar) is highly appreciated
• Very strong communication skills with the ability to explain platform behavior clearly to teams who use it daily
• English proficiency at B2 level (Upper-Intermediate) or higher
• Nice to have Familiarity with GCP beyond GKE and Vertex AI, such as BigQuery, Cloud Run, and IAM
• Nice to have Experience building AI agents, for example using Google's Agent Development Kit (ADK)
• Nice to have Experience with AWS Bedrock