Senior AI Infrastructure Engineer (Europe based - Remote)

Overview

Sword Health is shifting healthcare from human-first to AI-first through its AI Care platform, making world-class healthcare available anytime, anywhere, while significantly reducing costs for payers, self-insured employers, national health systems, and other healthcare organizations. Sword began by reinventing pain care with AI at its core and has since expanded into women’s health, movement health, and mental health. Since 2020, more than 700,000 members across three continents have completed 10 million AI sessions, helping Sword's 1,000+ enterprise clients avoid over $1 billion in unnecessary healthcare costs. Sword Health has raised more than $500 million from investors and is backed by 42 clinical studies and over 44 patents. Learn more at

What you’ll be doing

• Design, build, and maintain the inference infrastructure that powers Sword Health's AI products, ensuring models are served with high throughput, low latency, and cost efficiency.
• Own the end-to-end deployment pipeline for AI models—from real-time computer vision powering movement analysis to large language models driving conversational AI experiences.
• Architect and scale Kubernetes clusters for GPU-accelerated workloads, including autoscaling strategies, resource scheduling, and multi-model serving.
• Build and operate the infrastructure behind Sword Health's real-time AI agents, including WebRTC cluster provisioning and deploying speech-to-text and text-to-speech capabilities at low latency.
• Drive inference scaling strategies by evaluating and implementing techniques such as speculative decoding, continuous batching, and model parallelism to meet growing demand with cost efficiency.
• Develop and maintain Infrastructure as Code (Terraform) and GitOps workflows tailored to GPU-enabled, AI-specific environments.
• Instrument and monitor AI inference systems, building observability around GPU utilization, model latency, throughput, and error rates to ensure reliability and performance.
• Collaborate with ML Engineers, Data Scientists, and Product teams to translate model requirements into robust, production-ready infrastructure.
• Evaluate emerging AI infrastructure tools, frameworks, and hardware to keep Sword Health at the cutting edge of inference performance and efficiency.
• Mentor team members on AI infrastructure best practices, fostering knowledge sharing around GPU workloads, model serving patterns, and production ML systems.

What you need to have

• 5+ years of experience in infrastructure engineering, with at least 2 years focused on AI/ML workloads in production environments.
• Strong experience with Kubernetes for orchestrating GPU-accelerated workloads, including scheduling, resource management, and autoscaling for inference services.
• Hands-on experience with model serving and inference optimization frameworks for real-time computer vision and large language model workloads.
• Solid understanding of LLM inference optimization techniques, including speculative decoding, batching strategies, quantization, and inference scaling patterns.
• Experience provisioning and managing infrastructure for real-time AI systems, including WebRTC clusters and AI agent architectures.
• Familiarity with real-time video/computer vision inference pipelines and the infrastructure challenges of processing continuous visual data streams at low latency.
• Familiarity with speech-to-text and text-to-speech serving infrastructure and the challenges of running voice AI at low latency.
• Experience with Infrastructure as Code (Terraform or similar) and GitOps methodologies for managing complex, GPU-enabled environments.
• Working knowledge of GPU infrastructure - NVIDIA CUDA ecosystem, multi-GPU setups, and GPU monitoring/profiling.
• Strong Linux systems fundamentals and networking knowledge, particularly for latency-sensitive, real-time workloads.
• Fluent in English (written and oral).
• A proactive, ownership-driven mindset — you fix bottlenecks in an inference pipeline before they become problems.

What we would love to see

• AI Inference & Model Serving:
  • Experience with LLM serving engines such as vLLM, SGLang, or LLM-D.
  • Experience with NVIDIA Triton Inference Server and TensorRT for real-time computer vision workloads.
  • Familiarity with NVIDIA Riva or similar platforms for STT/TTS serving.
  • Understanding of speculative decoding, continuous batching, quantization, and model parallelism techniques.
• Kubernetes & Infrastructure:
  • Experience with Istio or similar service mesh.
  • Experience with Kafka for event streaming.
  • Experience with Prometheus, AlertManager, and Grafana for monitoring and observability.
  • Experience with Elasticsearch, Logstash, and Kibana (ELK) for log management.
  • Experience with Vault for secrets management.
  • Experience with Redis, MySQL, and DNS management.
  • Experience provisioning infrastructure on AWS, Azure, or GCP.
  • Good knowledge of cloud networking including VPC management, routing, NAT, and troubleshooting with tools like TCPdump.
• General:
  • Experience with WebRTC infrastructure and real-time media streaming.
  • Experience with Python, Go, or similar languages commonly used in ML infrastructure tooling.
  • Familiarity with SCRUM methodology.

Compensation and benefits

€66,500 - €104,500 a year.

This range includes base, variable, and equity. These compensation bands are starting points and may be adjusted based on impact and other factors.

US and Portugal benefits and perks are listed below. US applicants must have the legal right to work in the United States. This position does not offer relocation assistance. Sword Health complies with applicable civil rights laws and does not discriminate on the basis of age, ancestry, color, citizenship, gender, gender identity, medical condition, national origin, disability, race, religion, sexual orientation, or veteran status.

US - Sword Benefits & Perks

• Comprehensive health, dental and vision insurance
• Equity shares
• Discretionary PTO plan
• Parental leave
• 401(k)
• Flexible working hours
• Remote-first company
• Paid company holidays
• Free digital therapist for you and your family
• Eligibility: Full-time employees regularly working 25+ hours per week

Portugal - Sword Benefits & Perks

• Health, dental and vision insurance
• Meal allowance
• Equity shares
• Remote work allowance
• Flexible working hours
• Work from home
• Discretionary vacation
• Snacks and beverages
• English class
• Note: Eligibility and work rights vary by region

Sword Health complies with applicable civil rights laws and does not discriminate on several bases including age, gender, disability, race, religion, sexual orientation, and veteran status.