Master Thesis Large Language Models for Procedural Animations in Virtual Environments

SETLabs Research is a Munich-based research center founded in 2021 as a subsidiary of Virtual Vehicle Research GmbH in Graz, Austria. SETLabs partners with academia and industry for joint projects to enable knowledge transfer from basic and applied research into real-world applications. The center has domain-open expertise in modeling and simulation, hardware/software integration, systems engineering, machine learning, and artificial intelligence. The main research is conducted according to technologies within healthcare and mobility sectors.

Master Thesis

“Large Language Models for Procedural Animations in Virtual Environments”

The goal of this thesis is to investigate how large language models (LLMs) can be utilized to generate context-aware procedural behaviors for avatars in human-centric simulation environments. In simulation environments where robots interact with avatars and virtual agents, LLMs can serve as high-level reasoning modules that translate natural-language context into structured action plans. These plans govern how virtual agents behave, interact, and adapt to dynamic environments, supporting workflows such as task coordination, object manipulation, or situational response.

Traditional systems rely on manually scripted rules or stochastic animation graphs, which are often rigid and difficult to scale. This thesis explores whether LLM-guided high-level planning can provide a more flexible, semantically grounded, and scalable alternative. Specifically, the work aims to:

• Develop methods for using LLMs to produce symbolic behavior plans that describe what an avatar should do and how (e.g., action type, speed, style, timing).
• Integrate these symbolic plans with a motion execution layer in Unity that ensures physical fidelity, safety, and reproducibility.
• Evaluate the feasibility, benefits, and limitations of this LLM-driven procedural control approach across a small set of representative simulation scenarios.

The expected outcome is a validated prototype pipeline demonstrating how LLM-based behavior generation can improve semantic understanding, adaptability, and autonomy in simulation-based environments.

Your Tasks

Conduct state-of-the-art review of current approaches to procedural animation, and emerging work on LLMs for behavior control, game-content and interactive narrative generation.

Design and implement a hierarchical animation pipeline where an LLM provides high-level symbolic logic (e.g., selecting actions, parameterizing style/timing, triggering interactions).

Integrate the developed pipeline into SETLabs proprietary Unity-based simulator, enabling LLM-guided avatar control in realistic virtual environments.

Evaluate the system across 2–3 scenarios, measuring plan correctness, motion plausibility, and reproducibility.

Document and disseminate findings in the form of a scientific master’s thesis, and potentially a publication in computer graphics or computer vision avenue.

Literature

Park, J. S., O'Brien, J., Cai, C. J., Morris, M. R., Liang, P., & Bernstein, M. S. (2023, October). Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th annual acm symposium on user interface software and technology (pp. 1-22).

Wang, G., Xie, Y., Jiang, Y., Mandlekar, A., Xiao, C., Zhu, Y., ... & Anandkumar, A. (2023). Voyager: An open-ended embodied agent with large language models. arXiv preprint arXiv: .

Your Profile

Student currently enrolled in MSc. in informatics, games engineering, robotics, or related fields.

Strong programming skills in Python (required).

Familiarity with Unity + C# (beneficial).

Interest in large language models, simulation, and animation systems.
Background in machine learning and computer graphics (beneficial).

Our Offer

Opportunity to contribute to cutting-edge international research projects with a broad partner network.

Collaboration within an engaging and dynamic research team.

Flexible working hours and the option to work remotely.

Paid master’s thesis and potential mini-job employment.

Participation in scientific conferences.

Personal and professional development opportunities across career levels.

#J-18808-Ljbffr