Open PhD position on cognitively-inspired language modeling

Stellenbeschreibung:

The Human-Centered Data Science group is affiliated with the Institute of Computer Science and the Campus Institute Data Science (CIDAS) at the University of Göttingen. Our research is interdisciplinary at its core, and we cooperate closely with colleagues from other faculties (e.g., psychology, linguistics). We take a human-centered perspective on natural language processing and focus on cross-lingual and cognitively-inspired research questions.

KIND-LM: Cognitively-inspired interaction dynamics for sample-efficient language modeling. Computational models of language can generate remarkably fluent text, but their impressive performance comes at the cost of training on trillions of tokens with unsustainable computational resources. When trained under resource constraints, such models fall short of robust linguistic generalization and often fail to adapt to unseen contexts. Human learners, by contrast, acquire language from vastly smaller input and can flexibly adapt to new communicative situations from an early age. A central difference lies in the learning signal: while human acquisition is embedded in rich social interactions, language models are typically optimized for the narrow task of next-word prediction. This project develops a cognitively grounded approach for interactive language modeling that integrates feedback mechanisms inspired by child–caregiver communication. We propose a training setup in which a child model improves its linguistic competence through interaction with a more powerful parent model. Unlike existing teacher–student approaches, which assume unilateral feedback, we focus on the temporal and linguistic interaction dynamics and on the interaction initiative. We will build on our winning submission to the new interaction track of the BabyLM Challenge, which used a reinforcement loop and showed that even simplified feedback strategies can enhance functional linguistic competence without sacrificing formal accuracy. We propose to better align computational modeling with psycholinguistic evidence and systematically test cognitively more plausible interaction strategies. We will draw on mechanistic interpretability methods to better understand how interaction dynamics influence the representational structure of the model and how they can improve its ability to generalize to the long tail of the vocabulary distribution.

The project is a collaboration between Lisa Beinborn (Professor of Human-Centered Data Science) and Nivedita Mani (Professor of Psychology of Language). It advances research on cognitively-inspired sample-efficient modeling and contributes to the Priority Programme LaSTing (“Robust Assessment & Safe Applicability of Language Modelling: Foundations for a New Field of Language Science & Technology”).

Company

University of Göttingen

Qualifications

In this position, you have the chance to pursue a PhD degree. You are expected to:

conduct innovative research in the context of the project.
collaborate with the project partners and contribute to meetings of the priority programme
communicate research results in peer-reviewed proceedings and journals, and present them at international research conferences.
take an active role in the co-supervision of student theses related to the project.
engage in the activities and events of the research group.

Ideal Candidate

has obtained a very good master’s degree in computer science, cognitive science, computational linguistics, machine learning, or a related discipline.
has gained experience with natural language processing research and demonstrates a strong interest in the psycholinguistic aspects of the project outlined above.
can independently acquire and process new knowledge.
is a team player with good communication skills and an interdisciplinary mindset.
has obtained strong analytical and programming skills and is committed to further developing them (experience with large-scale experiments on GPUs is beneficial).