Postdoctoral Research Fellowship in AI Metacognition and Reasoning
Wisdom & Culture Lab, University of Waterloo
How can advanced AI systems better recognize when their reasoning may be wrong, their confidence is unwarranted, or further reflection is needed?
The Wisdom & Culture Lab at the University of Waterloo, directed by Professor Igor Grossmann, is seeking a postdoctoral fellow for a new interdisciplinary research program on metacognition in advanced AI systems.
The project sits at the intersection of AI, metacognition, reasoning, forecasting, and decision science. We are interested in pushing the field beyond whether AI systems can generate accurate answers, toward whether and how to monitor and regulate their inferences when evidence is incomplete and uncertainty is substantial. Decades of research on human metacognition, calibration, and judgment under uncertainty offer theories and measures that AI evaluation largely lacks. A central aim of this position is to bring that science to bear on AI reasoning.
The work builds on emerging research on machine metacognition and wise reasoning and will combine contemporary language models with insights from cognitive science, forecasting, and uncertainty-aware decision making. The project is designed to produce both fundamental scientific contributions and reusable research infrastructure.
The fellow will lead research on reasoning and metacognition in frontier and open-source language models, translate constructs and paradigms from cognitive science and decision science into rigorous evaluations of reasoning under uncertainty, and help shape a broader research program on machine metacognition. The position includes dedicated research-engineering support (infrastructure, model access, and large-scale runs), so the fellow can focus on scientific design. It also offers substantial scope for intellectual ownership (Waterloo policies allow scientists to retain copyright), publications, open-source research, and related independent work.
Responsibilities
-
Lead research on metacognition and reasoning in advanced AI systems
-
Translate constructs and methods from human metacognition and JDM research (e.g., calibration vs. resolution, confidence-accuracy relations, strategy selection) into model evaluations
-
Develop new experimental paradigms, benchmarks, and evaluation approaches
-
Design and run large-scale experiments across different language models and model families
-
Explore approaches for improving how models select and use reasoning strategies.
-
Build reproducible research pipelines, with research-engineering support
-
Lead and co-author papers, conference submissions, and open-source research outputs
-
Collaborate with researchers and trainees spanning AI, psychology, forecasting, and computational social science
Qualifications
Applicants should have completed, or expect to complete before starting, a PhD in cognitive science, psychology, judgment and decision making, computer science, natural language processing, machine learning, computational social science, HCI, or a related field.
Competitive candidates will have several of the following:
-
Research training in metacognition, reasoning, judgment under uncertainty, forecasting, or decision science
-
Solid Python programming skills (strong R users willing to work in Python are welcome)
-
Experience working with contemporary LLMs (e.g., running studies or analyses through APIs); experience with open-weight models is a plus
-
Experience with AI evaluation, benchmarking, computational experimentation, or model comparison
-
Strong quantitative and statistical skills
-
Familiarity with, or interest in learning, methods for prompting, adapting, or orchestrating language models
-
Experience with robustness testing, ablation or manipulation-check logic, multiverse analysis, or reproducible research
-
Strong scientific writing skills
Expertise in every area above is not required. Exceptional computational researchers interested in the underlying scientific questions are encouraged to apply even without a traditional psychology or forecasting background. Equally, psychologists and decision scientists with solid programming skills and curiosity about AI are encouraged to apply without a computer science background.
Employment details
-
Full-time postdoctoral fellowship, initially 12 months with anticipated renewal for a second year, subject to funding and satisfactory performance.
-
Based at the University of Waterloo; hybrid arrangements may be possible with regular interaction with the Waterloo research team.
-
Salary CAD $65,000-$70,000/year + benefits
-
Start date: immediately/Winter-Spring 2027, with some flexibility.
Application
Please submit a cover letter, CV, 2-3 page research statement, up to three representative publications/preprints, contact information for three referees, and optionally a link to code you have written (e.g., GitHub, OSF).
In your cover letter, briefly describe an AI evaluation or modeling problem where an apparently sensible performance metric could lead to a misleading conclusion, and how you would investigate it. Examples from human judgment research that you would extend to AI are welcome.
Applications will be reviewed beginning Nov 15 and continue until the position is filled. Send materials to waclab@uwaterloo.ca, subject line: AI Metacognition Postdoc. Questions: Igor Grossmann, igrossma@uwaterloo.ca