Combining RL with LLMs in Games

Description

In this group project, supervised by EA SEED, we were tasked to explore how to combine Reinforcement Learning (RL) with Large Language Models (LLMs) in the context of game development. The goal was to create a system for heirarchical game agents, where RL policies would be switched through the judgement of an LLM. Our implementation was done in Unity, where we created a simple 3D environment and trained RL models with specific behaviours such as combat and resource gathering. We then created a system where an LLM would decide which RL policy to use based on the current game state. For evaluation, we compared our agent against an end-to-end RL agent and wrote an ablation study on it. This project was conducted as a part of a course at KTH, the report can be found below.

Tech Stack

Report