Tau LLM Unity ML Agents Project

Introduction

Welcome to our Tau LLM Unity ML Agents project! This project aims to leverage the power of Unity's ML-Agents toolkit to create intelligent agents capable of learning and adapting to various tasks. Our focus is on optimizing the training loop, updating the TauAgent and AgentTrainer scripts, and fine-tuning the reward calculation.

Project Overview

Key Components

1. TauAgent: The core agent that interacts with the environment and learns from it.
2. AgentTrainer: Manages the training loop, including starting and ending training episodes.
3. AgentDelegator: An abstract class that handles the initialization and processing of the TauAgent.

Training Loop

The training loop is designed to ensure that the TauAgent learns efficiently. We use a binary reward signal to simplify the reward calculation and focus on key hyperparameters to optimize training.

Data Management

We use embeddings to represent the input and expected output vectors. The Embedding class encapsulates these vectors, ensuring they are correctly sized and formatted. A helper function, AddTrainingData, links the input and expected embeddings, creating a robust training dataset.

Code Snippets

AgentDelegator Class

public abstract class AgentDelegator<T> : MonoBehaviour where T : MonoBehaviour
{
    // Class implementation
}

AgentTrainer Class

public class AgentTrainer : AgentDelegator<AgentTrainer>
{
    // Class implementation
}

Embedding Class

[Serializable]
public class Embedding
{
    public int Id;
    public string Token;
    public List<double> Vector;
    public EmbeddingType Type;

    // Class implementation
}

Future Work

We plan to further optimize the training loop, explore different reward signals, and enhance the AgentEvaluator class for better evaluation metrics. Stay tuned for more updates!

Conclusion

Our Tau LLM Unity ML Agents project is an exciting journey into the world of machine learning and intelligent agents. We are committed to pushing the boundaries of what these agents can achieve. Thank you for following our progress!

Feel free to reach out if you have any questions or suggestions!

Github : https://github.com/p3nGu1nZz/Tau
YouTube : https://www.youtube.com/@p3nGu1nZz