TaskWeaver + Text Generation Web UI

Text Generation Web UI

git clone https://github.com/oobabooga/text-generation-webui
cd text-generation-webui
bash start_macos.sh --api --listen --extensions openai

TheBloke/Orca-2-7B-GGUF
orca-2-7b.Q4_K_M.gguf

TaskWeaver

git clone https://github.com/microsoft/TaskWeaver.git
cd TaskWeaver
conda create -n taskweaver python=3.11
conda activate taskweaver
pip install -r requirements.txt

Config

{
"llm.api_base": "http://0.0.0.0:5000/v1",
"llm.api_key": "null",
"llm.model": "orca2",
"llm.response" : "text"
}