Build Your Own Open Source Copilot with Pieces

Pieces 🌟 - Feb 6 - - Dev Community

Build your own Open-source Copilot with Pieces.

As of last year and into the beginning of 2024, copilot functionality and access have increased 10-fold as their usage has become more prevalent across all the applications that we interact with in our workflow. As each copilot emerged, we at Pieces noticed a trend of many different copilots belonging to specific tools, leading to tedious copying and pasting between copilots to maintain a unified understanding of our workflow, which adds to the costs of context switching.

Suddenly, we see a world without any tools that would allow users to consume Local Large Language Models like Mistral and Llama 2, as well as cloud LLMs like GPT-4 and Gemini, all from the same location. Since it’s often unclear which is the best LLM for coding, we wanted to give users options. Pieces as an ecosystem works in different plugins (like VS Code, Obsidian, JetBrains, Chrome, and more) all supported by Pieces OS, which handles the logic and language models that you consume in Pieces Desktop or the plugins that connect to it.

After developing our proprietary Pieces Copilot, we decided to support an open-source copilot that developers can build on their own to leverage various LLMs, set custom context for more personalized responses, and integrate with their favorite tools. This guide will help you do just that, serving as an open-source alternative to GitHub Copilot and other AI tools.

What is Pieces OS Client?

Pieces OS Client is a flexible database that provides a custom open-source copilot for saving, generating, and iterating on code snippets across your development workflow. It comes with built-in Local Large Language Models that can be used to ask questions and generate powerful code solutions no matter what dev tool you’re working in.

Since Pieces OS contains a large amount of useful tooling, our first step was to create a number of ways for developers to access and use the SDKs. In order to meet that need, we have taken all of the functionality that has been added to Pieces OS over the last two years and bundled it up in different language packages:

What Can My Open Source Copilot Do?

Here is a high-level list of some of the features you have access to when building your copilot open source:

  1. Switch between the following Large Language Models for your open-source copilot conversations. You can download these models on-device and use them without internet connection, or you can consume them in the cloud:
  • Local: Mistral AI (CPU + GPU)
  • Local: Llama 2 (CPU + GPU)
  • Cloud: GPT-3.5 Turbo + GPT-3.5 Turbo 16k + GPT-4 (you can also use your own OpenAI API key)
  • Cloud: Gemini
  • Cloud: Chat Bison + Code Chat Bison with Palm 2

  1. Add context to your conversations and questions to give your open-source code copilot additional seeded information to answer your questions without having to train a model on your data

  2. Ask questions about a specific code snippet that you have saved in the past

You can perform several other actions through the Pieces OS Client and can discover the other endpoints and SDKs through the Open Source by Pieces Repo.

Building your Open Source Copilot

When first using the SDK, it can be overwhelming to try to understand all of the classes available to you, so here is a brief introduction to some of the more powerful endpoints that are related to creating your own open-source copilot.

Universal Steps

  1. Download the SDK in the language that you prefer— this guide follows the Typescript Example using WebSockets and this repo:
  1. Be sure to install Pieces OS and get it running on your machine

Part 1: Creating Your Open Source Copilot

Adding the needed functionality can be super easy here, as you can create your first message without connecting or registering your application. If you want to send a single message and get back a response, you can do so with little to no configuration or data added:

CopilotStreamController.getInstance().askQGPT({
    query: userInput,
    setMessage
});
Enter fullscreen mode Exit fullscreen mode

Save this Snippet

You can see some of the configurations you will need to send the message in the CopilotStreamController on .askQGPT, but you can use the Pieces.QGPTStreamInput to send the message:

// @query == userInput from the index.ts
const input: Pieces.QGPTStreamInput = {
      question: {
        query,
        relevant: {iterable: []} 
    },
};
Enter fullscreen mode Exit fullscreen mode

Save this Snippet

You can read more about handleMessages, updating the UI, and generating the response from the full article Build Your Own Copilot in 10 Minutes.

Part 2: Downloading Local Large Language Models like Mistral AI

At the beginning of the article, we mentioned all of the different models that are accessible through Pieces OS Client and that one of the benefits of using the Client for LLLM management is being able to control downloading, deleting, and using models interchangeably.

In a few steps, you can access the model’s snapshot and select the model you want, along with all of the model.foundation parameters that are needed for identifying the model during the download process, or the readable name of the model. Here what that could look like in a file like modelsProgressController.tsx as the constructor:

public models: Promise<Pieces.Models>;

private constructor() {
    // the models snapshot 
    this.models = new Pieces.ModelsApi().modelsSnapshot();

    // filtering off our default model here:
    this.models.then((models) => {
          this.initSockets(
          // set intial model download value.
            models.iterable.filter(
              (el) =>
                el.foundation === Pieces.ModelFoundationEnum.Llama27B &&
                el.unique !== 'llama-2-7b-chat.ggmlv3.q4_K_M'
            )
          );
    });
}
Enter fullscreen mode Exit fullscreen mode

Save this Snippet

Then you could properly set, select, and use the models in your main() entry point to use later when passing the model.id into the ModelApi().modelSpecificModelDownload endpoint like the following. If we were going to first get the model information and then set the selected model for download, it would look like this:

// define your model values by accessing it from the .iterable on models
const mistralcpu = models.iterable.find((model) =>  model.foundation === ModelFoundationEnum.Mistral7B && model.cpu);

// starts the model download on click.
downloadMistralCpuButton.onclick = (e) => {
    new ModelApi().modelSpecificModelDownload({model: mistralcpu!.id})
}
Enter fullscreen mode Exit fullscreen mode

Save this Snippet

Then after the model has been downloaded, you can pass your model ID into the QGPTStreamInput now including the model parameter:

const input: Pieces.QGPTStreamInput = {
  question: {
     query,
     relevant: {iterable: []},
     // the updated parameter here:
     model: mistralcpu!.id
  },
};
Enter fullscreen mode Exit fullscreen mode

Save this Snippet

In this simple example, you could select the model, download it, and then use the model in your next query. For a more complete example, see the blog How to Build a Copilot Using Local LLMs to understand more about connecting the UI to these states and showing download progress.

Part 3: Attaching Context to Your Open Source Copilot Queries for Better Responses

In most copilot environments, you cannot attach context to your models without training them on the knowledge base to answer your questions first. With the Pieces OS Client, attaching context is straightforward and does not require waiting for a model to train. This allows for quick context swapping on a large scale from project to project, or on something as small as two separate lines of code.

To add files, you would need to add a file picker package to the project. But to speed up the process as developers are using this SDK, Pieces OS Client comes with a native OS file picker built in for handling the paths that are selected here.

You can use that file picker like so:

// opens the filepicker and returns Array<String> filepaths back.
new OSApi().pickFiles({filePickerInput: {}});
const files = [];

addFilesAsContext.onclick = () => {
    new Pieces.OSApi().pickFiles({filePickerInput: {}}).then((files) => {
            files.forEach((file) => {
            // Do anything to @file == string 
            })
    })
}

// then use relevanceInput to add your selected context as @paths:
const relevanceInput: Pieces.RelevanceRequest = {
      qGPTRelevanceInput: {
        query,
        paths: files,
    }
}
Enter fullscreen mode Exit fullscreen mode

Save this Snippet

Now you have created your relevance pipeline to let the user select and send over the first half of the contextual message. But to use context, we have to seed an asset, which requires the application to be created. Here is an example of creating an application and getting all of its values:

let application: Pieces.Application;

export async function getApplication() {
    if (application) return application;

    // PlatformEnum corresponds to the current operating system.
    const platform: PlatformEnum = window.navigator.userAgent.toLowerCase().includes('linux') ? PlatformEnum.Linux : window.navigator.userAgent.toLowerCase().includes('win') ? Pieces.PlatformEnum.Windows : Pieces.PlatformEnum.Macos;

    // Creating the Application Here, and setting up parameters
    let context: Context = await new Pieces.ConnectorApi().connect({
        seededConnectorConnection: {
            application: {
                name: Pieces.ApplicationNameEnum.OpenSource,
                version: '0.0.0',
                platform,
            }
        }
    });

application = context.application

}

// Once Context is created and configured for our application
// we can use application in our seed on relevanceInput.
// create your relevance input here:
relevanceInput.qGPTRelevanceInput.seeds = {
    iterable: [
      {
        // the type of relevance input that is being used.
        type: SeedTypeEnum.Asset,
        asset: {
            // the application we created and regisetered.
            application,
            format: {
                fragment: {
                string: {
                    // string value that the user supplied.
                    raw: userContextInput
                  }
              }
            }
        }
      }
    ]
  }
}

// creates the relevanceOutput that is needed for the QGPTStreamInput.
const relevanceOutput = await new QGPTApi().relevance(relevanceInput);
Enter fullscreen mode Exit fullscreen mode

Save this Snippet

Then the final step is to add that relevant input as the relevant parameter on question:

const input: Pieces.QGPTStreamInput = {
    question: {
        query,

        // the relevance:
        relevant: relevanceOutput.relevant,
        model: mistralcpu!.id
    },
};
Enter fullscreen mode Exit fullscreen mode

Save this Snippet

To see a full breakdown with more code examples and explanations, check out the article on Adding Context to your Local Copilot and try it out for yourself!

Conclusion

Building your very own open-source copilot with Pieces OS Client and the available SDKs should be an easy process; there are notes and code snippets throughout this article series that give a full view of how these key endpoints work and more. Check out all three of the articles here:

Keep your eye out for more articles on using the open-source code copilot, Local Large Language Models, and other Pieces Copilot functionality that can be found in Pieces OS Client. If you want to get more involved, check out our Open Source program on GitHub. If you’re interested in Pieces for Developers and want to learn more, check out our documentation or our desktop app to give it a try.

Image
Building an Investment Calculator in Angular with Tailwind CSS

webdev, beginners, programming, angular
Image
10 Things People Hate About Self Propelled Lightweight Folding Wheelchair
Image
css, 높은 특이도 부여

css
Image
The particular Sinonasal End result Test-22 or perhaps European Place Papers: That is Far more An indication of Image Benefits?
Image
Salesforce Communities: Engaging Customers in 2024

webdev, beginners, programming, salesforcecummunities
Image
Tile Adhesive Solutions
Image
ACC Ltd: Pioneers in High-Quality Cement and Ready-Mix Concrete
Image
Why Should You Include Integration Testing in Software Development Process

integration, testing
Image
Open Source Flutter Project: Map Customization, State Management Live Location Tracking & Sharing.

opensource, flutter, dart, github
Image
c++
Image
The cross-sectional research associated with Local treatments and also the COVID-19 crisis in Iran: Gossip and proposals.
Image
使用SEO來增強持續業務業務的博客廣告實踐 - 【SeoNo1.Ltd】
Image
7 Practical Tips For Making The Greatest Use Of Your Bifold Door Glass Replacement
Image
Pivot Entrance Doors as Architectural Statements
Image
The Rise of Aluminum Sliding Doors in Modern Homes
Image
Amazon S3 Bucket with Policy Details

webdev, aws, beginners, programming
Image
Migrating from WordPress to Jekyll: save money with a static site

wordpress, jekyll, webdev, php
Image
Discover LA FESTIN: Luxury Handbags & Fashion Accessories

fashion, women, product, bag
Image
How to Use the workflow_dispatch Event for Manual Triggers

devops, security, github, githubactions
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Terabox Video Player