This is a submission for the The Pinata Challenge

What I Built

I developed an AI Chat Assistant platform that leverages advanced semantic search to analyze and retrieve relevant data from a diverse content. The platform intelligently processes user prompts through the LangGraph tool, which determines when to invoke the RAG (retrieval-augmented generation) tool only when necessary, optimizing performance. This ensures that relevant references are efficiently retrieved to enhance the accuracy and relevance of the language model's responses.

The system supports multiple input formats, including images, YouTube links, and PDFs. For videos, it retrieves metadata via the YouTube API and generates concise summaries using a large language model (LLM). Similarly, it summarizes content from images, providing a comprehensive and versatile user experience across different media types.

Additionally, the platform integrates Stable Diffusion to generate images based on textual prompts. These generated images are stored in Pinata (IPFS). The platform also maintains a complete chat history, ensuring smooth continuity in conversations and allowing users to reference previous interactions effortlessly.

LangGraph Flow:

• Advanced Semantic Search:

  • Efficiently analyzes and retrieves relevant data from a vector store.
  • Utilizes the LangGraph tool to optimize workflows by invoking the RAG tool only when necessary.

• Multi-Format Input Support:

  • Processes images, YouTube links, PDFs, and text files.
  • Retrieves video metadata via the YouTube API and generates concise summaries using LLMs.
  • Summarizes content from images for seamless multi-media analysis.

• Image Generation:

  • Integrates Stable Diffusion to generate images based on user-provided text prompts.
  • Stores generated images using Pinata (IPFS) for decentralized access.

• Comprehensive Chat History:

  • Allows users to select which chats to store and easily reference previous interactions.

• Optimized Performance:

  • LangGraph tool ensures that RAG is only called when necessary, enhancing response relevance and efficiency.

Demo

Demo Video: Link

My Code

More Details

This platform integrates Stable Diffusion to generate images based on user textual prompts. The generated images are stored in Pinata (IPFS), which also maintains the chat history, it utilizes CDN for efficient access to the images.

TechStack used:

• NextJS
• Langchain
• LangGraph
• Pinata
• Huggingface API
• Azure OpenAI LLM
• Pinecone
• Firebase
• Redux