A Simple Overview of The Modern RAG Developer’s Stack

The Retrieval-Augmented Generation (RAG) approach is used in many cutting-edge applications, and a modern RAG stack consists of several components. A high-quality Large Language Model (LLM) is needed, and developers can choose between open models like Llama 3.3 or API-driven models like OpenAI's GPT-4. Frameworks like LangChain, LlamaIndex, and Haystack help glue components together and provide tools for tasks like search and calculation. Vector databases like Chroma, Qdrant, and Weaviate are used to store chunked knowledge and enable fast similarity search. Data extraction involves ingesting knowledge from diverse sources, including web scraping, document parsing, and APIs, and is typically automated using workflow tools. LLM access layers like Open LLM Hosts and Cloud Providers help decouple code from specific providers. Text embeddings like Sentence-BERT, BGE, and OpenAI Embeddings are used to enable retrieval, and their quality is evaluated using metrics like recall@k and mrr. Evaluation is crucial, and tools like RAGas, Giskard, and TruLens help measure metrics like relevance, accuracy, and cost. A visual overview of the stack shows how these components interact, and developers can use this stack to build high-performance AI applications.