Retrieval-Augmented Generation

IntroductionCopy Anchor Link

In the previous lesson, the business case of AnyCompany illustrated how FMs are not enough to provide qualified outputs on their own. In this lesson, you will learn the Retrieval-Augmented Generation (RAG) approach that allows FMs to query knowledge bases to provide accurate and up-to-date answers to customer prompts.

Enterprise datasetsCopy Anchor Link

Although LLMs can generate human-like text, image, audio, and more from prompts, this capability might not meet the specific needs of enterprises. Customized enterprise applications require these models to process relevant data from enterprise datasets.Enterprises gather vast amounts of internal data, including documents, presentations, user manuals, reports, and transaction summaries, all unfamiliar to the FM. When these models ingest and use enterprise data sources, they acquire domain-specific knowledge, enabling them to produce tailored, highly relevant outputs that meet enterprise needs.To provide the relevant enterprise data as additional context to the language model, along with the prompt, this addition helps the model deliver more accurate outputs. Determining the right context involves searching enterprise datasets using the prompt text. Vector embeddings play a crucial role in this process.

Vector embeddingsCopy Anchor Link

Embedding is the process by which text, images, and audio are given numerical representation in a vector space. Embedding is usually performed by a machine learning (ML) model. The following diagram provides more details about embedding.

Enterprise datasets, such as documents, images and audio, are passed to ML models as tokens and are vectorized. These vectors in an n-dimensional space, along with the metadata about them, are stored in purpose-built vector databases for faster retrieval.

You will now take a closer look at the embedding step.

Two words that relate to each other will have similar embeddings.

Here is an example of two words: sea and ocean. They are randomly initialized and their early embeddings are diverse. As the training progresses, their embeddings become more similar because they often appear close to each other and in similar context. For the purpose of this example, embeddings that are close together are represented by colors in the same palette. Therefore, the word sea and ocean use similar colors. However, stapler has a completely different embedding, so it uses a separate set of colors.

Words that relate to each other will have closer embeddings.

Storing vectorsCopy Anchor Link

The core function of vector databases is to compactly store billions of high-dimensional vectors representing words and entities. Vector databases provide ultra-fast similarity searches across these billions of vectors in real time.

The most common algorithms used to perform the similarity search are k-nearest neighbors (k-NN) or cosine similarity.

Amazon Web Services (AWS) offers the following viable vector database options:

• Amazon OpenSearch Service (provisioned)
• Amazon OpenSearch Serverless
• pgvector extension in Amazon Relational Database Service (Amazon RDS) for PostgreSQL
• pgvector extension in Amazon Aurora PostgreSQL-Compatible Edition
• Amazon Kendra

RAG in the context of AnyCompany's business caseCopy Anchor Link

Following is an updated version of AnyCompany's architecture diagram, including the RAG system. The chatbot is now able to query a database containing enterprise data and use it to provide more accurate and contextual answers to users.

Architecture diagram of AnyCompany's system, including a RAG approach for optimizing the model's performances.

You have learned about the fundamentals of RAG. In the next lesson, you will learn about agents and how they are used.