Fundamentals of ML and AI
Generative AI Fundamentals
Generative AI runs on foundation models (FMs): very large models pre-trained on internet-scale data. Instead of training a separate model for each task, one FM can be adapted to many tasks, such as text generation, summarization, information extraction, image generation, chat, and question answering.
FM lifecycle
The lifecycle is iterative:
- Data selection: massive, diverse, mostly unlabeled data, because it's easier to get at scale.
- Pre-training: usually self-supervised learning, where the model generates its own labels from the structure of the data. Continuous pre-training extends the model's knowledge later.
- Optimization: prompt engineering, RAG, or fine-tuning.
- Evaluation: metrics and benchmarks, plus whether the model meets business needs.
- Deployment: integrate the model into applications and APIs.
- Feedback and continuous improvement: monitor, detect bias and drift, and improve.
Amazon Bedrock gives API access to FMs from AI21 Labs, Anthropic, Cohere, Meta, Mistral AI, Stability AI, and Amazon.
Types of generative models
- Large language models (LLMs) are usually built on the transformer architecture.
- Tokens are the units of text the model processes: words, parts of words, or characters.
- Embeddings are vectors (lists of numbers) that represent tokens. Semantically similar tokens have nearby vectors, so "cat" sits close to "kitten".
- Diffusion models learn by adding noise to an image until only noise is left (forward diffusion), then learn to remove it step by step to create a new image (reverse diffusion). Best known for text-to-image.
- Multimodal models take and produce several data types at once, for example an image plus text in and a caption out.
- GANs (generative adversarial networks) pit a generator that creates fake data against a discriminator that tries to tell real from fake.
- VAEs (variational autoencoders) use an encoder to compress data into a latent space and a decoder to generate data from samples of that space.
Optimizing FM output
Ordered from fastest and cheapest to most involved:
| Technique | What it does | Changes model weights? |
|---|---|---|
| Prompt engineering | Designs the instructions and context given to the model | No |
| Retrieval-augmented generation (RAG) | Retrieves relevant documents and adds them to the prompt as context | No |
| Fine-tuning | Supervised training of a pre-trained model on a smaller, task-specific labeled dataset | Yes |