Essentials of prompt engineering
Modifying Prompts
Inference parameters
Which parameters are available depends on the model.
Randomness and diversity
| Parameter | What it does | Lower value | Higher value |
|---|---|---|---|
| Temperature | Reshapes the probability distribution of the next token | More deterministic, focused, repeatable | More diverse, creative, random |
| Top P | Samples only from the smallest set of tokens whose probabilities add up to P | Only the most likely tokens | A wider range of tokens |
| Top K | Samples only from the K most probable tokens | Focused, coherent (for example K = 10) | More varied (for example K = 500) |
Length
- Maximum length: caps the number of generated tokens to avoid runaway output.
- Stop sequences: tokens that end generation as soon as they appear, whatever the maximum length.
Prompting best practices
- Be clear and concise, in natural language.
- Include context, such as what the output is for.
- Use directives for the response type: summary, full sentence, list, length.
- Put the requested output at the end of the prompt.
- Start with a question: who, what, where, when, why, how.
- Give an example response in brackets.
- Break up complex tasks into subtasks or separate prompts, or ask the model to think step by step.
- Experiment with different prompts.
- Use prompt templates for consistent, reliable inputs.
Scenario update
Temperature 0.9 and top p 0.999 for more creative output, maximum length 5,000, the finance industry and SMB audience as context, and a list of report sections as the directive.