How Generative AI Works
How Generative AI Works
Follow the journey from large banks of training data to a generated response—and discover how testing, transformers and human feedback improve the system.
Training
Learn patterns from vast data sets.
Testing
Evaluate with controlled, unseen data.
Reinforcement
Learn from perceived output quality.
RLHF
Use human checks and rewards.
Inferencing
Respond to new data and prompts.
Stages of the Generative AI Process
A useful generative AI system is not created in a single step. It begins by learning from large quantities of data, is evaluated with information it has not previously seen, and is then improved through feedback. Only after the model has been trained and tested is it ready to receive new prompts and generate responses. Select each stage below to explore its purpose and place in the process.
Building Capability
Training gives the model its initial capability by exposing it to vast data sets. The model identifies patterns that can later be used to form new responses.
Checking Performance
Testing uses controlled, unseen data. Because the examples were not used for training, they provide a more meaningful assessment of how the model handles unfamiliar input.
Generating a Response
Inferencing occurs when a user supplies new data and a prompt to the trained and tested model. The output may be a prediction, recommendation or another generated response.
| STAGE | MAIN INPUT | WHAT HAPPENS | MAIN PURPOSE |
|---|---|---|---|
| Training | Vast data sets | The model learns patterns from information. | Develop the model's initial capability. |
| Testing | Controlled, unseen data | The model's responses and performance are assessed. | Evaluate accuracy and performance. |
| Reinforcement learning | Information about output quality | The model learns which kinds of responses are considered better. | Improve future outputs. |
| RLHF | Prompts, human checks and rewards | Human operators review responses and reward correct ones. | Use human judgment to guide improvement. |
| Inferencing | New data and a prompt | The trained and tested model generates a response. | Put the model into practical use. |
The Use of Data
Data is the foundation of generative AI. The model depends on training data to learn patterns and on separate test data to evaluate whether that learning can be applied successfully. The two data types have different purposes and should not be confused.
Training Data
Training data supplies the enormous banks of information from which the AI model learns. Rather than storing a ready-made response for every possible prompt, the model learns patterns that help it construct a suitable response when prompted.
- Pre-training data is the first broad batch fed to the model, before refinement or fine-tuning.
- Later training data is generally more focused or specific and is used to refine the model.
- The relevance and quality of this information influence what the model learns.
- Poor-quality training data can lead to poor-quality or unreliable generated output.
Test Data
Test data is a separate bank of controlled information used to assess the model after training. It provides a fairer evaluation because the model has not previously learned from these examples.
- It must be unseen data that was not used in any training capacity.
- It is used to evaluate the model's performance and output.
- It shows how the model handles information beyond its training examples.
- Its role is assessment, not teaching or fine-tuning.
Why must test data remain unseen?
If the same information were used for both training and testing, the evaluation would not clearly show how well the model responds to genuinely new examples. Keeping test data separate allows performance to be assessed using information that did not contribute to the model's learning.
Simple analogy: training data is similar to the material a candidate studies, while test data is similar to new questions used to assess whether the candidate understood the subject. Studying the exact test answers would weaken the value of the assessment.
| DATA TYPE | WHEN USED | ROLE | KEY EXAM PHRASE |
|---|---|---|---|
| Pre-training data | At the beginning | Provides the first broad bank of information. | Before refinement or fine-tuning |
| Training data | During model development | Teaches and further focuses the model. | Used to train the model |
| Test data | After or apart from training | Assesses model performance and output. | Controlled and unseen |
The Role of Transformers
A transformer is a deep learning architecture that processes data in parallel using "attention mechanisms" to understand context and relationships between words.
Transformers are central to the way modern generative AI constructs language. They improve the model's ability to predict what should come next in a response—not only the next word, but also likely phrases, sentences and paragraphs. Repeating this prediction process allows the model to produce connected responses that may run into thousands of words.
Prediction at Every Step
A transformer uses the prompt and the response produced so far as context. It considers possible continuations, identifies a likely next element and then repeats the operation as the response grows.
Read What Is Available
The prediction is influenced by the user's prompt and by the words already generated in the response.
Select a Likely Continuation
The transformer helps estimate which word, phrase or sentence is most likely to follow in that context.
Extend the Response
The newly selected content becomes part of the context, and the prediction process repeats to form a longer response.
Why transformers are required for long responses
A long response is not normally produced as one indivisible block. It is developed through continuing predictions, with each new part building on what came before. This continuing use of context enables responses containing many connected sentences and paragraphs.
However, the transformer is predicting plausible continuations; it is not guaranteeing that every statement is factually correct. Therefore, response length, detail and confident wording should never be treated as automatic evidence of accuracy.
The Role of Feedback
Initial training gives a model broad capability, but its responses still need refinement. Human feedback helps demonstrate what a desired response looks like and which model-generated responses are judged correct. The syllabus identifies two related but distinct approaches: supervised fine-tuning and reinforcement learning from human feedback.
| METHOD | WHAT THE HUMAN DOES | WHAT THE MODEL RECEIVES | MAIN IDEA |
|---|---|---|---|
| SFT | Creates the desired response to a prompt. | A prompt paired with a high-quality human-created response as training data. | Show the model what a desired answer looks like. |
| RLHF | Poses prompts, checks model responses and rewards correct responses. | Feedback based on human judgment of response quality. | Reinforce responses that humans consider correct. |
Provide the Target
A human writes a clear desired answer for a prompt. The prompt and answer are then used as a training example for the model.
Evaluate the Output
The model answers a prompt. A human checks that response and provides a reward when the response is considered correct.
Fine-tune Repeatedly
Feedback and fine-tuning are ongoing. Repeated adjustment explains why users may observe continuing improvement in generative AI responses.
Remember the Distinctions
Learning and Evaluation
Training data teaches patterns; unseen test data assesses performance.
Example and Reward
SFT supplies a human-created desired response; RLHF rewards good responses.
Training and Inferencing
Training develops the model; inferencing uses it with new data.
Length Is Not Accuracy
Transformers enable long responses, but those responses can still be wrong.