Understanding Data and Model Lineage
Data and model lineage refer to the detailed record of the origin, transformation, and evolution of data and models used in AI and generative AI systems. This information is important for understanding the origin, reliability, and potential biases or limitations of the data and models used in these systems.
What is source citation and data origins documentation?
Citing sources and documenting origins
Source citation and documenting data origins are essential tasks that contribute to securing your AI systems. These tasks help ensure the transparency, traceability, and accountability of the data and information used in the AI system. This is important for maintaining the integrity and trustworthiness of the system. These tasks involve providing information about the sources of the data used to train the generative AI model and the provenance of the data. Following is a description of source citation and how to document data origins.
- Source Citation
- Documenting Data Origins
Documenting data origins in the context of generative AI involves providing detailed information about the provenance, or the place of origin of the data used to train the model.
This includes the following:
- Details about the data collection process
- The methods used to curate and clean the data
- Any preprocessing or transformations applied to the data
Documenting the data origins is important for understanding the potential biases, limitations, or quality issues that might be present in the training data. This can ultimately impact the performance and reliability of the generative AI model.
Tools and techniques
By implementing the following tools and techniques, generative AI systems can effectively document the sources and origins of the data used in their development. This promotes transparency, accountability, and reproducibility. The following describes some of the common techniques and tools.
- Data lineage
- Cataloging
- Model cards
Model cards are a standardized format for documenting the key details about an ML model, including its intended use, performance characteristics, and potential limitations.
In the context of generative AI, model cards can be used to provide source citations and data origin documentation. This helps users understand the provenance (lineage) of the data used to train the model.
Model cards can include details about the datasets used, their sources, licenses, and any known biases or quality issues in the training data.
Amazon SageMaker Model Cards
You can use Amazon SageMaker Model Cards to document critical details about your ML models in a single place for streamlined governance and reporting.
Model cards can catalog details, such as the intended use and risk rating of a model, training details and metrics, evaluation results and observations. It also catalogs additional call-outs such as considerations, recommendations, and custom information. By creating model cards, you can do the following:
- Provide guidance on how a model should be used.
- Support audit activities with detailed descriptions of model training and performance.
- Communicate how a model is intended to support business goals.
Additional resources
Enforce Data Lineage
To learn how to implement a plan to enforce data lineage, choose the following link.
Documentation
Data Protection
To learn how data protection is addressed within the AWS Cloud Adoption Framework for Artificial Intelligence, Machine Learning, and Generative AI, choose the following link.
AWS Whitepaper
Amazon SageMaker Model Cards
To learn more about using Amazon SageMaker Model Cards to document critical details about your ML models, choose the following link.
Developer Guide
Next, you will learn some best practices for securing your data.