A model is only as good, and as lawful, as the data it learned from. Data governance for AI answers three questions: where did the data come from, were you allowed to use it for this, and is it fit for the purpose?
Provenance and rights
Record the source of every dataset, the licence or legal basis for using it, and any restrictions. Scraped web data, purchased datasets and customer data each raise different issues: copyright and database rights, website terms of use, contract terms and data protection. Courts in the US have distinguished between lawfully acquired and pirated training material, which makes provenance records a liability question, not a nicety.
Personal data
Under the GDPR and UK GDPR, the purpose limitation principle asks whether training is compatible with the reason the data was first collected. Minimisation asks whether you need the personal data at all, or whether anonymised or synthetic data would do. European regulators expect a legitimate interests assessment before training.
Quality and bias
For high-risk systems, Article 10 of the EU AI Act requires training, validation and testing data to be relevant, sufficiently representative and, as far as possible, free of errors, with examination for possible biases. These duties apply from December 2027. In the US, discrimination claims make the same point through litigation rather than regulation.
If you cannot say where your training data came from, you cannot say your model is lawful.
What to do
Keep a register of datasets with source, rights, personal data content and known limitations. Keep test data separate from training data. Repeat bias testing whenever the data or the model changes.
This article is general information on the law as at November 2026. It is not legal advice and has not been updated since that date. See the terms of use.