GDPR and AI

Artificial intelligence and data protection are sometimes discussed as though they were separate subjects. One concerns algorithms, models and automated decisions; the other concerns privacy notices, consent and the handling of personal information. In practice, the two meet remarkably quickly.

An AI system may be trained using information about people, use personal data when making predictions, infer things about individuals that they never told the organisation, monitor behaviour, generate profiles or contribute to decisions about employment, credit, insurance, healthcare and access to services. Even a general-purpose AI tool can create data-protection issues when an employee types personal or confidential information into a prompt.

For organisations using AI, GDPR therefore cannot be treated as a final compliance check carried out after the technology has been selected. Decisions about data, purpose, architecture, suppliers, testing, human oversight, retention and monitoring can all affect whether an AI system processes personal information lawfully.

This article is written primarily for managers, Business Analysts, testers, project and programme managers, product owners, governance and risk professionals and others involved in selecting, building, approving or using AI. It concentrates principally on the UK GDPR and Data Protection Act 2018, while identifying important differences from the EU GDPR where necessary. It is an educational guide rather than legal advice.

Start with a simple question: is personal data involved?

Before worrying about AI-specific issues, establish whether the system processes personal data at all.

Personal data is information relating to an identified or identifiable living person. Obvious examples include names, addresses, employee numbers and email addresses, but identification does not depend upon somebody's name appearing in a dataset. Location information, online identifiers, combinations of characteristics and other information may allow an individual to be identified.

This matters in AI because project teams can easily assume that information has ceased to be personal data simply because obvious identifiers have been removed. Pseudonymised data can still be personal data where individuals can be re-identified using additional information. Truly anonymised information falls outside GDPR, but anonymisation needs to be effective rather than merely asserted.

There is another complication. AI does not merely consume information; it can create it. A system may infer somebody's likely interests, behaviour, preferences, health status, financial position or other characteristics from apparently innocuous input data. Those inferences may themselves constitute personal data when they relate to an identifiable person.

The right starting point is therefore not simply to ask whether the database contains names. Map what information enters the system, what the system derives from it, what outputs it produces and whether any of those things relate to identifiable people.

The data-protection principles still provide the foundation

AI can make data processing technically complicated, but the fundamental questions remain remarkably familiar. Personal data must be processed lawfully, fairly and transparently. It must be collected for specified, explicit and legitimate purposes and not subsequently used incompatibly with those purposes. Organisations should process only the personal data that is adequate, relevant and necessary, keep it accurate where accuracy matters, retain it no longer than necessary and protect it appropriately.

The organisation must also be able to demonstrate compliance. That accountability principle is particularly important with AI because many of the decisions that determine data-protection risk occur long before the system reaches its users.

For a project team, these principles should become questions. Why do we need this information? What exactly are we going to do with it? Would the people concerned reasonably expect that? How long do we need it? Who will receive it? How will we protect it? What happens if it is wrong? What evidence shows that our decisions were reasonable?

Those questions are not obstacles to AI development. They are part of designing an AI system that an organisation can responsibly operate.

Lawful basis comes before processing, not after it

If an AI system processes personal data, the organisation needs an appropriate lawful basis for that processing. Under the UK GDPR these include consent, contract, legal obligation, vital interests, public task and legitimate interests, although not every basis is available or appropriate in every circumstance.

A common mistake is to ask, "What is our lawful basis for the AI?" That may be too broad a question. AI development and deployment can involve several different processing activities carried out for different purposes.

An organisation might use one collection of personal data to develop and test a model, process new customer information when the finished system is used, retain operational data for monitoring and subsequently use selected cases to improve the model. Those activities do not automatically have the same purpose or the same lawful basis.

The project should therefore identify the distinct processing operations, their purposes and the lawful basis relied upon for each. That analysis should take place before the processing begins and the reasoning should be documented.

Consent is not the universal answer

When personal data is mentioned, somebody will often suggest obtaining consent. Sometimes consent is entirely appropriate, but it is not a general permission slip that makes every AI use lawful.

Valid consent must satisfy demanding conditions. It needs to be freely given, specific, informed and unambiguous, and people must be able to withdraw it. That can make consent unsuitable where there is a substantial imbalance of power or where the processing cannot realistically stop if an individual withdraws permission.

Employment provides an obvious example. An employee may technically be asked to consent to an employer's use of an AI monitoring system, but the employment relationship can make it difficult to argue that the choice was genuinely free.

Equally, an organisation should not select legitimate interests merely because obtaining consent would be inconvenient. Where legitimate interests is appropriate, the organisation needs to identify the legitimate interest, show that the processing is necessary and balance that interest against the rights and freedoms of the people affected.

The lawful basis should describe the reality of the processing. It should not be selected retrospectively to justify a system the organisation has already decided to build.

Special category data needs additional care

Some personal information receives additional protection. Under the UK GDPR, special category data includes information revealing racial or ethnic origin, political opinions, religious or philosophical beliefs or trade-union membership, as well as genetic data, certain biometric data, health data and information concerning a person's sex life or sexual orientation.

Processing this information normally requires both an Article 6 lawful basis and an appropriate Article 9 condition. Criminal-offence data is also subject to additional rules.

AI creates a less obvious difficulty because a system may infer sensitive information even when the organisation did not explicitly collect it. A combination of behaviour, purchases, language, location or other features may enable a model to derive characteristics that the individual never provided directly.

Project teams therefore need to consider not only the fields intentionally supplied to a model, but what the model may be capable of deriving from them.

Training data is not a law-free zone

The fact that personal data is being used to train an AI model rather than to make an immediate decision about somebody does not remove the data-protection requirements.

An organisation still needs to understand where the training data came from, why it is entitled to process it, what people were originally told, whether the new use is compatible with the original purpose and what rights the individuals concerned retain.

This becomes particularly challenging when datasets have been assembled from multiple sources or obtained from third parties. The fact that information was publicly accessible does not automatically mean that it is free of data-protection obligations. Nor does purchasing a dataset transfer every compliance question to the seller.

The project should understand the provenance of important training data: where it originated, how it was collected, what permissions or lawful bases support its use, what transformations have been applied and whether its use for AI development was reasonably foreseeable.

The same concern applies when organisations use their own historical records. A customer may have provided personal data so that an organisation could deliver a service several years ago. That does not automatically establish that the information can now be reused for every conceivable AI experiment.

Purpose limitation collides with "perhaps the AI will find something"

One of the attractions of machine learning is its ability to discover relationships that people did not anticipate. That can sit uncomfortably beside the data-protection principle of purpose limitation.

An organisation should know why it is collecting and processing personal data. "Let's put everything into the model and see what it finds" is therefore a dangerous starting point when the dataset contains personal information.

Purpose limitation does not prevent organisations from innovating, but it requires them to define and justify what they are doing. If personal data collected for one purpose is later reused for another, the organisation needs to consider whether the new purpose is compatible with the original one or whether another lawful route is required.

This is another area where good Business Analysis and good data protection reinforce one another. A clearly defined business problem helps establish what information is actually necessary and why the organisation is processing it.

AI's appetite for data does not defeat data minimisation

Machine-learning teams often prefer more data. More examples can improve performance, and additional features may reveal useful patterns. GDPR, however, requires personal data to be adequate, relevant and limited to what is necessary for the purpose.

That creates a genuine engineering and governance tension. The answer is not that AI automatically overrides minimisation, nor that minimisation means an AI system must use the smallest imaginable dataset. The organisation needs to be able to explain why the personal information being processed is necessary and proportionate to the purpose.

For example, adding another demographic feature may improve a model's predictive performance. The team should still ask whether that improvement justifies processing the additional personal information, what new risks it creates and whether a less intrusive alternative could achieve an acceptable result.

This is exactly the kind of trade-off that should be documented rather than left buried inside a data-science notebook.

Accuracy means more than model accuracy

AI teams use the word accuracy in a technical sense, usually to describe a measure of predictive performance. GDPR's accuracy principle is related but not identical.

Personal data should be accurate and, where necessary, kept up to date. That raises several questions in an AI system. Is the source information factually correct? Are labels reliable? Are inferences being treated as facts when they are actually probabilities? Can incorrect information be corrected? If a source record is corrected, what happens to downstream systems and decisions?

A model may also be statistically impressive while being unreliable for a particular group. Conversely, a prediction can be statistically reasonable while being wrong about a particular individual.

Project teams therefore need to distinguish the accuracy of the underlying personal data, the statistical performance of the model and the correctness or reliability of a particular inference or decision. They are connected, but they are not the same thing.

Fairness is not achieved merely by removing names

Data protection requires personal data to be processed fairly. In practical terms, organisations should not use people's information in ways that are unjustifiably detrimental, unexpectedly intrusive or misleading.

AI makes this particularly important because apparently neutral data can produce systematically different outcomes between groups. Historical data may contain earlier inequalities. Variables such as postcode, education or employment history can act as proxies for characteristics that have been removed explicitly. Models may perform differently for populations that were poorly represented in their training data.

An organisation can therefore satisfy itself that a dataset contains no explicit field labelled ethnicity or sex and still operate an unfair system.

Fairness needs to be investigated throughout the lifecycle: in the definition of the problem, collection and labelling of data, selection of features, choice of model and thresholds, testing, deployment and subsequent monitoring.

This is where data protection and AI testing meet directly. Aggregate model performance is not enough. Teams may need to examine how errors and outcomes are distributed across relevant populations and investigate whether observed differences are justified.

Transparency is more than adding "we use AI" to a privacy notice

People should be given meaningful information about how their personal data is used. AI can make this difficult because the processing may be complex, distributed across several organisations or difficult to explain in ordinary language.

Complexity is not, however, a justification for opacity.

Transparency begins with the basics. People should be able to understand what information is being collected, why it is being used, who is responsible for the processing, who receives it, how long it is retained and what rights they have. Where AI contributes to decisions about them, additional explanation may be appropriate or legally required.

That does not necessarily mean explaining the mathematics of a neural network or publishing source code. An explanation that is technically exhaustive but incomprehensible to the person affected has achieved very little.

A useful explanation may instead describe what information mattered, how the system contributed to the decision, the principal factors influencing the outcome, the limitations of the system and what the person can do if they believe the result is wrong.

Automated decision-making: the UK and EU rules have diverged

This is one area where older GDPR explanations can now be actively misleading for UK organisations.

Under the EU GDPR, Article 22 continues to provide a right not to be subject to a decision based solely on automated processing, including profiling, where that decision produces legal effects or similarly significantly affects the individual, subject to specified exceptions and safeguards.

The UK position changed following the Data (Use and Access) Act 2025. UK law now permits significant decisions based solely on automated processing in a wider range of circumstances, provided the applicable requirements are met and appropriate safeguards are provided. Additional restrictions remain where special category data is involved.

Those safeguards include providing information about the decision and allowing the individual to make representations, obtain human intervention and contest the decision.

The distinction between a solely automated decision and an AI-assisted human decision therefore matters. In the UK framework, a decision is solely automated where there is no meaningful human involvement.

The word meaningful matters. A person who simply clicks "accept" on every AI recommendation without considering it does not necessarily transform automation into genuine human decision-making. Human involvement needs to be real rather than ceremonial.

Human involvement must be designed, not assumed

Adding a human reviewer can be an important safeguard, but only if the organisation designs a process in which that person can genuinely exercise judgement.

The reviewer needs sufficient information to understand the recommendation, enough competence to challenge it, enough time to consider the case and authority to reach a different conclusion. The system should not be designed in a way that makes disagreement technically difficult or organisationally unwelcome.

Testing should therefore look beyond whether an override button exists. Does the reviewer know when intervention is necessary? Can they see the information needed to make an independent judgement? What happens when they reject the AI recommendation? Is the action recorded? Does workload make meaningful review realistic?

A nominal human-in-the-loop can look reassuring on a process diagram while providing very little protection in practice.

Data Protection Impact Assessments belong near the beginning

A Data Protection Impact Assessment, or DPIA, is one of the most useful tools for an AI project involving personal data. Where processing is likely to result in a high risk to people's rights and freedoms, a DPIA is a legal requirement. Even where the threshold is not met, carrying one out can be valuable for a significant AI initiative.

The important word is impact. A DPIA should not become a form completed shortly before deployment to demonstrate that somebody remembered GDPR.

It should describe what the organisation intends to do, what personal data will be processed, why the processing is necessary, who may be affected, what could go wrong, how serious those consequences could be and what measures will reduce the risks.

For AI, that can include questions about bias, statistical error, unexpected inferences, vulnerable populations, transparency, human oversight, security, supplier access, data provenance and the consequences of incorrect decisions. It should also consider whether the same business objective could be achieved in a less intrusive way.

The DPIA should begin while those decisions can still influence the design. Discovering a serious privacy problem after the model has been trained, the supplier contracted and the operating process built leaves the organisation with far fewer options.

Privacy by design is particularly important with AI

Data protection by design and by default means considering data-protection requirements when a system is being designed rather than attempting to bolt them on afterwards.

In an AI project this may influence the data selected for training, whether information is anonymised or pseudonymised, the features supplied to the model, access controls, logging, retention, explanation facilities, human review, monitoring and the architecture used to separate data from models.

This is another reason GDPR cannot belong exclusively to the Data Protection Officer. By the time the DPO sees the finished system, many of the decisions that determine privacy risk may already have become expensive to reverse.

Business Analysts, architects, developers, data scientists, testers, security specialists, product owners and information-governance professionals all make decisions that can strengthen or weaken data protection.

Individual rights do not disappear inside a model

People retain data-protection rights when AI is involved. Depending upon the circumstances, these can include rights to be informed, obtain access to personal data, rectify inaccurate information, request erasure, restrict processing, object to processing and obtain data portability.

AI can make satisfying those rights technically difficult, particularly when large datasets and trained models are involved. Difficulty does not by itself make the rights disappear.

An organisation therefore needs to understand where personal information exists throughout the AI lifecycle. Can it identify the information held about an individual? Can inaccurate source data be corrected? What happens to copies in validation or test datasets? Has information been embedded in a form that is difficult to isolate? What happens if an individual objects to processing used for model improvement?

These questions are much easier to answer when they have influenced the architecture from the beginning.

Generative AI creates some deceptively ordinary privacy problems

Not every GDPR problem requires a sophisticated predictive model. Some arise because an employee opens a generative-AI tool and pastes something into the prompt box.

A manager might ask an AI assistant to summarise employee appraisals. A customer-service worker might paste a complaint containing names, contact details and account information. A recruiter might upload CVs for comparison. A developer might submit production logs containing user identifiers while asking for help diagnosing a fault.

At that moment the organisation needs to know what service is being used, where the information goes, what the provider does with prompts and uploaded files, whether the information is retained, whether it may be used to improve a model, what subprocessors are involved and whether personal data is transferred internationally.

A corporate subscription with appropriate controls may behave very differently from a free consumer service. The important point is that "we didn't build the AI" does not answer the data-protection question.

Buying AI does not outsource accountability

Most organisations will consume far more AI than they develop. Procurement is therefore becoming an important part of data-protection governance.

The organisation needs to understand the relationship between the parties. Who determines why and how personal data is processed? Is the supplier acting as a processor on the organisation's instructions, as an independent controller for some activities, or in a more complicated arrangement? The answer may differ between different processing operations within the same service.

Contracts need to reflect reality. They should address the nature and purpose of processing, types of personal data, categories of individuals, security, retention, subprocessors, international transfers, assistance with individual rights and incidents, and what happens to the information when the service ends.

AI adds further questions. Can the supplier use customer data or prompts to train its own models? Can that behaviour be disabled? What happens when the underlying model changes? What evidence is available about accuracy and bias? How are incidents reported? Can the organisation obtain enough information to explain significant decisions to the people affected?

A procurement process that checks price, functionality and an ISO certificate but never establishes what happens to the personal data has not completed its due diligence.

International transfers can be hidden in the architecture

Cloud-based AI services may involve providers, subprocessors and infrastructure in several countries. Personal data can therefore cross borders even though the employee using the system never sees it leave their browser.

Where personal data is transferred internationally, the organisation needs to consider the applicable UK or EU transfer rules, depending upon the regime concerned. The appropriate mechanism will depend upon the destination, relationship between the parties and circumstances of the transfer.

This is another reason to map data flows rather than relying upon the location of the organisation's own office or server. The relevant question is where the personal data is actually processed and made accessible.

Security becomes more complicated, not less important

AI introduces familiar security risks and some less familiar ones. Personal data may be exposed through poorly controlled datasets, insecure interfaces, excessive access, compromised suppliers or ordinary cyberattack. AI systems can also introduce model-specific attack surfaces and unexpected routes by which information can be revealed.

Generative systems deserve particular attention because users can submit free-form content and receive free-form responses. Organisations need to consider whether confidential or personal information can be disclosed through prompts, retrieved sources, model outputs, logs or integrations with other business systems.

Security testing should therefore follow the complete data flow rather than examining only the model. Authentication, authorisation, APIs, storage, logging, integrations, user interfaces, supplier infrastructure and operational processes can all affect the confidentiality, integrity and availability of personal information.

Retention is awkward when data has helped train a model

The storage-limitation principle requires personal data to be kept no longer than necessary for the purpose for which it is processed.

That sounds straightforward when dealing with conventional records. AI can make it more complicated because data may exist in raw datasets, cleaned datasets, training subsets, validation sets, test data, logs, backups and other derived forms.

The project should therefore establish retention rules for the different stages and copies rather than treating "the AI data" as a single thing.

Trained models create harder technical and legal questions where information from training data may be reflected in model behaviour. These questions cannot safely be solved with a blanket statement that the original dataset has been deleted. The organisation needs to understand its particular technology and processing sufficiently to know what deletion actually means.

Testing with personal data is still processing

Test environments sometimes receive less data-protection attention than production because they are regarded as temporary or internal. GDPR makes no such convenient distinction.

If personal data is copied from production into a test environment, it is still personal data and needs an appropriate purpose, lawful basis and protection. Access should be controlled, unnecessary fields removed and retention limited. Synthetic, anonymised or appropriately transformed data may be preferable where it can meet the testing objective.

There is, however, an important AI complication. A perfectly sanitised test dataset may fail to reveal real differences in model behaviour between populations. Testing for fairness and bias can sometimes require access to characteristics that the production decision itself should not use.

That is not an excuse to collect everything. It is a reason to define the testing purpose carefully, establish the appropriate legal conditions, restrict access and retain the information only as long as necessary.

What does this mean for Business Analysts?

Business Analysts can influence data protection before a single model is trained. Requirements determine what information the system needs, what decisions it supports, who is affected, what explanations are required and where human intervention occurs.

A BA should be able to challenge vague statements such as "the system will analyse customer data using AI". Which customer data? For what purpose? What output will be produced? Who receives it? Does it influence a significant decision? How long is the information retained? What happens if the customer objects or the information is wrong?

Requirements should also capture privacy behaviour. A system may need to support correction, deletion, restricted processing, audit trails, human review, retention rules and the information necessary to respond to a subject access request.

These are not merely legal annotations to technical requirements. They are requirements for how the system must behave.

What does this mean for testers?

Testers have an unusually useful role because data-protection claims can often be turned into testable propositions.

If the organisation says that information is deleted after a specified period, test it. If access is restricted by role, test the restrictions. If a human can override an automated recommendation, exercise that route. If a privacy notice says a particular field is not used for a decision, investigate whether the implementation agrees. If people can correct their data, follow the correction through downstream processing.

AI adds further questions. Does performance differ materially between relevant groups? Can sensitive information be inferred from supposedly innocuous features? Do model updates change previously established behaviour? Can a generative system disclose information supplied by another user or retrieved from an inappropriate source?

Testing cannot certify that an organisation is legally compliant. It can, however, provide evidence about whether the technical and operational claims on which compliance depends are actually true.

What does this mean for project and product managers?

The project manager's challenge is timing. Data-protection work needs to occur while the project can still respond to it.

The DPO or privacy specialist should not first encounter the system when asked to approve release. Data mapping, lawful-basis analysis, DPIA work, supplier due diligence, security requirements and individual-rights considerations may all affect architecture, contracts, cost and schedule.

Product managers also need to control function creep. An AI capability that begins with one clearly defined purpose can acquire additional uses very quickly once people discover what it can do. Each new use may alter the purpose, affected population, data requirements and risk.

"Technically possible" should therefore trigger another question: "Are we entitled to do it?"

What does this mean for managers approving AI?

Senior managers do not need to become data-protection lawyers, but they do need enough understanding to challenge the evidence presented to them.

Before approving an AI system involving personal data, management should be able to establish what information is being processed, why it is necessary, what lawful basis supports the processing, whether sensitive information is involved, who may be affected, what significant risks have been identified and how they have been controlled.

They should also understand the role of suppliers, whether international transfers occur, how people can exercise their rights, how important automated decisions are reviewed, what monitoring will continue after deployment and who is accountable when something goes wrong.

If those answers exist only in the head of a developer or supplier account manager, the organisation does not yet have strong governance.

GDPR and the EU AI Act overlap, but they are not interchangeable

For organisations operating within the scope of the EU AI Act, data protection and AI regulation need to be considered together.

The two regimes address different but overlapping concerns. GDPR focuses on the processing of personal data and the rights of individuals. The AI Act regulates particular AI practices, systems and models according to their use, risk and the role of organisations in the AI value chain.

A high-risk AI system may therefore need to satisfy AI Act requirements concerning areas such as risk management, data governance, technical documentation, human oversight, accuracy and monitoring while simultaneously satisfying GDPR requirements for the personal data it processes.

Neither framework provides a shortcut through the other. A system is not GDPR-compliant merely because it satisfies the AI Act, and a lawful data-processing operation does not automatically satisfy AI Act obligations.

For some projects, the sensible approach will be to connect the evidence. A GDPR Data Protection Impact Assessment and an AI Act fundamental-rights impact assessment, where required, should not become isolated exercises conducted by different teams that never compare conclusions.

Data protection continues after deployment

An AI system that was acceptable on launch day does not remain acceptable automatically.

Data changes. Populations change. Suppliers update models. New integrations are added. Employees discover new uses. Performance can drift and previously unlikely risks can become real. A system that originally supported human decision-making may gradually become something staff accept automatically.

Monitoring therefore needs to consider privacy and individual rights as well as technical performance. Organisations should review incidents, complaints, unexpected outputs, changes in data, changes in purpose and evidence that particular groups may be experiencing different outcomes.

Significant changes should trigger reassessment. That may mean updating a DPIA, revisiting the lawful basis, changing privacy information, performing new testing or reconsidering whether the processing remains justified at all.

Where should an organisation start?

The first practical step is visibility. Identify the AI systems and AI-enabled services being used across the organisation, including capabilities embedded inside products that were not originally purchased as "AI systems".

For each significant use, map the personal data. Establish where it comes from, why it is processed, what is inferred or generated, who receives it, where it goes, how long it is retained and which suppliers can access it. Identify the controller and processor relationships and the lawful basis for the relevant processing activities.

Then examine risk. Determine whether a DPIA is required, whether special category data is involved, whether decisions significantly affect people, whether meaningful human involvement exists, and whether fairness, security, transparency or individual rights create particular concerns.

Finally, connect those findings to the project rather than filing them in a privacy folder. Turn necessary controls into requirements, acceptance criteria, tests, contractual obligations, operational procedures and monitoring.

That is the difference between documenting data protection and engineering it into the system.

About this guide

This article provides an educational overview of data protection issues arising from the development and use of artificial intelligence. It is intended for professionals involved in AI projects and organisational decision-making and does not constitute legal advice.

UK data-protection law changed following the Data (Use and Access) Act 2025, including important changes concerning automated decision-making. At the time of writing, the Information Commissioner's Office is updating parts of its guidance to reflect those changes. Organisations making compliance decisions should therefore consult current ICO guidance and the applicable legislation, and obtain specialist advice where appropriate.

Organisations operating in the European Union or processing personal data subject to the EU GDPR should also check the EU regime separately. Although UK and EU data-protection law remain closely related, important differences now exist and should not be obscured by referring loosely to both simply as "GDPR".