Using AI in the company in compliance with data protection regulations: obligations, processes, evidence

Since the General Data Protection Regulation came into force in May 2018, every company bears the full burden of proof as soon as personal data is processed. With the EU AI Act (Regulation 2024/1689), applicable on a staggered basis from August 2, 2026 and fully effective from August 2, 2027, a second legal matter is added that is beginning to take effect in parallel. Anyone who uses ChatGPT, Microsoft 365 Copilot, Gemini, Claude or Mistral in their everyday work usually processes personal data and at the same time falls within the scope of the AI ​​regulation. The data protection officer thus becomes the central control authority for every productive implementation, the information security officer becomes the partner for the technical protective measures.

This article provides the list of obligations, the test sequence and the documents that an audit, a data protection supervisory authority or an internal auditor want to see. It explains when a data protection impact assessment is mandatory according to Art. 35 GDPR, how a reliable AI policy must be structured, how order processing according to Art. 28 GDPR is secured with US providers and EU-based platforms and what additional obligations the EU AI Act imposes on high-risk systems, general AI models (GPAI) and operators. The templates, reporting lines and appointment certificates shown in CIVAC can be found as concrete cross-references in the text.

Generative AI touches at least four legal matters in parallel and usually other sector-specific regulations. Firstly, the GDPR as soon as prompts, training data or outputs have personal references. The well-known obligations apply here: legal basis according to Art. 6 GDPR, consent according to Art. 7 if relevant, information obligations according to Art. 13 and 14, list of processing activities according to Art. 30, data protection impact assessment according to Art AI models (GPAI) know their own transparency and documentation obligations. Thirdly, the BDSG, which in Section 26 links the processing of employee data to strict conditions and in Section 38 regulates the obligation to appoint a data protection officer. Fourth, the Works Constitution Act: According to Section 87 Paragraph 1 No. 6 BetrVG, the works council has a mandatory right of co-determination because generative AI is generally considered a technical device for behaviour and performance control.

In addition, sector-specific requirements with their own supervisory regimes apply. Banks are bound to BAIT and MaRisk, insurers to BaFin's VAIT, the public sector to OZG and EGovG requirements, KRITIS operators to NIS-2 with 24-hour early warning and 72-hour follow-up notification. Anyone who uses AI for credit decisions, applicant selection, law enforcement or critical infrastructure quickly ends up in the high-risk class of the EU AI Act with its requirements for risk management, data quality, logging, human supervision and conformity assessment. The external data protection officer organises these layers and translates them into concrete specifications for departments, IT and law. In the CIVAC workspace, the assignment is stored as a control table: use case, affected legal norm, responsible role, evidence document, next check. The appointment certificate, signed, filed, verifiable.

Before an AI tool is released, the use case must be clearly described. Four questions guide you through the classification and at the same time provide the reasoning for the later documentation. First, what data flows as input? Pure company figures, anonymous text modules or personal data such as applicant CVs, customer names, health information or employee data? Second, what happens to the output? Is it checked as a draft by a human, stored in a database or does it flow automatically into a decision with an external impact on those affected? Third: Which risk class according to the EU AI Act is relevant? Minimal risk, limited risk with transparency obligation, high risk according to Annex III or prohibited practice according to Article 5? Fourth: who is the supplier, who is the operator, who is the importer, who is the dealer? These roles from Art. 3 EU AI Act determine the respective cascade of duties.

In practical terms, this means: Marketing, which uses ChatGPT to design generic headlines, is not the same as HR, which pre-sorts applications and therefore falls under Annex III No. 4. A legal department that summarizes general contract clauses without entering client names has a different risk profile than a service centre that dumps tickets with plain text data into a cloud LLM API. The CIVAC workspace template “AI use case” records the purpose, data categories, legal basis, recipient, storage location, deletion period, model provider and risk class in a structured form. The next obligations are automatically derived from this: DPIA yes or no, AVV required, technical and organisational measures (TOM) according to Art. 32 GDPR, training obligation according to Art. 4 EU AI Act, works council participation. Others run compliance like a filing cabinet. We run it like software. Anyone who skips the classification loses the most important argument before any supervision: that the risk assessment took place beforehand and not after the incident, and that the evidence was documented.

Art. 35 GDPR requires a data protection impact assessment (DPIA) if processing is “likely to result in a high risk to the rights and freedoms of natural persons”. The Data Protection Conference (DSK) of the German supervisory authorities expressly listed the “use of artificial intelligence to process personal data to control interaction with those affected or to evaluate personal aspects” in its must-have list of October 17, 2018, updated in 2023. This means that a DPIA is mandatory for most productive AI applications involving people, not an option. The European Data Protection Board (EDPB) has further specified the requirements in its Opinion 28/2024 on AI models and emphasised that the risk assessment also includes model training, inference and subsequent use.

The DPIA consists of six building blocks: systematic description of processing, assessment of necessity and proportionality, identification of risks for those affected, measures to reduce risks, consultation of the data protection officer and, if necessary, consultation of the supervisory authority in accordance with Art. 36 GDPR with high residual risk. With generative AI, the typical risks are: unintentional disclosure of data via logs or training, hallucination with incorrect person assignment, lack of information and deletion ability against model weights, discrimination due to training bias, transmission to unsafe third countries, lack of robustness against prompt injection, lack of explainability of output. The CIVAC DPIA template maps each building block as a structured field and links risks with concrete measures. The DPO countersigns in the workspace and the date remains stored in an audit-proof manner. Deadline begins as soon as we become aware of it. Anyone who only follows up with a DPIA after an incident loses an argument about fault before the supervisory authority and in the fine proceedings according to Art. 83 GDPR. Fines range up to 20 million euros or 4% of global group sales.

Almost every productive AI solution is software-as-a-service. This means that the provider is regularly a processor in accordance with Art. 28 GDPR, and an order processing agreement (AVV) is required. The mandatory components are conclusively regulated in Art. 28 Para. 3 GDPR: subject, duration, type and purpose of processing, type of data, categories of data subjects, obligations and rights of the person responsible. In addition, there is the subcontractor list, the technical and organisational measures, the support obligations for inquiries from those affected, the deletion and return obligations at the end of the contract and audit rights. The AVV is available online from OpenAI, Anthropic, Google, Microsoft, Mistral or Aleph Alpha; The legal review must keep an eye on the subprocessor system, the storage locations, the telemetry transmission and the deletion and training clauses.

The third country transfer to the USA remains politically sensitive. The EU-US Data Privacy Framework has been in effect since July 10, 2023 and provides a valid legal basis with the adequacy decision, but is a politically fragile mechanism after the experience with Safe Harbor and Privacy Shield. If you want to be on the safe side, choose providers with EU data residency, standard contractual clauses (SCC 2021/914) plus transfer impact assessment and contractually guaranteed encryption with customer keys (BYOK). Microsoft offers the EU Data Boundary, AWS the AI ​​Service Opt-Out, Anthropic European inference regions, Aleph Alpha completely German hosting. CIVAC is the compliance platform and officer-as-a-service and provides the comparison per provider as an audit template in the workspace: storage location, training clause, SCC module, subprocessors, certifications such as ISO/IEC 27001:2022 and SOC 2 Type II. Licence the workspace for your internal representatives or have our representatives order it. The auditor calls, the evidence is ready.

A productive AI introduction rarely fails due to technology, often due to governance. Three documents form the minimum framework. Firstly, the AI ​​policy, which describes for the workforce which systems are approved, what data can be entered, who approves, what labelling requirements apply to AI-created content, how hallucinations are treated and how incidents are reported. Secondly, the training documents that operationalize Article 4 of the EU AI Act (AI competence). Since February 2, 2026, providers and operators have been obliged to ensure that their own staff and people working on their behalf have the necessary AI competence, measured in terms of previous training, experience, area of ​​application and risk profile. Thirdly, the works agreement, which reflects the right of co-determination in accordance with Section 87 Paragraph 1 No. 6 BetrVG and creates clarity for the workforce, staff council and supervisory authority.

A two-stage training requirement has proven to be effective in practice. All employees complete a basic module of 30 to 45 minutes with a knowledge test and a mandatory certificate. Power users from HR, law, finance, sales, research and development receive an in-depth module with use cases from their own area as well as concrete negative examples. Participation is documented in the workspace, reminders for refreshers run automatically after 12 months. Audit-proof, documented, Section 26-proof. The works council should be involved in the pilot phase early on; The company agreement typically contains purpose limitation, data categories, technical blocks, evaluation and logging rules, a right of termination if the purpose is not met, and periodic reporting to the co-determination bodies. Anyone who presents these three documents has laid the foundation for every supervisory review, every audit question and every application for contracts in which public or private clients require proof of the proper use of AI. Supplier audits, IT due diligence in the M&A process and cyber insurers are increasingly explicitly checking this triad.

The EU AI Act has been in force since August 1, 2024 and will apply on a staggered basis. Prohibitions according to Art. 5 and AI competence according to Art. 4 have been in effect since February 2, 2026. The GPAI obligations and the penal regime apply from August 2, 2026. High-risk systems according to Annex III are fully covered from August 2, 2027, embedded high-risk systems from Annex I even until August 2, 2027 with transitional provisions. Four risk classes can be distinguished. Unacceptable (banned): social scoring, real-time remote biometric identification in public spaces with narrow exceptions, emotion-detecting systems in the workplace and education, manipulative practices. High: AI in recruiting, creditworthiness, critical infrastructure, law enforcement, justice, migration, education. Limited: Chatbots, deepfakes and AI-created content with transparency requirements. Minimal: spam filter, recommendation logic, spelling aids.

For operators, i.e. most companies, the central obligations are: compliance with the provider's operating instructions in accordance with Art. 26, ensuring human supervision, quality of input data, logging and storage of logs, informing the employee representatives before putting a high-risk system into operation in the workplace, informing those affected about the use, reporting serious incidents. From August 2026, GPAI models will have their own requirements: technical documentation, copyright policy, training data overview within the meaning of Art. 53, in the case of “systemic risk”, additional assessment, red teaming and reporting of serious incidents. Fines range up to 35 million euros or 7% of global group sales according to Art. 99 AI Act. The CIVAC workspace template “AI Act Inventory” organises per system: risk class, provider/operator role, list of obligations, deadlines, responsible role, level of evidence and next review. You can find more background on the schedule in our Overview of the EU AI Act obligations. The responsible market surveillance authority in Germany, within the scope of federal competencies, will be the Federal Network Agency, supplemented by sector-specific authorities such as BaFin and BSI.

Art. 32 GDPR requires a risk-appropriate level of technology. A minimum list of technical and organisational measures has been established for generative AI. Identity and authorisation management with single sign-on, multi-factor authentication and conditional access prevents shadow accounts and unaudited tool use. Data loss prevention on end devices and in the browser blocks entries of credit card numbers, health insurance numbers, social security numbers or source code in non-approved tools. Logging and auditability of prompts is mandatory in regulated industries; storage is separated according to purpose and client. Role-based model selection separates non-critical standard applications from highly sensitive workloads, for example via separate tenants or model routing.

At the model level, this includes training opt-out, retention control (data is deleted after a session or not saved at all), encryption in transit (TLS 1.2 or higher) and at rest (AES-256), customer key management via KMS or HSM, protection against prompt injection through input sanitization and against model hallucination through downstream validation or retrieval augmented generation with reliable sources. At the process level, this includes penetration tests, regular effectiveness testing of the TOM at least annually, a well-established data breach reporting chain in accordance with Art. 33 GDPR with a 72-hour deadline and a documented procedure for supervisory inquiries. The 93 controls according to ISO/IEC 27001:2022 can mostly be mapped one-to-one to AI workflows; Annex A.5 (Information Security Policies), A.8 (Asset Management), A.5.34 (Privacy and PII) as well as the new controls A.5.23 Cloud Services and A.8.28 Secure Coding are central nodes. In the CIVAC workspace you maintain the TOM as a living inventory, each entry with the person responsible, inspection date and evidence attachment. The external information security officer interlinks the TOM with the ISMS and ensures that GDPR and ISO 27001 do not run in parallel, but point to each other. This creates a uniform control system with a common reporting line to management from two obligations.

Three obligations are regularly underestimated in practice with generative AI: data breaches, rights of those affected and deletion. Art. 33 GDPR requires a report “immediately, if possible within 72 hours” of becoming aware of a data breach to the responsible supervisory authority, and if the risk is high, also to those affected in accordance with Art. 34. A data breach in AI can occur through accidental entry of sensitive data into a public model, through a log leak at the provider, through prompt injection with data exfiltration, through incorrect client isolation in multi-tenant architectures or through incorrect data breaches Addressees in an AI-generated mailing. The reporting chain must be well established between the department, IT security, DPO, board of directors and supervisory authority. For companies subject to NIS 2, the 24-hour early warning to the BSI in accordance with Section 32 BSIG is added in parallel, followed by the 72-hour follow-up report and the final report after one month.

The rights of those affected under Articles 15 to 22 of the GDPR are demanding for large language models. Information, correction and deletion cannot be easily fulfilled against a model weight. In practice, a distinction is made: inputs (prompts) and outputs are classic processing, here all rights apply without restriction. Training or fine-tuning data requires AVV clauses with opt-out and documented deletion workflows from the provider. Automated individual decisions according to Art. 22 GDPR are only permitted under strict conditions; A final human decision with documented reasons is usually the simplest, legally compliant solution. The CIVAC workspace template “Affected Inquiry” guides you through the 30-day processing period according to Art. 12 GDPR, documents communication and decisions, archives attachments in an audit-proof manner and automatically creates an activity history. With regard to the EU AI Act, the “Serious Incident” template supplements the reporting obligations to the responsible market surveillance authority in accordance with Article 73, currently with a 15-day deadline for high-risk systems.

AI projects rarely fail because of the technology, more often because of the evidence. If you want to convince an auditor, a client in the procurement process, an external auditor or a supervisory authority, you need less a glossy strategy than an orderly, dated and comprehensibly accountable documentation: AI use cases as an inventory, DPIA with date and signature of the DSB, AVVs with all providers and a documented subprocessor list, TOM with effectiveness test, AI guideline and proof of training in accordance with Art. 4 AI Act, works agreement with the works council, Reporting paths for data breaches and serious incidents under the AI Act. These documents need to be alive, not yellowing in a filing cabinet.

CIVAC is the compliance platform and officer-as-a-service for exactly this requirement. In the workspace you will find 490 ready-to-use audit templates, including AI Use Case, DPIA, AVV Audit, TOM Inventory, AI Policy, Training Evidence, AI Operating Agreement and AI Act Inventory. EU data residency, 93 controls according to ISO/IEC 27001:2022, audit-proof storage and reporting line to management are built in. The appointment certificate for the external data protection officer is issued within the CIVAC SLA of 2 working days, compared to 2 to 6 weeks in the classic market. Licence the workspace for your internal representatives or have our representatives order it. Turn reading into an assignment. Write to info@civac.de or use the contact form on civac.de, we will respond on the same working day with a proposal, appointment and draft appointment certificate. An initial meeting usually takes 30 minutes, the complete onboarding workshop takes half a day, and the appointment of external representatives is then completed in 48 hours. You also get a permanent main contact person, a documented escalation level for the weekend incident and a quarterly reporting obligation to management with status traffic lights and key figures. The appointment certificate, signed, filed, verifiable.