Security, privacy, and agentic AI in a regulatory view: From definitions and distinctions to provisions and reflections
Organizations: Department of Informatics, University of Oslo, Norway
Abstract
The rapid proliferation of artificial intelligence (AI) technologies has led to a dynamic regulatory landscape, where legislative frameworks strive to keep pace with technical advancements. As AI paradigms shift towards greater autonomy, specifically in the form of agentic AI, it becomes increasingly challenging to precisely articulate regulatory stipulations. This challenge is even more acute in the domains of security and privacy, where the capabilities of autonomous agents often blur traditional legal and technical boundaries. This paper reviews the evolving European Union (EU) AI regulatory provisions via analyzing 24 relevant documents published between 2024 and 2025. From this review, we provide a clarification of critical definitions. We deconstruct the regulatory interpretations of security, privacy, and agentic AI, distinguishing them from closely related concepts to resolve ambiguity. We synthesize the reviewed documents to articulate the current state of regulatory provisions targeting different types of AI, particularly those related to security and privacy aspects. We analyze and reflect on the existing provisions in the regulatory dimension to better align security and privacy obligations with AI and agentic behaviors. These insights serve to inform policymakers, developers, and researchers on the compliance and AI governance in the society with increasing algorithmic agencies.
Figures & tables
| Short name of the document | Publish date | Full name and link to the document |
|---|---|---|
| EC communication on Unlocking Data for AI | 2025-11-19 | European Commission Communication from the commission to the European Parliament and the Council Data Union Strategy Unlocking Data for AI ( link ) |
| Proposal for Digital Omnibus on AI | 2025-11-19 | European Commission Proposal for a Regulation of the European Parliament and of the Council amending Regulations (EU) 2024/1689 and (EU) 2018/1139 as regards the simplification of the implementation of harmonised rules on artificial intelligence (Digital Omnibus on AI) ( link ) |
| EC Commission Staff Working Document on Digital Omnibus on AI | 2025-11-19 | Commission Staff Working Document Accompanying the documents Proposal for a Regulation of the European Parliament and of the Council Amending Regulations (EU) 2016/679, (EU) 2018/1724, (EU) 2018/1725, (EU) 2023/2854 and Directives 2002/58/EC, (EU) 2022/2555 and (EU) 2022/2557 as regards the simplification of the digital legislative framework, and repealing Regulations (EU) 2018/1807, (EU) 2019/1150, (EU) 2022/868, and Directive (EU) 2019/1024 (Digital Omnibus) Amending Regulations (EU) 2024/1689 and (EU) 2018/1139 as regards the simplification of the implementation of harmonised rules on artificial intelligence (Digital Omnibus on AI) ( link ) |
| Decision on Draft Recommendation on equality and AI | 2025-11-17 | Council Decision (EU) 2025/2350 of 13 November 2025 on the position to be taken on behalf of the European Union within the Committee of Ministers of the Council of Europe on the Draft Recommendation on equality and artificial intelligence ( link ) |
| EC communication on Apply AI Strategy | 2025-10-08 | European Commission Communication from the commission to the European Parliament and the Council Apply AI Strategy ( link ) |
| EC communication on European Strategy for AI in Science | 2025-10-08 | European Commission Communication from the commission to the European Parliament and the Council A European Strategy for Artificial Intelligence in Science Paving the way for the Resource for AI Science in Europe (RAISE) ( link ) |
| Short name of the document | Publish date | Full name and link to the document |
|---|---|---|
| Opinion on ethical AI and access to supercomputing for start-ups | 2024-12-04 | Opinion of the European Committee of the Regions – Ethical Artificial Intelligence and access to supercomputing for start-ups ( link ) |
| Decision on signing Europe framework on AI, human rights, democracy and the rule of law | 2024-09-04 | Council Decision (EU) 2024/2218 of 28 August 2024 on the signing, on behalf of the European Union, of the Council of Europe Framework Convention on Artificial Intelligence and Human Rights, Democracy and the Rule of Law ( link ) |
| Regulation (EU) 2024/1689 (AI Act) | 2024-07-12 | Regulation (EU) 2024/1689 of the European Parliament and of the Council of 13 June 2024 laying down harmonised rules on artificial intelligence and amending Regulations (EC) No 300/2008, (EU) No 167/2013, (EU) No 168/2013, (EU) 2018/858, (EU) 2018/1139 and (EU) 2019/2144 and Directives 2014/90/EU, (EU) 2016/797 and (EU) 2020/1828 (Artificial Intelligence Act) (Text with EEA relevance) ( link ) |
| Proposal for signing Europe framework on AI, human rights, democracy and the rule of law | 2024-06-26 | Proposal for a Council Decision on the signing, on behalf of the European Union, of the Council of Europe Framework Convention on Artificial Intelligence, Human Rights, Democracy and the Rule of Law ( link ) |
| Council Regulation (EU) 2024/1732 in EuroHPC initiative to boost trustworthy AI | 2024-06-19 | Council Regulation (EU) 2024/1732 of 17 June 2024 amending Regulation (EU) 2021/1173 as regards a EuroHPC initiative for start-ups in order to boost European leadership in trustworthy artificial intelligence ( link ) |
| Report on EU AI ambitions | 2024-05-31 | Special report 08/2024: EU Artificial intelligence ambition – Stronger governance and increased, more focused investment essential going forward ( link ) |
| AI concept | Regulatory definition | Exemplification |
|---|---|---|
| AI system | A machine-based system that is designed to operate with varying levels of autonomy and that may exhibit adaptiveness after deployment, and that, for explicit or implicit objectives, infers, from the input it receives, how to generate outputs such as predictions, content, recommendations, or decisions that can influence physical or virtual environments. (EU AI Act, Article 3 (1)) | Spam filter, intelligence traffic control system, automatic grammar check, etc . |
| LLMs | Advanced AI models that excel in understanding and generating human-like language. (EC Communications on boosting startups and innovation in trustworthy AI, 3.2) | Llama, OpenAI, and other models from HuggingFace. |
| GAI | Systems such as sophisticated large language models that can create new content, ranging from text to images, by learning from extensive training data. (Opinion on harnessing the potential and mitigating the risks of AI, chapter 2) | Segment Anything Playground, Midjourney, etc . |
| GPAI model | An AI model, including where such an AI model is trained with a large amount of data using self-supervision at scale, that displays significant generality and is capable of competently performing a wide range of distinct tasks regardless of the way the model is placed on the market and that can be integrated into a variety of downstream systems or applications, except AI models that are used for research, development or prototyping activities before they are placed on the market. (EU AI Act, Article 3 (63)) | ChatGPT Base Model, Gemini 3 Pro. Large GAI models are also a typical example for a GPAI model. |
| GPAI system | An AI system which is based on a general-purpose AI model and which has the capability to serve a variety of purposes, both for direct use as well as for integration in other AI systems. (EU AI Act, Article 3 (66)) | ChatGPT Enterprise, Google Gemini, MS Copilot, etc . |
| Agentic AI | AI systems that can independently make decisions and take actions. This enables agents to understand language, reason about tasks, take actions autonomously to achieve predefined objectives, and interact with the world around them, orchestrating interactions including with humans. (EC Communication on Unlocking Data for AI, chapter 2) | Devin AI, Google Colab. |
| Concept | Regulatory definition | Exemplification |
| Fundamental right to personal data protection | Referred to GDPR (Proposal for signing Europe framework on AI, human rights, democracy and the rule of law) Referred to Article 8(1) of the Charter of Fundamental Rights of the European Union (Regulation (EU) 2016/679) Everyone has the right to the protection of personal data concerning him or her. Such data must be processed fairly for specified purposes and on the basis of the consent of the person concerned or some other legitimate basis laid down by law. Everyone has the right of access to data which has been collected concerning him or her, and the right to have it rectified (Charter of Fundamental Rights of the European Union) | N/A |
| Privacy and data governance | AI systems shall be developed and used in compliance with existing privacy and data protection rules, while processing data that meets high standards in terms of quality and integrity (Amendments adopted for the proposal on AI Act) | N/A |
| Personal data | The definition is referred to Article 4, point (1), of Regulation (EU) 2016/679 (EU AI Act) Any information relating to an identified or identifiable natural person (‘data subject’); an identifiable natural person is one who can be identified, directly or indirectly, in particular by reference to an identifier such as a name, an identification number, location data, an online identifier or to one or more factors specific to the physical, physiological, genetic, mental, economic, cultural or social identity of that natural person (Regulation (EU) 2016/679) | Phone number, ID number, name, home address, bank account, biometric data like facial images or dactyloscopic data |
| Non-personal data | The definition is referred to as data other than personal data as defined in Article 4, point (1), of Regulation (EU) 2016/679 (EU AI Act) | Locally produced weather forecast, energy consumption prediction, demand forecast of buildings |
| Security | Their is no universal, stand‑alone definition of security particularly for AI and information system, to the best of our knowledge. Below we provide the most relevant regulatory definitions closely related to security | N/A |
| Network and information security | The ability of a network or an information system to resist, at a given level of confidence, accidental events or unlawful or malicious actions that compromise the availability, authenticity, integrity and confidentiality of stored or transmitted data and the related services offered by or accessible via those networks and systems (Regulation (EU) No 526/2013) | N/A |
| AI concepts | Privacy and personal data protection provisions (not a exhaustive list) |
|---|---|
| AI systems | AI systems should make best efforts to respect general principles … in line with the Charter of Fundamental Rights of the European Union …, including the protection of fundamental rights, human agency and oversight, technical robustness and safety, privacy and data governance , …. (Amendments adopted for the proposal on AI Act) The indiscriminate and untargeted scraping of biometric data from social media or CCTV footage to create or expand facial recognition databases add to the feeling of mass surveillance and can lead to gross violations of fundamental rights, including the right to privacy . The use of AI systems with this intended purpose should therefore be prohibited (Amendments adopted for the proposal on AI Act) Throughout the recruitment process … of persons …, such systems may perpetuate historical patterns of discrimination, for example against women, certain age groups, … . AI systems used to monitor the performance and behaviour of these persons may also undermine the essence of their fundamental rights to data protection and privacy . This Regulation applies without prejudice to Union and Member State competences to provide for more specific rules for the use of AI-systems in the employment context (Amendments adopted for the proposal on AI Act) … for the training, validation and testing of AI systems … the European health data space will facilitate non-discriminatory access to health data and the training of artificial intelligence algorithms on those datasets, in a privacy-preserving , secure, timely, transparent and trustworthy manner (Amendments adopted for the proposal on AI Act) The right to privacy and to protection of personal data must be guaranteed throughout the entire lifecycle of the AI system. … data minimisation and data protection by design and by default … are essential when the processing of data involves significant risks to the fundamental rights of individuals. Providers and users of AI systems should implement state-of-the-art technical and organisational measures … . Such measures should include not only anonymisation and encryption, but also the use of increasingly available technology … (Amendments adopted for the proposal on AI Act) To the extent that it is strictly necessary for the purposes of ensuring negative bias detection and correction in relation to the high-risk AI systems, the providers … may exceptionally process special categories of personal data … subject to appropriate safeguards … including technical limitations on the re-use and use of state-of-the-art security and privacy-preserving (Amendments adopted for the proposal on AI Act) Any processing of biometric data and other personal data involved in the use of AI systems for biometric identification, other than in connection to the use of real-time remote biometric identification systems in publicly accessible spaces for the purpose of law enforcement as regulated by this Regulation, should continue to comply with all requirements resulting from Article 10 of Directive (EU) 2016/680 (EU AI Act) … suggests including and documenting privacy and data security measures … in the development and training of AI. The inclusion and documentation of robust privacy and data security measures, including encryption, access controls, and regular audits, protect sensitive data from cyber threats (Opinion on ehtical AI and access to supercomputing for start-ups) |
| GPAI | General-purpose AI models could pose systemic risks … In particular, international approaches have so far identified the need to pay attention to risks … the facilitation of disinformation or harming privacy with threats to democratic values and human rights (EU AI Act) … the AI Act allows providers of high-risk AI systems to exceptionally use sensitive personal data – which is otherwise prohibited by the GDPR – for the purpose of bias detection and correction. This facilitates effective AI training and testing. The possibility of relying on this legal basis should be extended to providers of all AI systems and general-purpose AI models (EC Commission Staff Working Document on Digital Ommibus on AI) |
| AI concepts | Security provisions (not a exhaustive list) |
|---|---|
| AI systems | … AI Act will guarantee the use of trustworthy AI and ensure the transparency, safety and required human oversight. In addition, complementary regulations ensuring cybersecurity and privacy are key … mitigating the risk of potential misuse … particularly in contexts like biowarfare (EC Communication on boosting startups and innovation in trustworthy AI) High-risk AI systems should perform consistently throughout their lifecycle and meet an appropriate level of accuracy, robustness and cybersecurity (EU AI Act) To ensure a level of cybersecurity appropriate to the risks, suitable measures, such as security controls, should therefore be taken by the providers of high-risk AI systems, also taking into account as appropriate the underlying ICT infrastructure (EU AI Act) … the assessment of the cybersecurity risks, associated to a product with digital elements classified as high-risk AI system … should consider risks to the cyber resilience of an AI system as regards attempts by unauthorised third parties to alter its use, behaviour or performance, including AI specific vulnerabilities such as data poisoning or adversarial attacks … (EU AI Act) |
| GAI | Generative AI can exponentially increase the capacity to learn and replicate patterns found in cyber threats … thereby assisting cybersecurity professionals … generative AI can also be used by cybercriminals to organise sophisticated cyber-attacks … internal security actors will also need to be well equipped to address the use of generative AI by cybercriminals (EC Communication on boosting startups and innovation in trustworthy AI) |
| GPAI | The providers of general-purpose AI models presenting systemic risks should be subject … to obligations aimed at identifying and mitigating those risks and ensuring an adequate level of cybersecurity protection, regardless of whether it is provided as a standalone model or embedded in an AI system or a product (EU AI Act) … providers (of general-purpose AI models) should ensure an adequate level of cybersecurity protection for the model and its physical infrastructure, if appropriate, along the entire model lifecycle. Cybersecurity protection related to systemic risks associated with malicious use or attacks should duly consider accidental model leakage, unauthorised releases, circumvention of safety measures, and defence against cyberattacks, unauthorised access or model theft. That protection could be facilitated by securing model weights, algorithms, servers, and data sets, such as through operational security measures for information security , specific cybersecurity policies, adequate technical and established solutions, and cyber and physical access controls … (EU AI Act) |
AI Governance
Artificial intelligence governance studies how laws, institutions, and technical controls shape AI development, deployment, accountability, and social effects.
See all 379 papers