III. Possible Solutions: Data Protection Violations in Connection with AI
Completion requirements
3. Data Protection
How can we address the privacy risks arising from AI systems?
Jamillah Knowles / Better Images of AI / Data People / Licenced by CC-BY 4.0
The industry and researchers continuously investigate and develop techniques that could be applied to minimize privacy risks in the context of AI systems. Some examples of these techniques are anonymization, pseudonymization, encryption, differential privacy, and federated learning.
Regarding regulations, the EU General Data Protection Regulation (GDPR) and the EU AI Act support this process.
Regarding the EU General Data Protection Regulation (GDPR):
User Consent
To ensure that individuals in the EU are informed about the processing of their data and can give consent, the EU adopted the General Data Protection Regulation (GDPR) in 2016, which became enforceable in May 2018.
Based on the GDPR, individuals, companies, and organizations must comply with strict requirements when processing (i.e., collecting, storing, and managing) personal data of people in the EU. These individuals must be informed in clear and understandable language about how their data is processed so that they can understand what they are consenting to.
According to the GDPR, individuals in the EU whose data is to be processed have the following rights:
- Right to Information: Before personal data is collected, individuals have the right to be informed about who is collecting the data, for what purpose it is used, and how it is processed.
- Right of Access: After personal data is collected, individuals have the right to request access to their data and supplementary information.
- Right to Rectification: Individuals have the right to have their personal data corrected if it is inaccurate or incomplete.
- Right to Erasure (Right to be Forgotten): Under certain circumstances, individuals can request that the data controller delete their personal data.
- Right to Data Portability: Individuals can request that personal data they have provided, where processed by automated means based on consent or a contract, be returned to them or transferred to another company.
- Right to Object: Individuals can object to the processing of their personal data and request that a company or organization stop processing their data, especially for direct marketing purposes, including profiling.
Read more in: https://gdpr-info.eu/chapter-3/.
Data Security Measures and Notification Obligations in Case of Data Breaches
The GDPR recognizes the risks of data breaches when processing personal data and places responsibility for appropriate technical and organizational measures to protect the data on the controller and processor.
In the event of a data breach, the organization holding the personal data must notify the supervisory authority within 72 hours of becoming aware of the breach (Read more in: https://gdpr-info.eu/art-33-gdpr/). If the breach is likely to pose a high risk to the rights and freedoms of the affected individuals and this risk is not mitigated, the affected individuals must also be informed (read more in: https://gdpr-info.eu/art-34-gdpr/).
Profiling and Automated Decision-Making
Regarding profiling and automated decision-making systems, the GDPR ensures that individuals have the right not to be subject to a decision based solely on automated processing, unless they have explicitly consented, or it is necessary for a contract, or it is authorized by EU or Member State law (Read more in: https://gdpr-info.eu/art-22-gdpr/).
In such cases, the organization must inform the individual about the automated decision-making, give them the right to have the automated decision reviewed by a person, and provide the opportunity to challenge the decision.
An example: If a bank automates the decision to grant a loan, the person must be informed and given the opportunity to challenge the decision and request a human review.
Regarding the EU AI Act:
The EU's AI Act, came into force on 1. August 2024. It was born on the 13th. March 2024 by the EU Parliament and published in the Official Journal of the EU. The main objective of the EU AI Act is to ensure that the use of AI systems is in line with the fundamental rights protected by the "EU Charter of Fundamental Rights".
The EU AI Act divides AI systems into four risk categories:
- Unacceptable risk: Systems that violate EU values and fundamental rights are prohibited (e.g., social scoring modeled on China).
- High risk: High-risk AI systems are classified in this category because they pose a high risk to the health, safety, or fundamental rights of natural persons. These systems are permitted on the European market but must meet conformity assessment requirements and binding specifications regarding data and data management, documentation and record-keeping, transparency, and the provision of information to users. High-risk AI systems include systems in areas such as medicine, transportation, employment, or law enforcement.
- Limited risk: Systems that must meet transparency requirements (e.g., chatbots that are clearly labeled as such).
- Minimal risk: Applications such as games or AI-assisted word processing tools, which are hardly regulated.
High-risk AI systems must be developed using high-quality training, validation, and test datasets to minimize risks and discriminatory outcomes. Providers of high-risk AI systems may, in exceptional cases, process special categories of personal data to detect and correct biases related to high-risk AI systems. However, the following conditions must be met:
(a) the bias detection and correction cannot be effectively fulfilled by processing other data, including synthetic or anonymised data;
(b) the special categories of personal data are subject to technical limitations on the re-use of the personal data, and state-of-the-art security and privacy-preserving measures, including pseudonymisation;
(c) the special categories of personal data are subject to measures to ensure that the personal data processed are secured, protected, subject to suitable safeguards, including strict controls and documentation of the access, to avoid misuse and ensure that only authorised persons have access to those personal data with appropriate confidentiality obligations;
(d) the special categories of personal data are not to be transmitted, transferred or otherwise accessed by other parties;
(e) the special categories of personal data are deleted once the bias has been corrected or the personal data has reached the end of its retention period, whichever comes first;
(b) the special categories of personal data are subject to technical limitations on the re-use of the personal data, and state-of-the-art security and privacy-preserving measures, including pseudonymisation;
(c) the special categories of personal data are subject to measures to ensure that the personal data processed are secured, protected, subject to suitable safeguards, including strict controls and documentation of the access, to avoid misuse and ensure that only authorised persons have access to those personal data with appropriate confidentiality obligations;
(d) the special categories of personal data are not to be transmitted, transferred or otherwise accessed by other parties;
(e) the special categories of personal data are deleted once the bias has been corrected or the personal data has reached the end of its retention period, whichever comes first;
Here you can read the full Article 10: https://artificialintelligenceact.eu/article/10/.
Profiling Systems and Automated Decision Systems
Regarding AI systems used to assess the creditworthiness of natural persons, the current AI Act classifies these systems as high-risk AI systems, as they determine individuals’ access to financial resources or essential services such as housing, electricity, and telecommunications.
Biometric Recognition Systems
Regarding biometric AI systems, the current AI Act prohibits the use of AI systems for “real-time” remote biometric identification of individuals for law enforcement purposes in publicly accessible spaces, except in specific situations.
Biometric AI systems not intended for law enforcement but used for “real-time” or “after-the-fact” remote biometric identification of individuals should be classified as high-risk AI systems.