The previous example raises a question:

• Given that AI models can produce inaccurate results due to a lack of generalizability, shouldn’t a human check whether the AI model correctly detected phone usage?

According to principles for responsible AI, responsible AI systems should allow humans to monitor, assess, and freely adopt or override the decisions made by such systems, while decisions made automatically without human involvement should be avoided. AI systems should support human autonomy and decision-making.

For example, the Dutch police’s AI system should provide mechanisms for humans to monitor, assess, and freely adopt or override the decisions made by the system.

However, human supervision presents many challenges, such as:

• Bias automation: Humans tend to accept AI system outputs uncritically. This problem is exacerbated by a lack of professionals with the interdisciplinary knowledge required to effectively supervise complex AI systems. Cognitive and psychological limitations also play a role, as the demands of interpreting complex AI outputs and maintaining attention over extended supervision can lead to fatigue or errors.

• Autonomy of AI systems: The autonomy of AI systems further complicates the issue, as these technologies often operate with minimal human intervention, making real-time monitoring difficult. This challenge is amplified by the scale at which AI systems operate, processing massive amounts of data and performing tasks at a speed beyond human cognitive capacity, making continuous human supervision impossible.

• Data-related issues: Poor data quality, biases in accurately representing relevant features in training datasets, and systemic inequalities can affect the reliability and ethical alignment of AI. Supervision becomes particularly difficult when these biases are subtle or deeply embedded and require sophisticated methods to detect and mitigate. Additionally, the ethical and societal impacts of decisions based on AI models, especially in areas like healthcare, access to social systems, and criminal justice, require nuanced supervisory mechanisms, which are often underdeveloped or entirely absent.