Standards Development

Scope

This document provides requirements and technical formats for logging of AI systems in accordance with the record keeping requirements in the AI act.

It provides a technical format for conformity assessment inputs. Wherever possible, it will use definitions and metrics defined in existing standards.

It defines a syntactical and semantic definition for log formats, including machine-readable schemas. Finally, it establishes recommendations on how data can be transmitted and stored for post-market monitoring purposes.

Defining substantial modification is not in scope.

Purpose

Requirements in the draft AI Act

Based on the draft AI Act submitted to coreper on 3rd November 2022, this expert observes the following requirements:

Recital (33) - Technical inaccuracies of AI systems intended for the remote biometric identification of natural persons can lead to biased results and entail discriminatory effects. This is particularly relevant when it comes to age, ethnicity, race, sex or disabilities. Therefore, ‘real-time’ and ‘post’ remote biometric identification systems should be classified as high-risk. In view of the risks that they pose, both types of remote biometric identification systems should be subject to specific requirements on logging capabilities and human oversight.

Article (14):

1. High-risk AI systems shall technically allow for the automatic recording of events (‘logs’) over the duration of the life cycle of the system

2. In order to ensure a level of traceability of the AI system’s functioning that is appropriate to the intended purpose of the system, logging capabilities shall enable the recording of events relevant for

(i) identification of situations that may result in the AI system presenting a risk within the meaning of Article 65(1)1 2 or in a substantial modification;

(ii) facilitation of the post-market monitoring referred to in Article 613; and

(iii) monitoring of the operation of high-risk AI systems referred to in Article 29(4).

4 (sic). For high-risk AI systems referred to in paragraph 1, point (a) of Annex III, the logging capabilities shall provide, at a minimum:

(a) recording of the period of each use of the system (start date and time and end date and time of each use);

(b) the reference database against which input data has been checked by the system;

(c) the input data for which the search has led to a match;

(d) the identification of the natural persons involved in the verification of the results, as referred to in Article 14 (5).

Analysis

The Chapter II requirements say that logs are relevant throughout the lifecycle. This can be interpreted to mean that the outputs of conformity assessment may be in scope. On the other hand, the lifecycle can be interpreted to mean after conformity assessment. Even so, when the system is updated the outputs of logging from past operation may be relevant to the updated conformity assessment (following substantial modification).

Noting that the AI act does not directly link conformity assessment and marketing surveillance / record-keeping. However, given the technical data will have at least some commonality, it would seem prudent to at least have a common ontology and schema.

The purpose of the logging includes to support monitoring of operations, and the specific risks posed by the system are relevant to the logging requirements. Logging based on the actual risks is not possible, but the format should be suitably extensible to allow for customisation.

Logging shall identify natural persons involved in the verification of results in relation to biometrics systems.

The post-market monitoring system needs to support input provided by users or other sources of data.

Logging and conformity assessment technical inputs/outputs may provide some of this data.

Stakeholder consultation

Consultation with AI auditors already performing assurance work, and a proposed AI notified body, the following problems are noted:

- The capacity and capability of assurance actors;

- The potential frequency of conformity assessment; 

- Communication issues with regard to the definitions of metrics and the transmission of technical conformity assessment inputs;

- Technical issues with regard to the traceability of machine learning models.

Additional points

It can also be construed that a real-time logging and monitoring solution may be required in some cases. This may be required to provide outputs to regulatory actors such as conformity assessment bodies in real time. This may significantly enhance the effectiveness of monitoring of high-risk systems.

Additionally, increased automation of conformity assessment and monitoring can help reduce the financial burden upon SMEs, and conformity assessment bodies. This could be especially true if the standard is not entirely provided in PDF form, but also as a machine-readable schema with a supporting open source library4.

Finally, the security aspects of such a solution are likely to be critical.

Consultation with the EC

This proposal was briefly discussed with the EC observers at JTC 21 in Brussels. The conversation indicated that (a) the alignment between conformity assessment and post-market monitoring may make sense; (b) realtime transmission of the data may not be required, depending on the situation, more of a storage and “blackbox” approach where data is retrieved when needed; (c) if this can indeed support SMEs it will be very welcome.

Liaison with ISO/IEC SCs

ISO TC 46 / SC 11 Archives / Records Management is considering a PWI in a similar area, joint work would be beneficial with regard to many aspects, including the evidential value of logging. There may also be value in liaison with ISO/IEC JTC 1/SC 27/WG 5 – Identity management and privacy technologies.

Proposed Structure

This proposal is for a single PWI to explore this topic further. However, discussion in JTC 21 / WG 3 indicated support for a multi-part series, e.g. :

xxxx-1 : AI system logging – record keeping requirements [potential harmonized standard]

xxxx-2 : AI system logging – conformity assessment requirements