Risk based approaches to A1 - Assertion 4

Risk-based approaches should be a part of assessing whether an AI or machine learning model should be used for making certain decisions, or for determining what additional controls need to be in place.

Formal frameworks for risk management have generally been implemented in most large companies. Either consciously or sub-consciously, we make decisions taking account of the risks involved. At their most formalised, risk management frameworks include an explicit definition of risk appetite and tolerances and formalised practices to identify, measure, manage and report on levels of risk against that appetite.

By nature, machine learning systems are based on algorithms that are complex and unpredictable, which introduces additional risks to be managed and adds some complexity to those already being managed. As an example, there is a risk that customers and competitors who know about and who are impacted by the implementation of a machine learning model may change their behavior, and in doing so distort the inputs to the model. This often happens in models that aim to predict fraud, multi-party competitive scenarios and in cybersecurity, whether it is through deliberate behavior change in order to circumvent the prediction or an unintended resulting impact. Consequently, because the nature of risks changes in such a way, the integrity of the models being used will degrade faster than it would in an uncompromised environment.13

Machine learning algorithms are typically developed to improve functional performance, and to provide the ‘best’ possible response to a question that humans would either not be able to answer, or at least not answer quickly. This involves a machine, built with complex functions that often will not provide visibility or insight into the logic followed, or the structure of the decision process. Moreover, considering that the machine learning algorithms are trained with input data generated by people, the algorithm’s decision-making process is characterised by the same bias that applies to human decisions, and influenced by the culture, assumptions, points of view and stereotypes of people.14

Typically, traditional approaches to risk management have not been structured to accommodate the kinds of variables that are introduced by AI and machine learning. They do not take account of the ‘black box’ effect that exists in Opaque AI and machine learning models. Accordingly, companies will need to update their risk management frameworks to consider factors such as data ethics and to align that with their corporate values, or a similar demonstrable measure, in order to be able to justify and explain to individuals the intent behind the companies’ use of data

Google, Microsoft and many other companies have defined principles around fairness, safety,benefits to humanity, and other similar principles in an effort to define up-front whether a project should be undertaken. As is inevitable in using those terms, the measurement of those benefits will vary depending upon each company and who within them applies the principle. While such principles can be viewed in similar terms to ‘data ethics’, when applied to AI and machine learning projects they need to be embodied within a broader risk framework that considers a broad range of criteria including business risk, model risk, reputational risk , data accuracy to properly determine the risks associated with such initiatives.

Another key aspect is business continuity in the context of adoption of AI and machine learning, in that a company must still be able to carry on business whenever the model fails to operate.

A key element of a company’s overall approach to risk management needs to be the development of a governance framework that allows it to evaluate and evidence a range of considerations: suitable corporate responsibility; the application of appropriate behaviour towards its customers and employees; and compliance with any relevant current and future regulatory requirements.

Send me a message if you are interested in the complete paper and I will be happy to provide it with all of the contained reference sources.