Magazine #3 | Autumn 2023
How Sustainable Is My AI?
As part of the SustAIn project, we have done pioneering work in compiling comprehensive indicators that can be used to assess the social, environmental and economic sustainability of AI systems. Using a Self-Assessment Tool, organizations that develop AI in-house or purchase it externally can now test the sustainability of their AI systems with a digital app.
Everyone is talking about Artificial Intelligence, and not just because of the European Union’s planned AI regulation. But this also raises the increasingly pressing question of how sustainable the AI systems used by companies and organizations are. The resource and especially energy consumption of AI can be immense, and in times of an energy and climate crisis, we cannot sweep its ecological impact under the rug. The systems also leave much to be desired in terms of social and economic sustainability. Within the AI industry, market power is concentrated among a few large companies, leading to barriers to entry for smaller firms. Furthermore, exploitative working conditions prevail along the entire AI value chain and decisions automated with the help of AI can lead to discrimination or to a cultural dominance of Western values, which AI systems latently propagate around the world.
With our project “SustAIn: The Sustainability Index for Artificial Intelligence”, we have presented the very first comprehensive blueprint with which the sustainability of AI systems can be assessed and improved. Through it, we would like to make a contribution to ensuring that sustainability is put into practice in the development and use of AI. With our Self-Assessment Tool, we are now providing organizations with a questionnaire that can help determine how sustainable their AI systems are. Our traffic-light color scheme helps companies rank their answers, and we also provide recommendations on how to make systems more sustainable.
The Criteria Catalog
The basis for the questionnaire used in the Self-Assessment Tool comes from the criteria and indicators developed as part of the SustAIn project, all of which reflect the current state of discussion on the social, environmental and economic sustainability of AI systems. We have identified 13 overarching criteria for assessing the sustainability of AI systems. These criteria have been broken down into over 40 indicators, which have been deployed in a way that makes them practical to use.
An Example
To provide one example: The social sustainability criterion “self-determination and data protection” includes the indicator “ensuring informational self-determination”. This self-determination can be implemented by letting those affected by AI systems know how their personal data will be used by the systems. They should be given control over the use of this data with, for example, opt-in or opt-out functions. In the current discussions about the mass use of protected data to train large language models and chatbots like ChatGPT, it has become clear time and again how important such approaches are for preventing personal data from being misused or copyrights from being infringed upon.
The 13 Sustainability Criteria for AI Systems
Transparency and Accountability
Non-Discrimination and Fairness
Technical Reliability and Human Supervision
Self-Determination and Data Protection
Inclusive and Participatory Design
Cultural Sensitivity
Market Diversity and Exploitation of Innovation Potential
Distribution Effect in Target Markets
Working Conditions and Jobs
Energy Consumption
CO2 and Greenhouse Gas Emissions
Sustainability Potential in Application
Indirect Resource Consumption
The Questionnaire
The questionnaire for the Self-Assessment Tool was developed based on this set of criteria, indicators and deployments. Not all of the questions included in it are relevant to all organizations that are developing or using AI. If, for example, an organization produces AI systems that do not make direct decisions about people but are used only in industrial processes, questions about their fairness become unnecessary. Organizations must answer between 48 and a maximum of 66 questions in total to obtain an assessment of the sustainability of their AI systems. They must indicate, for example, whether they follow a Code of Conduct that lists fundamental values and standards in the implementation and use of AI systems, or whether sustainability certifications are available for the data centers and hardware they use. Since the questionnaire is based on a broad understanding of sustainability, it covers a wide variety of areas. Presumably, the questions will have to be answered by multiple departments in the organization.
*Note: Unfortunately, the self-assessment tool is currently only available in German.
The Appraisal
The results of the Self-Assessment Tool are made available to organizations as a PDF download at the end of the questionnaire. They are not visible to third parties and are not stored. Our graphics are intended to provide guidance to organizations on which areas of their AI system are faring well in terms of sustainability and where there is room for improvement. The questionnaire is evaluated based on a point system. Due to the complexity and multilayered nature of the topic, we have decided against calculating a sustainability score from the responses. Instead, we provide organizations with a granular, differentiated assessment that does not aim to provide AI products with a “green” label, instead seeking to make the organizations that develop or use such systems aware of their sustainability.
The results of the Self-Assessment Tool are made available to organizations as a PDF download at the end of the questionnaire. Our graphics are intended to provide guidance to organizations on which areas of their AI system are faring well in terms of sustainability and where there is room for improvement.
The Recommendations
Our online Self-Assessment Tool also provides organizations with concrete recommendations for action. There is no ready-made recipe for the sustainability of AI, and decisions on how to make them more sustainable must be made on a case-by-case basis. Beyond simple technical measures, numerous micro decisions must also be made in the planning and development process. The most important factor is a cultural shift at the organizational level, through which sustainability becomes a guiding principle for all decisions that are made. Such a shift must be planned, moderated, executed, evaluated and technically implemented. Our tool is designed to pave the way for that to happen.
What Comes Next?
Through the AI Act, the EU is pushing for sustainable AI systems. But even apart from such regulatory approaches, all organizations should address the sustainability of their AI systems. Indeed, we need to act now before unsustainable AI ecosystems have become established to the point that they are difficult to modify. In expectation of further regulatory action, various industry standards are already being renegotiated against the backdrop of demands for AI system sustainability. We hope to bolster that development with the sustainability approach conceived in the SustAIn project.
Read more
How sustainable is your AI?
Use SustAIn's self-assessment tool for your AI
*Note: Unfortunately, the self-assessment tool is currently only available in German.
