Do We Need Explainable AI in Companies? Investigation of Challenges, Expectations, and Chances from Employees' Perspective

by   Katharina Weitz, et al.

By using AI, companies want to improve their business success and innovation chances. However, in doing so, they (companies and their employees) are faced with new requirements. In particular, legal regulations call for transparency and comprehensibility of AI systems. The field of XAI deals with these issues. Currently, the results are mostly obtained in lab studies, while the transfer to real-world applications is lacking. This includes considering employees' needs and attributes, which may differ from end-users in the lab. Therefore, this project report paper provides initial insights into employees' specific needs and attitudes towards (X)AI. For this, the results of a project's online survey are reported that investigate two employees' perspectives (i.e., company level and employee level) on (X)AI to create a holistic view of challenges, risks, and needs of employees. Our findings suggest that AI and XAI are well-known terms perceived as important for employees. This is a first step for XAI to be a potential driver to foster the successful usage of AI by providing transparent and comprehensible insights into AI technologies. To benefit from (X)AI technologies, supportive employees on the management level are valuable catalysts. This work contributes to the ongoing demand for XAI research to develop human-centered and domain-specific XAI designs.


How to design an AI ethics board

Organizations that develop and deploy artificial intelligence (AI) syste...

Big Tech Companies Impact on Research at the Faculty of Information Technology and Electrical Engineering

Artificial intelligence is gaining momentum, ongoing pandemic is fuel to...

Explainable AI, but explainable to whom?

Advances in AI technologies have resulted in superior levels of AI-based...

AI Transparency in the Age of LLMs: A Human-Centered Research Roadmap

The rise of powerful large language models (LLMs) brings about tremendou...

Building AI Innovation Labs together with Companies

In the future, most companies will be confronted with the topic of Artif...

What Do End-Users Really Want? Investigation of Human-Centered XAI for Mobile Health Apps

In healthcare, AI systems support clinicians and patients in diagnosis, ...

An Exploratory Study of AI System Risk Assessment from the Lens of Data Distribution and Uncertainty

Deep learning (DL) has become a driving force and has been widely adopte...

Please sign up or login with your details

Forgot password? Click here to reset