Opening the “black box” with explainable AI – a conversation with Menna El-Assady

Artificial intelligence tools are not a “done-deal”, users should keep some human agency in certain tasks, and the main question “is how do we use AI in the right way, in the right context” –and not if we should use it. Those are some of the key points explained by Dr. Menna El-Assady, assistant professor in the Department of Computer Science and head of the Interactive Visualization & Intelligence Augmentation (IVIA) Lab at ETH Zurich, one of the leading international universities for technology and the natural sciences. She talked to iMEdD at the sidelines of the SNF Nostos 2026, and brought a critical –but largely positive– technical viewpoint to the technological issue that has defined our decade.

Using the method of explainable artificial intelligence (XAI), her goal is to improve human-AI collaboration, that is to say try to “bridge” the divide between them, so that a task is performed better than if it were handled by AI alone or by a human alone. This is what can lead to intelligence augmentation, since human intelligence and artificial intelligence are not equally strong in the same domains. When the conversation led to the work routines of journalists, she noted that methods like argumentation mining can provide extra ways to analyse text, and she also accentuated the need for a measured approach to using AI, while at the same time encouraging media organization to be transparent in trying out tools, and noting that IVIA Lab is open to collaboration in finding solutions in these new-age problems.

What is explainable AI? 

Explainable AI is the study of understanding an AI model’s internal decision-making. It must fulfill two criteria: faithfulness and comprehensibility. First, we cannot just produce an explanation that sounds plausible if it doesn’t reflect what is actually happening inside the model; it must be faithful. Second, it has to be comprehensible to the humans viewing it.

These two essential criteria are often tackled by two different communities. The machine learning community focuses heavily on faithfulness –what we call algorithm-centered explainability. They produce impressive work, but it is often difficult for non-experts to understand. Conversely, the human-centered community creates highly plausible, easily understandable explanations, but we often cannot guarantee they accurately reflect the model’s internal processes. The big challenge is bridging these approaches into a joint model, ensuring the explanations we see are both faithful and comprehensible.

How can we open the black box of AI in a way that is understandable for the wider audience?

Our approach is called contextual interactive explainability. We aim to explain AI models within the specific contexts and to the specific audiences using them. We create interactive models that allow people to ask questions about what is happening in real-time. The core challenge is delivering explainability at the right time, with the right content, and through the right medium.

Should an explanation be verbal or visual? Do you need to understand the model’s internal mechanisms, or do you just need to know if it is hallucinating during your current task? Answering these questions requires understanding the stakeholders. We develop different explainable AI methods for machine learning developers and end-users, modulating the approach based on the audience.

How does your work relate to the journalistic community and the media industry in general?

Part of our work involves augmenting how people use AI. For example, if you are writing a text using a language model, you might not know whether it is hallucinating. We developed a technique –self-consistency checks– that produces small annotations alongside the AI-generated text, indicating whether a piece of information might be fabricated.

On a technical level, instead of asking the model to produce a single outcome, we prompt it to generate 20 or 100 variations of the same query. We then break down the text and check the different claims. For instance, if you ask it for a biography and it claims the person was born in 1991 in one version and 1998 in another, those claims are logically inconsistent. We annotate these inconsistencies using a technique called natural language inference. Journalists don’t necessarily need to verify every fact from scratch; the tool simply flags the logically inconsistent pieces for review. We applied this directly in our “Relic” application at the IVIA Lab.

What could XAI and intelligence augmentation offer to the journalistic community? 

Journalists frequently analyze massive amounts of text. Over the last 15 years, I have worked in a field called argumentation mining, using AI to identify the structure of arguments within texts and networks of spoken data. We collaborated with political scientists and linguists to understand how people structure arguments –which traces back to Aristotle’s concepts of logos, ethos, and pathos. We tried to make these qualitative concepts measurable, which led to our visual interactive tools.

Whether working with journalists, medical professionals, or architects, our approach is to study what a highly skilled expert does, then determine where AI can assist that process and where it might hinder or mislead. We are always eager to collaborate with professionals who have a vision for a tool that doesn’t quite exist yet.

AI models use data found on articles written by journalists, and many journalists use chatbots to streamline processes and ask questions instead of googling them. Is this a form of intelligence augmentation or is it a one-way street in favor of the LLMs?

That is an important question. AI offers incredible utility, but we must not lose our critical thinking. There are many routine tasks I gladly delegate to AI. These are manual, structured, and unambiguous tasks where the input and output are relatively standard, and the cost of an error is low. It would be a mistake not to use AI for these, as it saves immense time.

However, there are other tasks that are ambiguous, have multiple solutions, or rely on our personal morality, style, and judgment. The cost of getting these wrong is high, so we must retain our human agency. If you delegate all your decision-making, your unique writing style, and everything that makes your work distinctly yours, you are no longer the true author.

A recent study examined the distribution of rare words in writing, highlighting how AI tends to overuse certain terms, making texts read as dull and uniform. If you can clearly distinguish between the tasks you should delegate and those you must own, you will find a clear, effective way to use AI.

So, it’s not that much of intelligence augmentation.  

Right. There is a saying floating around: the real danger isn’t artificial intelligence, but human stupidity. We need to figure out which tasks to delegate and whether we are using the right tools. This requires rethinking our workflows to ensure we are engaging with AI correctly. In many professions, we find that our current workflows are inadequate.

In some cases, we should deliberately step away from AI. I often advise students to use AI to draft routine emails faster, but to step away from the computer entirely when they need to engage in deep, critical thinking. Formulating your own thoughts first will almost always yield better results than relying immediately on AI. Knowing when to draw that line is the crucial new skill we all must learn.

You are on the Ιndependent Ιnternational Scientific Panel on AI of the United Nations, and you referenced in April 2026 the issue of provenance. How realistic is it to watermark AI journalistic content? And why is it important to do that?

A major trend right now is figuring out how to watermark AI-generated content, and various techniques exist. Some colleagues are working on methods that calculate the probability that a specific model produced a sequence of words. However, this has a significant caveat: if you run a highly common text, like the U.S. Constitution, through these checkers, they will flag it as 100% AI-generated simply because the text is heavily memorized by the models.

While watermarking is not my primary expertise, it is becoming increasingly vital to verify authorship. More importantly, we must combat misinformation to preserve public trust in journalism and fact-based institutions. Losing that trust would be a massive societal issue.

The audience does not trust journalism, and at the same time the use of chatbots is rising. Also, we don’t trust artificial intelligence that much, according to several polls. How can we exit this cycle? 

I will answer this through the lens of science, and we can bridge that to journalism. In science, publishing a peer-reviewed paper is a cornerstone process. But we are currently facing a crisis: an overwhelming number of papers are published daily, and there are far too few reviewers. It has become almost impossible to keep up with the state of the art. We want to maintain the peer-review model, but we need to rethink its execution.

For instance, we could introduce “live papers” that continuously update with new results rather than serving as static final outcomes, and we should discuss failures as openly as successes. The scientific publishing model must evolve because it is no longer sustainable.

For journalism, I would ask: What are the equivalent structures that need rethinking? Do we want journalists to focus primarily on investigative reporting and establishing facts –which is more crucial than ever? Or should they focus on framing arguments, breaking down echo chambers, and creating a shared reality? In an age where AI and social media isolate people into echo chambers, we have the technological tools to produce better journalism, but we must be deliberate about how we redesign the field.

If you were advising a newsroom that is heavily using AI or even creating its own tool, which some news organizations have done, what’s the one mistake you tell them to avoid or the one thing that you say to them to encourage? 

I would advise them to empower their journalists. Provide them with the resources needed for high-quality investigative work and train them to use AI critically. Do not just buy new tools and drop them into the newsroom. The focus should be on teaching people to critically judge when and how to use specific tools.

I strongly advise against outright forbidding certain technologies. When management bans tools, you inevitably see the rise of “shadow AI,” where people use them anyway, but in an unauthorized and unmonitored manner.

People often treat AI as a “take it or leave it” product dictated by big tech companies. But professionals have much more agency than they realize. I am not a journalist, so I don’t know exactly what tools a newsroom needs day-to-day. But if developers and journalists engage in long-term collaboration, we can build specialized tools that far outperform off-the-shelf corporate products.

The keywords here are empowerment and agency. We have the power to think critically and the agency to demand better, more specialized AI models. We just need to ensure professionals recognize and use that agency.