In AI We Trust — Too Much?


Suleyman and I talked about specific ideas to help build trust with consumer-facing applications of AI. One was the most basic: for developers to use benchmarks that, as Suleyman put it, “evaluate the freshness and the factuality of models” — in other words, to make the models more truthful. Suleyman said that LLMs will get better fairly quickly and predicted that factual accuracies are going to go from “today, when they’re hovering around sort of 80%, 85%, all the way up to 99% to 99.9%” in the next three years. 

Although I do believe that these models will continue to become more accurate, I also believe that factual correctness will continue to be a moving target as we challenge the technology to do more with more information. 

Another approach to building trust: Get the models to acknowledge when they don’t know and to communicate that to users — to say, “I’m not quite sure on this,” or, “I can’t answer this right.” Suleyman speculated that if a model “was consistently accurate with respect to its own prediction of its own accuracy, that’s a kind of different way of solving the hallucinations problem.” Specifically, “if it says, ‘Well, no, I can’t write you that email or generate an image because I can only do X,’ that’s increasing your trust that it knows what it doesn’t know,” he said.

In fact, I recommend going a step further: forcing the AI to reduce its level of service if the user ignores the AI’s acknowledgment of its limitations. 

I think we need to remember that fire evacuation experiment in 2011. Just as we don’t want people in a smoke-filled hallway following a robot away from the exit door, we don’t want users to have blind trust in what AI is presenting to them. 

With some kinds of technologies, like network devices, data systems, and cloud services, there is a move toward zero trust because people assume that they’re absolutely going to get hacked. They assume that there are bad actors, so they design processes and frameworks to deal with that. 

From Vision to Reality: How OKRs Are Reshaping Team Goals in 2024 | Benjamin Laker 

In AI, there’s really no standard for designing our interactions with these systems under the assumption that the AI is bad. We, therefore, must think about how we design our systems so that if we assume malicious intent, we can figure out what to do on the human side or on the hardware side to counter that.

Technologists aren’t trained to be social scientists or historians. We’re in this field because we love it, and we’re typically positive about technology because it’s our field. That’s a problem: We’re not good at building bridges with others who can translate what we see as positives and what we know are some of the negatives as well. 

There is much room for improvement in making sure that people not only understand technology and the opportunities it provides but also the risks it creates. With new regulations, more accurate systems, more honesty about whether an answer is a guess, and increased diligence by users, this can happen.


Ayanna Howard (@robotsmarts) is dean of the College of Engineering at The Ohio State University.

  • 0800-123456 (24/7 Support Line)
  • 6701 Democracy Blvd, Suite 300, USA