Most of the technology we use daily fails at some point. Our computer freezes. Apps go down for a few hours. Voice assistants like Siri and Alexa don’t always understand us the first time. Our cars display cryptic symbols that send us to the mechanic for translation. While no one enjoys these blips and errors, we’ve come to a tacit agreement with the technology that drives our days. We don’t expect perfection, but we expect it to work most of the time — and to derive value when it works.
Yet, we do expect perfection from AI. When a Gen AI chat interface delivers erroneous information or a generated image includes an extra finger or two, many see this as a fundamental flaw of AI and are ready to dispel the entire technology. One bricked phone doesn’t cause distrust of all smart phones, but one AI “hallucination” can spark headlines about the end of trust in technology.
Why? Because AI feels different. Its intelligence, pervasiveness, uncanny ability to feel like a peer, and complexity shapes our perfectionism bias towards AI.
The Perfection Expectation
For years, businesses have been using AI and machine learning in innovative and efficient ways, such as product recommendations, credit scoring, fraud detection, and customer care. Those systems quietly shaped modern life, yet most people never saw them in action so they rarely judged their mistakes.
Gen AI changed the game with its ubiquitous availability to anyone with a smartphone or a computer. ChatGPT alone reports that it has 800 million weekly users. Browsers feature Gen AI search summaries. Business productivity tools offer Gen AI capabilities for copy and image creation. Whether or not you wish to engage with AI, it’s here.
Generating content at such a massive scale is bound to have some hallucinations and errors. Fundamentally, LLMs predict what’s probable, rather than actually produce factually accurate information. They match patterns and predict the next statistically plausible word or pixel. Due to human-based reinforcement learning, AI capabilities continue to advance as engagement skyrockets, but errors still occur.
Mistakes can feel personal, too. Gen AI is made to converse like a human; it’s an always-on work assistant and confidante. Subconsciously, we hold it to human-level accountability. When Gen AI hallucinates a citation or generates an image with too many fingers, it can feel like a betrayal of the overall promise of AI.
But for AI engineers and scientists like myself, errors are an expected step in the development of newer technologies like Gen AI and agentic AI. What’s critical is how AI is ethically built and used and how errors are mitigated.
Why Black-Box Technology Makes Businesses Nervous
Despite these concerns, businesses are pursuing the promise of AI-driven applications: greater personalization, efficiency, easeful service, and innovation. Any worries about AI imperfection center around risk, ROI, and expensive real-world consequences: initiatives that fail due to lack of reliability and therefore adoption, lawsuits caused by hallucinations or bias, and reputational damage.
This concern is fueled by the black-box nature of generative and agentic AI. The outcomes of rule-based AI, such as when matching an intent to a response in an AI-based customer care platform, are pre-programmed, transparent, and auditable. Traditional predictive AI, such as for sales forecasting, relies on structured data and statistical models. AI decisions can be traced with relative confidence.
With trillions of data points as inputs, deeply layered non-linear classifiers, and a large number of tunable parameters, Gen AI functions as a black box lacking complete explainability. It’s virtually impossible to trace an output to an exact training source, especially as many platforms build upon proprietary and open-weight models like Gemini, GPT, Claude, Mistral, and Llama, which may disclose benchmarking and testing procedures, but not data sources.
Agentic AI will further compound this complexity and frustrate the desire for glass-box explainability. Agentic architecture essentially operates like a distributed computer network, such as with microservices. Multiple AI agents autonomously coordinate and communicate to perform multi-step tasks. Gen AI is used to interpret the main goal, reason through trade-offs, and dictate precisely how actions will be executed.
Due to this black-box nature of Gen AI and agentic AI, many organizations are increasingly scrupulous of their AI vendors, demanding a higher level of infallibility and accountability than they might from other technologies.
Bringing Transparency to the Black Box of AI
As AI matures, so will our expectations. Experience will teach both businesses and consumers that imperfection doesn’t signal failure — it means opportunity for refinement. What matters most is knowing that people and processes are in place to identify, measure, and mitigate those imperfections responsibly.
For now, businesses building and using AI must contend with these high expectations and even welcome the scrutiny. In this wild west moment, industry-wide guardrails are still forming. Regulatory bodies are scrambling to keep up with advances and organizations must do their best to form their own ethical frameworks, internal governance, and oversight processes.
At Interactions, we believe that transparency is key to building more trust in AI. Our openness regarding our ethical AI practices — such as documenting data provenance, testing and benchmarks, human-directed training, accountability and integrity regarding how issues are identified and mitigated, and standards for vetting third-party models and solutions — creates visibility into what feels unknowable.
To learn more about our work in shaping the future of responsible AI, visit our AI Trust Council site.
