There’s a fundamental tension in AI product design. Users expect the system to always have an answer. But in high-stakes technical domains, an answer that’s wrong is worse than no answer at all.
At VectorAutomate, we made a deliberate design decision early on: our system must be able to refuse to answer when it doesn’t have sufficient evidence. This post explains why we made that choice and how we implemented it.
The Hallucination Problem
Large language models are prone to a well-documented behavior called hallucination — generating text that sounds authoritative but is factually incorrect. In consumer applications, this is an annoyance. In field service for complex equipment, it’s a liability.
Consider a technician troubleshooting an error code on a sterilization system. If the AI provides incorrect calibration values, the equipment could malfunction during use. If it provides an incorrect disassembly procedure, the technician could damage the unit or injure themselves.
The standard approach to mitigating hallucination is to improve the model — better training data, better fine-tuning, better prompting. These help, but they don’t eliminate the problem. No amount of model improvement can guarantee zero hallucinations.
Our Approach: Refuse Rather Than Guess
VectorAutomate takes a different approach. Instead of trying to make the model never hallucinate, we built a system that detects when the model is likely to hallucinate and prevents the response from reaching the user.
This works through a multi-stage confidence assessment:
Retrieval confidence — After searching the customer’s document corpus, we evaluate how well the retrieved passages match the query. If the best matching passages have low relevance scores, we flag the response as potentially unsupported.
Generation confidence — During response generation, we monitor the model’s own confidence signals. Claims that the model generates with low internal confidence are flagged for additional verification.
Citation verification — After generation, we verify that every technical claim in the response is actually supported by the cited passages. Claims that can’t be verified against the source material are removed.
If, after these stages, the remaining response doesn’t provide substantive, well-supported guidance, VectorAutomate refuses to answer. The technician sees a clear message explaining that the available documentation doesn’t contain sufficient information to answer the question, along with a suggested escalation path.
Refusal as a Signal
Here’s where it gets interesting. Most AI systems treat a non-answer as a failure to be minimized. We treat it as a signal to be captured.
Every refusal in VectorAutomate is logged and categorized. Over time, these refusals create a map of your documentation gaps — the topics where your technical documentation doesn’t provide adequate coverage for the questions your technicians are actually asking.
This turns the AI system into a continuous improvement engine. The more it’s used, the more precisely it identifies where your documentation needs work. And as documentation improves, refusal rates drop, and the system becomes more useful.
The Cultural Challenge
The hardest part of building a system that refuses to answer isn’t the technology — it’s the expectation management. Users are conditioned by consumer AI products to expect an answer to every question. When VectorAutomate says “I don’t have enough information to answer this safely,” some users initially perceive this as a product failure.
We’ve addressed this through two mechanisms. First, the refusal message is carefully worded to be helpful, not dismissive. It explains why the system can’t answer, what documentation would be needed, and what the technician should do instead. Second, we provide analytics that show the value of refusals — the documentation gaps they’ve surfaced, the incorrect guidance they’ve prevented, and the improvement trends over time.
Conclusion
In enterprise AI for technical service, the ability to say “I don’t know” isn’t a limitation. It’s a safety feature, a compliance feature, and a continuous improvement feature. We believe it’s one of the most important design decisions we’ve made, and it’s one we’ll continue to invest in.