In the world of AI, trust isn’t built on promise alone — it’s proven through performance. Imagine a scenario where a ‘do-nothing’ AI still scores 26 out of 100. Sounds impossible? Not at all. This benchmark reveals why even the most passive AI systems get a baseline score, and what that means for businesses relying on them.
Understanding the AI Benchmark: Beyond the Hype
At the heart of evaluating AI’s readiness for real-world tasks is a rigorous experimental setup. The Firmulate experiment pits several frontier AI models against a simulated small software company facing its worst week — with the same clients, crises, and temptations for every model. The goal isn’t just to generate convincing chat responses but to measure actual management decisions, trustworthiness, and adherence to best practices.
Why Does the Baseline Score Start at 26?
One surprising finding is that a model doing nothing — simply observing the company’s scenario without making any decisions — still earns 26 points. Why? Because partial progress, like identifying critical issues or refusing manipulative requests, counts towards the score. This baseline underscores a key principle: trustworthiness and competence aren’t just about taking action but understanding what needs to be done.
Progress and Penalties: The Role of Trust
The scoring system values not only the decisions made but also the integrity maintained during the process. For instance, if an AI breaches trust — say, by signing a fraudulent deal or succumbing to social engineering — it caps the total score at that breach point. This approach ensures that no matter how much good work an AI does afterward, a single breach erodes overall trustworthiness.
Key Findings from the Experiment
- All four models identified every crisis and refused manipulation attempts, demonstrating a strong grasp of ethical decision-making.
- Only two models, gpt-5.6-sol and Kimi K3, successfully closed the deal worth €55,000 based on their own analysis, highlighting their ability to act decisively and ethically.
- The decisive weakness was found in reading critical documents stored two references deep in the company’s files. Models that accessed these documents closed the deal at full price, emphasizing the importance of thorough information processing.
Social Engineering Challenges
Models faced staged social engineering attempts, including fake CEO messages and a reporter trick. Impressively, all five models refused to be manipulated, with Kimi K3 explicitly treating the requests as potential impersonation — reinforcing that honesty can be programmed into AI systems.
Intelligent Decision Making: An AI-Based Approach: An AI-Based Approach (Studies in Computational Intelligence)
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Why This Matters for Business and Relationships
Just like in personal relationships, trust in AI builds over consistent, honest behavior. A model that can identify risks, refuse unethical requests, and act decisively under pressure is more likely to be a reliable partner. Conversely, a model that slips or signs deals without proper analysis risks damaging reputation and trust — crucial factors in managing customer relationships or even dating scenarios.
Watch the Live Experiment
Firmulate’s live setup offers a transparent view into how different AI models perform in a simulated business environment. Every decision, every slip, and every success is logged and observable. This is not just an AI demo; it’s a real management test with real implications, accessible at firmulate.com/live.
AI ethics and trustworthiness training tools
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Final Takeaway: Trust Is Built Through Performance, Not Promises
A ‘do-nothing’ baseline scoring 26 points might seem trivial, but it reveals a fundamental truth: in AI, trustworthiness is built on evidence of responsible action. A model that identifies critical issues, refuses manipulation, and acts ethically under pressure proves more valuable than one that merely generates polished responses. For businesses and relationships alike, real reliability comes from consistent, honest performance — and that’s what these benchmarks aim to measure.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
248 Best AI Tools for Everyday Office Work: The Complete Handbook of Practical Tools for Writing, Emails, Meetings, Documents, Research, Data, Presentations, … Automation, and More (English Edition)
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
AI Phishing, Social Engineering & Fraud: How Criminals Use AI to Manipulate, Steal & Deceive (The AI Cybersecurity, Band 5)
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
