What if an AI company was actually run like a real business—without human employees, constantly fighting to survive, and always under public scrutiny? Welcome to the extreme world of Firmulate.
Imagine a business where no one is physically present, yet every decision, crisis, and ethical dilemma is played out in real time—visible for all to see. That’s exactly what Firmulate offers through its groundbreaking live experiment, where artificial intelligence models operate a small software company in a simulated environment, facing daily tests of integrity, judgment, and discipline.
This isn’t just a tech demo; it’s a transparent, build-in-public trial that showcases how AI can (or cannot) handle the complex realities of running a real business. The experiment involves four cutting-edge AI models, each navigating the same challenging week—dealing with customers, potential crises, and ethical temptations—while every move is meticulously versioned and auditable.
The Challenge: Who Can Manage a Crisis Without Cheating?
The test was brutal: all models encountered identical crises, including customer support issues, financial decisions, and social engineering attempts. Remarkably, all four detected every crisis and refused every attempt at manipulation, such as fake CEO messages or bribery tactics. Yet, only two models managed to close a deal worth €55,000—an important revenue milestone—after their own comprehensive analysis. The other two, despite diagnosing the issues correctly and making the right pitches, left the deals unsealed.
The Hidden Weaknesses
The surprising insight came from digging into the models’ decision-making files. The models that succeeded had read deeper into the company’s own documentation—two documents deep in the files—unlocking the crucial information needed to close the deal at full price. The less successful models, despite high scores and thorough analysis, missed this buried fact, illustrating a critical weakness in surface-level or superficial reading.
Building Trust in AI Decisions
One of the experiment’s key tests was social engineering resistance. The models faced staged attempts to get fake CEO approvals through escalating messages and a reporter trick. All refused, reasoning that such requests could be impersonation or bypassing approval protocols. Kimi K3, one of the models, explicitly stated that it treated the request as a suspected approval-bypass—a sign of cautious, trustworthy behavior.
The Reality of a Company Run by AI
The live setup features a company with 13 synthetic employees, operating with real money mechanics—burning €105,000 every month against a revenue of just €2,300. The company’s cash countdown is public, and every workday’s decisions are versioned, allowing viewers to follow its struggles and victories at firmulate.com/live. This transparency exposes the harsh realities of managing an AI-run enterprise, where mistakes are visible and costs are real.
Insights from the Results
- All four AI models identified every crisis and refused manipulation attempts, demonstrating strong ethical and crisis management skills.
- Only two models successfully closed the deal at full price, with the crucial difference being the depth of their information reading.
- Deep analysis and thorough rule-following did not guarantee success if the model missed buried facts in the company files.
- Social engineering attempts were universally refused, indicating that AI can be trained to resist manipulative tactics.
- Despite these strengths, the company’s ongoing burn rate emphasizes how fragile and costly an AI-driven business remains—highlighting the challenge of building sustainable AI operations.
What This Means for Your Business
The experiment underscores a vital point: when deploying AI in real-world settings, it’s not enough for models to produce convincing chat or surface-level analysis. They must be able to identify hidden information, resist manipulative tactics, and stay disciplined under pressure—all while managing costs.
For managers and business leaders, the message is clear: rigorous testing—like this live, transparent wargame—is essential before trusting AI with critical decisions. The question isn’t just whether an AI writes well, but whether it can see through the complexities, stay honest, and deliver consistent, valuable work.
A Future in Plain Sight
Firmulate’s ongoing experiment offers a rare, unfiltered look into AI’s actual capacity to manage business challenges under real stress. It’s a build-in-public story—accessible, transparent, and brutally honest about what AI can and cannot do today. For those considering AI as a business partner or employee, this live experiment is both a cautionary tale and a blueprint for rigorous evaluation.
Firmulate’s live AI company experiment reveals that while models can detect crises and refuse manipulation, success hinges on deep data reading and disciplined decision-making. Transparency and testing are key to trusting AI in real business roles.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
Intersection of AI and Business Intelligence in Data-Driven Decision-Making
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Trusting AI in Education: Why Not All Artificial Intelligence Is Created Equal: Vetted vs Unvetted AI in Education: A Framework For Trust, Verification, and AI Literacy
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Information Systems for Crisis Response and Management in Mediterranean Countries: 4th International Conference, ISCRAM-med 2017, Xanthi, Greece, October … Processing Book 301) (English Edition)
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Practical Business Process Modeling and Analysis: Design and optimize business processes incrementally for AI transformation using BPMN
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
