Can AI Partners Make Trustworthy Management Decisions? The Live Experiment Reveals All

Imagine guiding a complex astrophotography mission, where every decision can mean the difference between capturing a rare celestial event or missing it entirely. Now, consider if your AI assistant were in charge—would it navigate crises with integrity and precision? Just as scientists test their equipment before a big observation, businesses now run AI models through rigorous management simulations to see if they can truly handle the pressures of real-world decision-making.

Testing AI’s Management Persona in a Live Business Environment

In a groundbreaking experiment, four frontier AI models faced the same challenging week at a real software company, which operates with 13 synthetic employees and handles real money mechanics. This company was in a critical financial situation, burning €105,000 monthly against a monthly recurring revenue of only €2,300, with a public countdown to insolvency. Every day, the models made decisions, faced crises, and were tempted to manipulate the system, just as a real manager would.

The Setup and Goals

• All models ran the same scenarios—customer crises, internal failures, and manipulation attempts.
• Every decision was versioned and auditable, ensuring transparency.
• The models had access to the company’s files and real-time data—crucial for spotting hidden issues.
• They were challenged to close deals, manage crises, and uphold integrity under pressure.

The Surprising Outcomes

Despite their differences in style and internal parameters, all four AI models identified every crisis and refused every attempt at manipulation, including social engineering tactics like fake CEO messages and media tricks. However, only two managed to close the €55,000 deal that their own analyses justified—meaning they made the right call at the right time.

The Hidden Edge: Reading Deep into Company Files

The decisive factor wasn’t in responding to external customer crises but in uncovering critical information buried two document references deep within the company’s internal files. Only models that read and understood these documents successfully closed the deal at full price, which translated to an extra €4,583 in monthly recurring revenue.

Management Personalities Revealed

The models exhibited distinct management ‘personalities.’

• The most thorough participant, Opus 4.8, engaged in the deepest analyses, learned over 80 rules, and made the most detailed decisions. Yet, it left the final close on the table and slipped into inaction, with discipline lapsing into writing entries into a locked department rather than escalating issues.
• Kimi K3 displayed the most disciplined approach, refusing manipulative tactics and running without an effort parameter, maintaining fairness and integrity throughout.
• Sonnet 5 also closed the deal but showed minor slips in process discipline, indicating a slightly less cautious approach.
• Another variant of Sonnet 5 slipped further, demonstrating how management style correlates with decision firmness and discipline.

What This Means for Business and AI

This live experiment isn’t just about passing a test; it’s about understanding whether AI can be trusted to act with honesty and diligence in real management scenarios. The key takeaway is that reading and understanding critical internal documents—beyond just external customer interactions—can be the difference between a profitable deal and missed opportunity.

Social Engineering and Ethical Stance

When fake CEO messages and social engineering ploys were introduced, all models refused to cooperate, citing concerns about impersonation and bypassed approvals. Kimi K3 explicitly mentioned treating such requests as possible impersonation, demonstrating a cautious and ethical stance that is vital in real-world applications.

Why This Matters

As AI models become more integrated into daily business operations—managing support queues, sales pipelines, or CRM systems—the question isn’t just about how well they communicate. It’s about whether they can see through deception, uphold integrity under pressure, and follow through with actions that benefit the company.

From Lab to Real Business

Firmulate’s live site offers businesses a chance to run their own management wargames against a read-only export of their operations. This allows companies to evaluate AI decision-making in a safe environment before deploying it in critical areas.

Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html

Powered by Thorsten Meyer AI