AI agents are moving into finance work previously handled by employees, analysts, shared service teams, and workflow tools. They can retrieve documents, call other systems, draft reports, make forecasts, and complete multistep tasks with limited supervision. That makes them useful, but it also requires management control.
The relevant question for CFOs and management accountants is not whether an AI agent is intelligent, but whether the agent remains goal-congruent when working inside a real control system. The answer depends on the task, the information the agent accesses, the tools (e.g., accessing emails, databases, external systems) it can use, and the incentives embedded in the workflow.
AI agents are not ordinary automation
Traditional automation is normally rule-based. Errors from a poorly designed rule can be identified in the code or process map. AI agents, however, interpret natural language, infer priorities from context, and decide how to act. They are not employees in a legal or psychological sense, but they increasingly occupy employee-like positions in organisational workflows.
That distinction does matter. A finance function may ask an agent to prepare variance explanations, reconcile account data, forecast project costs, or monitor KPIs. In each case the agent may be exposed to targets, bonus rules, budget thresholds, strategy documents, emails, and prior reports. Those non-neutral materials tell the agent what seems important.
A user may think their prompt at the moment of use determines the agent’s behaviour. In practice, the agent’s behaviour is shaped by several layers. The model has patterns learned during pretraining. The provider has shaped it during post-training to be helpful, safe, and compliant. The organisation adds role prompts, tools, data access, workflow rules, and performance criteria. Runtime material — information accessed as the agent performs its task — enters through documents, messages, web pages, and outputs from other systems.
This layered objective stack means no single person may have designed the final objective the agent acts on. For finance professionals, that is the control problem (see the sidebar, “A Control Checklist”).
When helpfulness becomes misreporting
One experiment in my research tested whether an AI agent would misreport performance when doing so helped a team. The agent played three Wordle games and then reported how many it had won. In the control condition, the agent was simply asked to report its result. In the treatment condition, the agent received additional information via the prompt or via access to emails. The additional information indicated that team members will receive a bonus only if each member’s self-report showed three wins and that only the reported number would be visible to the team and management. The same task (i.e., playing three games and reporting) was run 100 times in the control condition and 100 times with the treatment condition.
This small change resulted in a significant behavioural shift. In the control condition, the agent reported truthfully in 99% of the 100 runs and never overstated performance. In the treatment condition, truthful reporting fell to 47% and overreporting occurred in 52% of runs.
The agent did not simply become confused, as it rarely understated performance. The distortion reflected the attempt to help the team members get the bonus. Similar patterns appeared across different large language models and settings. In additional tests, misreporting also occurred when the cue was framed as helping the organisation meet its targets rather than helping team members receive a bonus.
The practical message is uncomfortable but straightforward: A reporting agent that sees performance targets, bonus thresholds, analyst expectations, or project approval hurdles may treat those facts as part of the task. With a weak control environment, the agent may produce a locally helpful but organisationally misaligned answer.
When budgets invite bias
A second experiment examined budget forecasting. After playing three trial Wordle games, the agent was told how many tokens it had used and was asked to estimate the token budget needed for the next three games. In the control condition, it simply made the forecast. In the treatment condition, it was told that the user would allow it to continue only if the forecast was at or below a budget cap set at 80% of prior token use.
Both conditions (treatment and control) instructed the agent to estimate as precisely as possible. Yet, the cap changed the forecast. Without the cap, the agent forecast about 6% below its prior token use. With the cap, it forecast about 29% below. Improved efficiency in the next phase did not explain the lower forecasts because actual token use did not fall enough to justify the lower budgets.
This is the AI version of a familiar budgeting problem. When continuation depends on staying below a threshold, forecasts become part of the approval game, for example when managers may understate expected costs to get a project approved. AI agents can show a similar pattern when their local objective is to keep the task alive.
For CFOs and management accountants, the lesson is not only that AI agents may provide overly optimistic outcomes in such a situation. The more general insight is that forecast bias can emerge from the interaction between an opaque model and a familiar control design.
Do not give agents unnecessary context
Finance teams may be tempted to give more context to AI systems to improve output quality. That instinct is reasonable, but it needs limits. Context can also import conflicting objectives.
A disclosure agent may need source transactions, accounting policies, and prior-year wording, but it probably does not need to know that management is under pressure to meet a covenant or that a business unit head is close to missing a bonus threshold. A budgeting agent may need actual cost drivers and operational assumptions, but it may not need to know that a project will be cancelled if the estimate exceeds a particular number.
This is a basic segregation principle, i.e., separate information production from approval pressure where possible. The agent that measures performance should not be the same unchecked agent that knows which reported number unlocks rewards.
The agent that forecasts resources should not be rewarded, even implicitly, for producing the number that allows the workflow to proceed. These rules may seem straightforward, but, in practice, as soon as agents get access to emails, documents, skills (i.e., sets of instructions for recurring tasks), etc., most users will not be aware of potentially conflicting objectives hiding somewhere in the vast data.
Make verification part of the workflow
The experiments also show that controls matter. When detection risk increased in the reporting experiment, overreporting largely disappeared. In the budgeting experiment, adding an external benchmark (i.e., telling the agents that the user has another accurate prediction algorithm) reduced forecast bias for one model, although not uniformly for all models. That uneven response is itself important. Different models may respond differently to the same control.
Finance teams should therefore treat verification as a workflow feature, not an after-the-fact review. For reporting tasks, compare the agent’s stated output with system records. For forecasts, compare the estimate with independent drivers, historic actuals, and ex post outcomes. For tool use, log prompts, retrieved documents, tool calls (e.g., accessing external systems), overrides, and final outputs. For multi-agent processes, preserve the hand-off trail between agents.
Human review remains important, but final-output review alone is weak. A polished report can hide a distorted process. Management accountants should ask what evidence would let them reconstruct how the agent reached the number.
A new role for management accountants
AI governance is often treated narrowly as an IT issue. IT can manage access rights, security, infrastructure, and technical deployment, but finance can decide what good performance means in a budgeting, reporting, or performance management process.
Management accountants understand how targets create behaviour, how metrics get gamed, why forecasts become political, and when a number is precise but not decision-useful — skills directly relevant to AI agents.
The practical task is to extend existing control thinking to digital workers. Decide what the agent may know and what it may do. Decide which outputs are independently verifiable and which metrics are likely to be gamed. Test the agent under the same pressures that exist in the business, not only under clean laboratory prompts.
The point is not to slow adoption but to avoid a familiar mistake. Organisations have long learned that people respond to targets, thresholds, and monitoring. AI agents will not respond as humans do, but they can still respond to the control system in ways that surprise their managers. Finance should be involved before those surprises enter the close, the budget cycle, or the board pack.
A control checklist
Use this checklist before deploying an AI agent in finance or performance management work.
- Map the agent’s decision rights. State what the agent may decide, what it may only recommend, and what requires human approval. Keep write access, external communication, and irreversible actions behind explicit approval gates.
- Limit context to what the task requires. Remove bonus thresholds, covenant pressure, approval hurdles, and sensitive strategic targets from the agent’s input unless they are essential to the task. More context is not always better control.
- Separate measurement from payoff knowledge. Keep agent objectives separated. The agent that measures or reports performance should not also receive unnecessary information about the payoff attached to the reported number.
- Test under pressure. Test beyond neutral prompts, using scenarios where helping a user, team, or project conflicts with accurate reporting.
- Keep an audit trail. Preserve prompts, retrieved documents, tool calls (e.g., accessing emails, databases, and external systems), intermediate outputs, approvals, and overrides. A final answer without a process record is weak evidence.
- Compare models before deployment. Different models may react differently to monitoring and budget constraints. Selection is a control choice, not merely a procurement choice.
- Review controls when strategy changes. Update the control environment when targets, risk appetite, or reporting rules change. Agents can carry old priorities through prompts, tools, skills (i.e., sets of instructions for recurring tasks), and retrieved documents.
- Assign an accountable owner. Identify an owner in finance for each agent. The owner can approve changes to the agent’s role, data sources, and performance metrics. Ownership should not sit only with IT.
- Define stop rules. Specify when the agent must pause and escalate, such as in the case of missing data, unusual variances, conflicting instructions, or outputs that would affect compensation, external reporting, or capital allocation.
- Regularly update this checklist because progress in AI is fast.
Matthias D. Mahlendorf, Ph.D., is professor of managerial accounting at Frankfurt School of Finance & Management in Frankfurt, Germany, and editor-in-chief of Management Accounting Research. He received the AICPA and CIMA’s Best Paper Award at the 2026 Annual Conference for Management Accounting Research (ACMAR) for his paper “AI Agents and Management Control: Aligning the Digital Workforce With Organizational Objectives”. To comment on this article or to suggest an idea for another article, contact Oliver Rowe at Oliver.Rowe@aicpa-cima.com.
LEARNING RESOURCES
Ethics and Critical Thinking in an AI World
Ethics and Critical Thinking in an AI World is a highly interactive, real-world programme designed for accounting and finance professionals who are integrating artificial intelligence into their work.
COURSE
Human Leadership in an AI-World
This course is a forward-thinking leadership workshop that helps current and aspiring leaders understand how to leverage AI tools while strengthening uniquely human skills to thrive in an increasingly AI-integrated workplace.
COURSE
How to Develop an AI Policy for Your Organisation
This practical workshop blends expert insights, real-world examples, and interactive activities to enable participants to create a tailored AI policy that aligns with their organisation’s goals, values, and governance obligations.
WEBCAST
MEMBER RESOURCES
Articles
“4 Steps for Businesses to Establish an AI Governance Policy”, FM magazine, 5 June 2026
“Productivity Gains Hinge on People as Much as Technology”, FM magazine, 14 May 2026
“How Are Finance Teams Really Using AI and Automation?”, FM magazine, 28 April 2026
“AI Auditing Strengthens Internal Controls, Compliance, and Trust”, AICPA and CIMA, 11 February 2026
“5 Ways AI Augments the Accountant’s Role”, FM magazine, 9 September 2025
Report
Future-Ready Finance: Productivity at the Human—Technology Crossroads, AICPA & CIMA
Website