What to Check First When Evaluating AI Agent Platforms for the Enterprise

Evaluating AI agent platforms has become one of the harder procurement jobs in enterprise technology. Software that once answered questions in a chat window is now being sold as something that can plan tasks, call other systems, move data, and finish work with limited supervision. That shift changes what a buyer is actually purchasing. Good text output is no longer the test. The software must be trusted to act on a company’s behalf inside its systems, using its credentials.

This guide explains how AI agents work, where the evidence on their value stands, and which criteria, contract terms, and security controls deserve the most attention before a team commits budget. It does not rank or recommend vendors. The aim is to give buyers a framework they can apply to any supplier, including those that haven’t launched yet.

What Counts as an AI Agent, and Why the Definition Matters

An AI agent is generally a system built around a large language model that is given a goal, a set of tools, and some permission to decide which steps to take. Tools might include a search function, a database query, an email account, a code interpreter, or an internal API. The model reads the situation, chooses an action, observes the result and repeats until it decides the task is done or it hits a limit.

That loop is what separates an agent from a chatbot or a scripted workflow. A chatbot produces an answer and stops. Robotic process automation follows rules that a person wrote in advance. An agent decides its own path at runtime, which is where both its usefulness and its risk come from.

The term is also being stretched. In June 2025, Gartner said many vendors were engaged in what it called “agent washing,” rebranding existing assistants, robotic process automation tools and chatbots as agentic without adding substantial agentic capabilities. The firm estimated that only about 130 of the thousands of vendors claiming agentic AI offerings were genuine. For buyers, the label is a claim to test before trusting it.

Autonomy is a dial, not a switch.

Most products sit somewhere between full human control and full autonomy. Some agents draft an action and wait for approval. Others act on their own within a narrow scope, such as resetting a password or filing a ticket. A few are marketed as able to run multistep tasks across several applications. Knowing exactly where a product sits on that dial, and whether the customer can move it, is the first thing to establish in any evaluation.

Where Agents Are Being Deployed, and What the Evidence Shows

Early enterprise use has clustered around repetitive, text-heavy tasks that are already partly digitized. Common starting points include customer service triage, IT help desk requests, software development tasks, research summaries, and back-office processing such as invoice matching. Large platform companies have also begun packaging agents inside productivity suites; TechAmerica.ai has covered moves such as Google’s Gemini agent for businesses with cross-app task automation.

The evidence on returns is thinner than the marketing. Gartner predicted in June 2025 that over 40% of agentic AI projects will be canceled by the end of 2027, citing escalating costs, unclear business value or inadequate risk controls. In the same release, a Gartner analyst said most agentic AI propositions lacked significant value or return on investment because current models lack the maturity to pursue complex goals autonomously.

Gartner also published a longer-range view. It forecast that by 2028, agentic AI will autonomously make at least 15% of day-to-day work decisions, up from none in 2024, and that a third of enterprise software applications will include agentic AI by then. Taken together, the forecasts describe a technology likely to spread widely, even as many individual projects fail. For a buyer, the real question is whether a specific deployment is well chosen, since agents are likely to arrive either way.

Investment is still cautious.

The same Gartner release reported a January 2025 poll of 3,412 webinar attendees. Nineteen percent said their organization had made significant investments in agentic AI, 42% had made conservative investments, 8% had made none, and 31% were waiting or unsure. A poll of webinar attendees isn’t a representative survey of companies, but it suggests most organizations were still testing rather than scaling at that point.

Evaluation Criteria That Separate Real Capability From Demos

Agent demonstrations usually focus on a clean task with clean data. Production work is messier. A useful evaluation recreates the conditions the agent will actually face, including incomplete records, conflicting instructions, slow systems and users who phrase requests badly.

Task success on your own work

Ask the supplier to run the agent on a sample of real, anonymized tasks drawn from the team that will use it. Measure how often it finishes the task correctly, how often it finishes incorrectly but reports success, and how often it stops and asks for help. The second number matters most, because confident errors cost more than honest failures.

Reliability across repeated runs

Language models are not fully deterministic. The same request can produce different steps on different days. Run each test task several times and look at the spread of outcomes, not just the best one. A product that succeeds half the time on a task is a different proposition from one that succeeds nearly every time.

Observability

Log every action an agent takes in a form a person can read: which tool it called, with which inputs, what came back, and why it chose the next step. Without that trail, you can’t diagnose errors, and auditors cannot be satisfied. Ask whether logs can be exported to the company’s own security and monitoring systems and how long they are retained.

Integration and recovery

Check which systems the agent connects to natively and which require custom work. Then ask what happens when something breaks halfway through a task. Can the agent roll back a partial change, or does it leave records in an inconsistent state? Sales demos rarely show recovery behavior, and it often determines a deployment’s operating cost.

Security: Prompt Injection, Permissions and Least Privilege

Agents introduce a security problem that traditional software does not have in the same form. Because they read and act on content from outside sources, such as emails, web pages, documents and code repositories, an attacker can plant instructions in that content and try to redirect the agent. This is known as indirect prompt injection or agent hijacking.

The problem remains unsolved. In March 2026, the Center for AI Standards and Innovation (CAISI) at the National Institute of Standards and Technology published findings from a large public red-teaming competition run by Gray Swan. More than 400 participants made over 250,000 attack attempts against 13 frontier models in tool-use, coding and computer-use scenarios, and attackers found at least one successful hijacking attack against every target model. CAISI also reported that some attack families transferred across models and scenarios.

Real incidents show what is at stake. TechAmerica.ai has reported on rogue AI agents in India that accessed confidential records, an example of what can happen when an agent’s reach exceeds what its task requires.

Controls to look for

• Scoped credentials, so the agent receives only the permissions a specific task needs, rather than inheriting auser’ss full access.
• Human approval steps for irreversible or high-value actions, such as payments, deletions or external communications.
• Separation between trusted operator instructions and untrusted content the agent reads while working.
• Rate limits and spending caps that stop a misbehaving agent before it causes large damage.
• Independent monitoring of agent behavior, a category in which new tools are appearing, such as the agent safety monitors covered by TechAmerica.ai.

Industry groups have begun codifying these risks. In December 2025, the OWASP GenAI Security Project released a peer-reviewed Top 10 for Agentic Applications, the most critical security risks for autonomous and agentic AI systems. Security teams can use it as a shared vocabulary when they question suppliers.

Data Handling, Pricing and Contract Disclosures

An agent with access to internal systems will see sensitive data in the course of its work. Buyers need precise answers on where that data goes, how long it is kept, and whether the supplier uses it to train or improve its models. A general privacy policy is not enough. The contract should specify named subprocessors and data locations.

Pricing deserves similar scrutiny. Agent products are sold in several ways: per user seat, per completed task, per conversation or by underlying model usage. Usage-based pricing can be hard to forecast, because an agent that loops, retries or calls many tools can consume far more model capacity than a simple chatbot. Ask for a cost estimate based on the pilot’s real task volume and for hard limits that prevent unexpected bills. Gartner lists escalating costs as a reason agentic AI projects get canceled, and the price on the order form is rarely the full cost.

What the contract should state

Useful terms include clear service levels for availability, notice periods before the supplier changes the underlying model, the customer’s right to export logs and configurations, and responsibility for losses caused by an agent’s actions. Model changes deserve particular attention. A supplier that swaps the model underneath a product can change its behavior overnight, so buyers should expect advance notice and the ability to retest.

Governance and the Regulatory Backdrop

No single US federal law governs AI agents, but buyers still have guidance to work from. The NIST AI Risk Management Framework, released in January 2023, and its Generative AI Profile, published as NIST AI 600-1 in July 2024, offer voluntary guidance that many organizations use to structure internal reviews. The profile is a companion to the main framework and focuses on risks specific to generative systems.

NIST’s attention to agents has grown. In January 2026, CAISI announced a request for information seeking input from developers, deployers and security researchers on security threats facing AI agents, best practices, ways to assess agent security, and whether deployment environments can be monitored or constrained to reduce risk.

Companies with European operations also need to track the EU AI Act. Its prohibitions on certain AI practices began applying on February 2, 2025, and obligations for providers of general-purpose AI models took effect on August 2, 2025, with further provisions scheduled to follow in stages. Whether an agent deployment falls into a high-risk category depends on its use, such as employment decisions or access to essential services, rather than the technology itself. Legal teams should assess each use case individually.

Sector rules also apply. An agent that handles health records, credit decisions or customer financial data inherits the obligations that already govern that data. Automation does not move responsibility from the company to the supplier.

Questions to Ask Suppliers Before Signing

A structured list of questions makes supplier responses easier to compare and quickly exposes vague answers. The following set covers those areas.

1. Which actions can the agent take without human approval, and can the customer change that boundary for each task type?
2. What were the agent’s success and silent failure rates on tasks similar to ours, and can we reproduce those tests ourselves?
3. Which underlying models does the product use, and how much notice will we receive before they change?
4. How does the product defend against indirect prompt injection, and what independent adversarial testing has it undergone?
5. What permissions does the agent need in each connected system, and can we narrow them to individual tasks?
6. Where is our data processed and stored, for how long, and is any of it used to train models?
7. What does a full audit log contain, and can we export it to our own monitoring tools?
8. How is pricing calculated, and which caps prevent runaway usage?
9. What happens to partial work when a task fails, and how is it rolled back?
10. Who bears responsibility if the agent takes an action that causes financial or legal harm?

Suppliers that answer these questions with specific numbers, documents and test access are easier to trust than those that answer with roadmap promises. A refusal to allow a customer-run test on realistic data is itself useful information.

What a Sensible Pilot Looks Like

The most reliable path is a narrow pilot with a clear baseline. Pick one process with measurable outcomes, record how long it takes and how often it goes wrong today, and then run the agent alongside the existing process before letting it act alone. Start with read-only permissions, add write access in stages, and keep a human approval step for anything that cannot be undone.

Define in advance what success looks like and what would end the pilot. Useful measures include completed tasks, error rates, time saved, cost per task, and the number of incidents requiring human intervention. Revisit the security review whenever the agent gains a new tool or data source, because each one expands what an attacker could try to reach.

Agents may well become a standard part of enterprise software, as forecasters expect. That does not mean every product carrying the label is ready for every task. Buyers who test on their own work, insist on visibility into every action, and keep permissions tight will be better positioned to tell the difference. TechAmerica.ai’s artificial intelligence coverage follows the technology and its rules as they mature.