Visit Tool

Autonomous AI agents are AI systems that can make decisions, use software tools, and complete tasks with limited human help. Businesses use them to automate workflows, analyze information, and improve productivity. However, autonomous AI also creates risks involving data security, compliance, and accountability.

Marinela Profi, Global Market Strategy Lead for AI Agents and Generative AI at SAS, explains why businesses need strong governance before allowing AI agents to operate independently. Her insights focus on controlling agent permissions, improving data quality, testing systems, and keeping humans involved in important decisions.

Ahead of AI & Big Data Expo Europe in Amsterdam, Profi discussed how enterprises can build AI systems that operate within clear boundaries. Her main message is simple: businesses must consider not only how quickly they deploy AI but also how safely they control its actions.

What Is Autonomous AI Agent Governance?

Autonomous AI agent governance is the process of setting rules, permissions, monitoring systems, and human approval requirements for AI agents that perform tasks independently.

Key principles include:

• Restrict access to necessary data and tools.
• Test complete workflows before and after deployment.
• Turn governance policies into technical controls.
• Use accurate and reliable data.
• Require approval for high-risk actions.
• Monitor agent activities and maintain recovery options.
• Keep businesses accountable for AI-driven decisions.

What Are Autonomous AI Agents?

Autonomous AI agents are software systems that can work toward a goal by processing information, planning tasks, using tools, and taking actions with limited human intervention.

Unlike a basic chatbot that mainly answers questions, an AI agent can interact with databases, update records, and trigger business workflows.

For example, a customer service agent might review a refund request and check an order database. A company could allow it to process small refunds automatically while requiring manager approval for larger payments.

This flexibility makes AI agents useful, but incorrect actions can create serious problems. Effective governance must therefore control both what an agent says and what it does.

1. Why Traditional AI Deployment Methods Can Fail

Traditional software development often involves releasing a feature, observing user behavior, and making improvements. This approach can work when mistakes have limited consequences.

Autonomous AI agents introduce additional risks because they can act before a human reviews their decisions. An agent might update customer records, trigger a payment workflow, or send information to another system.

The SAS Data & AI Impact research cited in the interview reported that trust in generative AI was 76%, compared with 66% for agentic AI. The interview also reported that 89% of agents deployed in production were already taking actions rather than only assisting users, with more than half operating with limited or no human approval.

These findings highlight why businesses need stronger controls as AI systems become more independent.

How Can Businesses Deploy AI Agents Safely?

Before deployment, organizations should ask three questions:

• Authority: What actions can the agent perform?
• Risk: What could happen if it makes a mistake?
• Recovery: Can the organization stop or reverse an incorrect action?

Clear permissions, monitoring systems, and recovery procedures help businesses manage these risks without preventing useful innovation.

2. How Poor Architecture Creates AI Governance Risks

AI governance problems often begin outside the model itself. They may result from weak data management, excessive permissions, missing activity logs, and policies that exist only in documents.

For example, a company investigating an incorrect AI decision may need to identify the data used, the rules applied, and the actions performed. Without reliable records, explaining the decision becomes difficult.

Profi describes this problem as trust debt. Poor architectural choices may seem inexpensive during a pilot but become costly when businesses expand their AI operations.

The SAS research cited in the interview found that only 17.5% of organizations reported fully optimized data infrastructure with capabilities such as lineage, governance, validation, and explainability.

What Should AI Governance Architecture Include?

A reliable architecture should include:

• Data lineage to track information sources and usage.
• Access permissions based on roles and tasks.
• Activity logs recording important actions and tool calls.
• Technical controls that enforce company policies.
• Validation processes to identify errors.
• Clear responsibility for system oversight.

Building these controls into the architecture from the beginning makes AI systems easier to monitor and manage.

3. Why AI Proofs of Concept Are Not Enough

A proof of concept (PoC) tests whether an AI agent can perform a particular task. However, success in a controlled demonstration does not prove that the system is ready for real business operations.

Production environments involve changing permissions, outdated information, unavailable tools, and unexpected requests. An agent may also pass incorrect information to another application, affecting connected workflows.

How Should Businesses Test AI Agents?

Companies should test the complete system rather than checking only whether the AI model produces a correct answer.

Important tests include:

1. Tool failures: What happens when an application becomes unavailable?
2. Data quality: Can the agent identify missing or outdated information?
3. Permission changes: Does it stop when access is removed?
4. Unexpected requests: Can it reject actions that violate company policies?
5. Recovery: Can the business correct the effects of a failed action?
6. Audit records: Can the team reconstruct what happened?

Testing should continue after deployment because models, data, and software environments change over time.

A successful demonstration proves capability, but production testing helps establish operational readiness.

4. Why One AI Layer Should Not Control Every Business System

Connecting an AI model to all company data and tools may appear efficient. However, unrestricted access can create security and compliance risks.

An AI agent might identify a useful action without having permission to perform it. For example, it could recommend changing a customer account but should not automatically modify sensitive information without appropriate checks.

Profi emphasizes the importance of combining AI technology with business rules and organizational collaboration.

How Can Enterprises Set Better Boundaries?

A safer architecture separates AI reasoning from the systems that enforce permissions and policies.

• AI reasoning: Interprets information and recommends actions.
• Business rules: Check whether proposed actions are permitted.
• Access controls: Restrict the data and tools an agent can use.
• Approval workflows: Send high-risk actions to authorized people.
• Monitoring: Record activities and identify unexpected behavior.

This structure allows AI agents to support business operations without giving a model unrestricted control over the enterprise.

5. How Verification Changes for Autonomous AI Agents

Traditional AI verification often measures accuracy, bias, and model performance. Autonomous AI requires additional checks because its outputs can trigger real actions.

An agent might interpret information correctly but still use the wrong tool, exceed its permissions, or perform an inappropriate action.

Businesses must therefore verify the complete decision-and-action process.

What Should AI Verification Check?

A strong verification process should establish:

• Whether the agent was authorized to act.
• Whether it used reliable data and appropriate tools.
• Whether it followed business rules.
• Whether human approval was necessary.
• Whether the action produced the expected result.
• Whether its activities can be audited.

The SAS research cited in the interview reported that 66% of organizations classified as having trustworthy AI used formal verification and validation processes, compared with 15% of organizations at earlier stages of maturity.

These findings highlight the importance of structured testing. Verification should continue after deployment, especially when models, permissions, or connected systems change.

6. Why Data Quality Matters for AI Automation

Autonomous AI agents can make useful decisions only when they have access to relevant and reliable information.

Businesses often store information across separate systems. If customer records, billing details, and account statuses conflict, an AI agent may struggle to identify the correct information.

Adding automation will not automatically solve this problem. Organizations must improve data quality, integration, and ownership before expanding AI operations.

How Can Businesses Prepare Data for AI Agents?

Companies should focus on:

• Data quality: Correct missing, inaccurate, and outdated records.
• Data integration: Connect relevant systems securely.
• Clear ownership: Assign responsibility for maintaining information.
• Data lineage: Track where information originates and how it changes.
• Access security: Prevent unauthorized information retrieval.

The interview also cited a life sciences example in which scientists at a major biopharmaceutical company reportedly spent as much as 80% of their time finding, cleaning, and reconciling data before analysis.

This example shows why reliable data foundations are essential for effective AI automation.

7. How Governance Can Make AI Deployment Faster

Some businesses view governance as a process that slows development. However, reusable policies and automated controls can help teams deploy AI agents more efficiently.

Without established rules, every project may require separate discussions about security, legal requirements, permissions, and risk.

A better approach is to define policies and enforce them through technical systems. For example, a company can specify which information an agent may access, which actions require approval, and what activities must be recorded.

What Does Effective AI Governance Look Like?

The SAS research cited in the interview reported that organizations with high AI trustworthiness scores were 15 times more likely to report strong or high AI return on investment than organizations with low scores.

This is an association reported by the study, not proof that governance alone causes higher returns. However, it highlights the importance of trustworthy AI operations.

How Much Autonomy Should AI Agents Have?

The right level of autonomy depends on risk, uncertainty, and how easily an action can be reversed.

A low-risk task, such as sorting documents, may be suitable for independent execution. A decision involving medical care, financial transfers, or sensitive information may require human approval.

Businesses can use three levels of autonomy:

1. Independent execution: Agents complete routine, low-risk tasks within defined limits.
2. Autonomy with escalation: Agents handle normal cases but escalate unusual or risky situations.
3. Human approval: An authorized person approves high-impact actions before execution.

For example, an AI agent might categorize customer support tickets automatically but require manager approval before issuing a large refund.

Human oversight should remain meaningful. Employees need enough information and time to review decisions rather than approving every action automatically.

The goal is to let AI handle appropriate tasks while people retain control over important decisions.

Conclusion

Autonomous AI agents can help businesses automate workflows, improve productivity, and support complex decisions. However, greater autonomy increases the need for reliable data, security controls, verification, and human oversight.

Marinela Profi’s insights highlight an important principle: governance should be part of AI architecture from the beginning. Clear permissions, continuous testing, and enforceable policies help businesses expand automation while managing risk.

The goal is not to give AI agents unlimited freedom. It is to provide the right level of authority, monitor their actions, and keep people accountable for the results.

SAS sponsored AI & Big Data Expo Europe. According to the original interview, Profi was scheduled to present Why the Leaders Who Win the Agentic AI Race Won’t Be the Ones Who Moved Fastest, with SAS experts also available at booth #304.

Frequently Asked Questions

1. What is autonomous AI agent governance?

It is the process of controlling how AI agents access data, make decisions, use tools, and perform actions through permissions, monitoring, testing, and human oversight.

2. Why do autonomous AI agents need human oversight?

Human oversight helps prevent harmful or unauthorized actions, especially when decisions affect money, health, privacy, or other important interests.

3. How can businesses make AI agents safer?

Businesses can restrict permissions, improve data quality, test workflows, record activities, monitor performance, and require approval for high-risk actions.

4. How does AI governance improve business efficiency?

Reusable policies and automated controls reduce repeated manual reviews, help teams apply consistent rules, and support safer AI deployment.

5. What is the difference between generative AI and agentic AI?

Generative AI creates content such as text or code. Agentic AI can plan tasks, interact with tools, and take actions toward a goal with varying levels of independence.