AI agents are transforming industries by automating workflows, enhancing productivity, and enabling intelligent decision-making. Businesses are leveraging AI agents to process insurance claims, manage IT service desks, optimize supply chain logistics, and even assist healthcare professionals in analyzing medical records. The potential is vast, and we’re excited to introduce two powerful innovations in Azure AI Foundry:
- Responses API: A powerful API enabling AI-powered applications to retrieve information, process data, and take action seamlessly.
- Computer-Using Agent (CUA): A breakthrough AI model that navigates software interfaces, executes tasks, and automates workflows.
Together, these capabilities empower businesses to reimagine AI not just as an assistant—but as an active digital workforce. Enterprise customers will soon gain access to these innovations driving automation, efficiency, and intelligence at scale.
Enhancing AI Agents with the Responses API
The Responses API is the key to unlocking agentic AI in Azure AI Foundry, transforming how enterprises harness AI for real-world impact. It is the new foundation for leveraging Azure OpenAI Service’s powerful built-in tools, combining the simplicity of the Chat Completions API with the advanced capabilities available through Assistants API and Azure AI Agent Service. The Responses API enables seamless interaction with tools like CUA, function calling, and file search—all in a single API call. This API enables AI systems to retrieve data, process information, and take actions—seamlessly connecting agentic AI with enterprise workflows.
How the Responses API Works
The Responses API provides a structured response format that allows AI to interact with multiple tools while maintaining context across interactions. It supports:
- Tool calling in one simple API call: Now, developers can seamlessly integrate AI tools, making execution more efficient.
- Computer use: Use the computer use tool within the Responses API to drive automation and execute software interactions.
- File search: Interact with enterprise data dynamically and extract relevant information.
- Function calling: Develop and invoke custom functions to enhance AI capabilities.
- Chaining responses into conversations: Keep track of interactions by linking responses together using unique response IDs, ensuring continuity in AI-driven dialogues.
- Enterprise-grade data privacy: Built with Azure’s trusted security and compliance standards, ensuring data protection for organizations.
By consolidating retrieval, reasoning, and action execution into a single API, the Responses API simplifies AI agent development, reducing the complexity of orchestrating multiple AI tools within an automation pipeline.
This scalability makes it well-suited for enterprise use cases across industries such as customer service, IT operations, finance, and supply chain management, where AI-powered automation can streamline workflows and improve efficiency. For even greater flexibility and control, organizations can explore Azure AI Agent Service, which offers additional tools and models for developing and scaling AI agents. Azure AI Agent Service integrates with Semantic Kernel and AutoGen, enabling seamless multi-agent orchestration for more complex scenarios requiring multiple agents to collaborate on tasks.
Empowering AI Agents with the Computer-Using Agent
The Computer-Using Agent (CUA) is a specialized AI model in Azure OpenAI Service that allows AI to interact with graphical user interfaces (GUIs), navigate applications, and automate multi-step tasks—all through natural language instructions. Unlike traditional automation tools that rely on predefined scripts or API-based integrations, CUA can interpret visual elements, adapt dynamically, and take action based on on-screen content.
What makes the Computer-Using Agent unique?
- Autonomous UI navigation: Can open applications, click buttons, fill out forms, and navigate multi-page workflows.
- Dynamic adaptation: Interprets UI changes and adjusts actions accordingly, reducing reliance on rigid automation scripts.
- Cross-application task execution: Operates across web-based and desktop applications, integrating disparate systems without API dependencies.
- Natural language command interface: Users can describe a task in plain language, and CUA determines the correct UI interactions to execute.
With today’s announcement, developers can start building additional agentic capabilities right away with CUA. As enterprises look to deploy this technology at scale, we are evaluating integration with Windows 365 and Azure Virtual Desktop to enable CUA automation to run seamlessly in a managed host environment on Cloud PCs or virtual machines (VMs), ensuring consistent performance while maintaining enterprise compliance and security standards.
Ensuring secure and trustworthy AI automation
As AI systems become more autonomous, ensuring security, reliability, and alignment with human intent is critical. The CUA model is one of the first agentic AI models capable of directly interacting with software environments, bringing new challenges in misuse prevention, unintended actions, and adversarial risks. To address these, Microsoft and OpenAI have implemented a multi-layered safety approach spanning the model, system, and deployment levels.
The CUA model is developed with safeguards to refuse harmful tasks, reject unauthorized actions, and prevent misuse. At the system level, Microsoft implements enterprise-grade content filtering and execution monitoring to help detect and prevent policy violations. To minimize unintended actions, CUA is designed to request user confirmations before executing irreversible tasks and to restrict high-risk actions such as financial transactions.
Microsoft’s Trustworthy AI framework further ensures real-time observability, logging, and compliance auditing for enterprise deployments. Automated and human-in-the-loop detection systems monitor execution patterns, identifying anomalous behaviors and enforcing governance policies. These safeguards are continuously refined based on internal red-teaming, external audits, and real-world testing to strengthen protection against prompt injections, adversarial manipulations, and unauthorized access. Given the current reliability level of the CUA model—particularly in non-browser environments—human oversight remains strongly recommended for sensitive operations.
As AI agents evolve, Microsoft is committed to transparency, security, and ongoing risk mitigation. By combining CUA’s built-in safeguards with Azure’s enterprise compliance and governance tools, organizations can deploy AI-powered automation with confidence, ensuring safe and responsible AI adoption at scale.
Getting started with CUA and Responses API
Azure AI Foundry continues to push the boundaries of AI-powered automation. Enterprise customers will gain access to the Responses API and CUA in Azure OpenAI Service in the coming weeks.
We’re excited to see how developers and businesses innovate with these new capabilities.
The post Announcing the Responses API and Computer-Using Agent in Azure AI Foundry appeared first on Microsoft Azure Blog.
Leave a Reply