At Build 2024, Microsoft has announced a series of updates aimed at enhancing its platforms for building generative AI-powered applications. The updates, revealed at the Build conference, target Azure AI Studio and Copilot Studio, two key tools in Microsoft's AI ecosystem.
Azure AI Studio: New Capabilities and Integrations
Azure AI Studio, Microsoft´s unified platform for developing and deploying generative AI apps via Azure OpenAI Service and more than 1.600 other models and tools, is now generally available. It allows developers to create generative AI applications using pay-as-you-go inference APIs. This “model-as-a-service” approach initially includes models from Nixtla and Core42 AI, with future additions from Cohere, Stability AI, and AI21 Labs.
The studio now supports code-first development experiences through integrations with Azure Developer CLI and AI Toolkit for Microsoft Visual Studio Code. Additionally, it includes new tools for training, debugging, and monitoring AI-powered applications, offering features to visualize trends and set custom alerts.
Enhanced Security and Compliance
To address security concerns, Azure AI Studio integrates with Microsoft Purview, which helps prevent unauthorized data access and manage AI app usage. It also introduces tools to prevent “jailbreaks” of AI models and detect hallucinations, where models generate incorrect information.
Copilot Studio: Advanced AI Bots and Customization
Copilot Studio introduces Copilot agents, AI bots designed to handle specific tasks and roles independently. These agents use memory and contextual knowledge to navigate business workflows, learning from user feedback and seeking assistance when necessary.
Developers can now extend and customize these AI-powered copilots using extensions and connectors. Extensions allow for the integration of specific instructions, database knowledge, and plugin actions, enabling tasks like expense reporting and employee onboarding. Connectors provide access to various data sources, including public websites, SharePoint, OneDrive, and third-party applications.
New Search and Speech Capabilities
Microsoft has also enhanced search relevance within Azure AI Search, improving data retrieval for retrieval-augmented generation (RAG) and enterprise search. These enhancements include hybrid search, re-ranking, and increased storage capacity, which boost the scalability of generative AI applications.
Azure AI Speech is receiving updates as well, with new features like speech analytics and video dubbing services, enhancing the capabilities of AI-driven speech applications.
New Model Types and APIs
A new model type, “Custom generative,” is launching in preview, allowing users to create models from single documents without extensive data labeling. Additionally, the Azure OpenAI Service is adding an Assistants API to help developers build more advanced virtual assistants and chatbots.
Phi-3-Vision and AI Safety Features
Microsoft is introducing Phi-3-vision, a multimodal large language model (LLM) designed for visual, chart, graph, and table reasoning. This model will be available in Azure AI Studio's model-as-a-service catalog.
To enhance AI safety, new features include Custom Categories for content filtering and Prompt Shields to mitigate malicious prompt injection attacks.
Guidance and Reference Architectures
Microsoft has also released a series of reference architectures and implementation guidance to assist developers in designing and optimizing AI-powered applications, ensuring they can effectively leverage the new tools and capabilities introduced.
