Observability from cloud to edge in Azure

This post is co-authored by Rahul Bagaria, Principal Product Manager, Azure Monitor Customer Success

Our customers are transforming their digital environments, whether migrating workloads to Azure, building new cloud-native apps, or unlocking new scenarios at the edge. As they combine these strategies to meet their business needs, they must also maintain their existing environments. It’s critical that customers can monitor the health, performance, and security of their mission-critical systems, and an observability solution is a best practice for streamlined well-governed management.

With Azure Monitor, our approach to observability centers on simplicity. We know from our customers how important it is to have monitoring available out-of-the-box and to easily get started with samples and recommendations including effective alerts, optimal queries, and customizable reports. To ensure that you can monitor your largest production workloads anywhere in a trustworthy manner, we have designed our data platform to handle thousands of terabytes of data per day easily and reliably. We also understand that our customers have their workloads in Azure, their on-premises datacenters, edge, and multi-cloud environments, and thus extend Azure Monitor through Azure Arc so that our customers have the flexibility to run their applications anywhere and monitor or manage them effectively.

With 99 percent of the top 1,000 Azure customers using advanced capabilities of Azure Monitor, large enterprises using Azure offer a master class in well-monitored environments across cloud, on-premises datacenters, and the edge. As customers including Ernst & Young, Nokia, LinkedIn, Mercedes Benz, and Asos.com trust Azure Monitor for their observability needs, several themes emerge for companies of all sizes to be successful in building a well-monitored environment.

Ability to observe at any level across the stack and get deep insights

• Rich insights: You can take advantage of curated visualizations, reports, and diagnostic tools for specific resources, using insights from Azure Monitor. Application Insights provides application performance management (APM) capabilities, and you can use VM insights, Container insights, or Network Insights (and many more) for infrastructure monitoring.
• Distributed tracing: Tracing is a key pillar of observability, and with Azure Monitor, you can easily correlate transactions end-to-end from apps to dependencies to infrastructure. There are multiple topology views built-in like Application Map, VM Map, and Network Map for you to visualize the architecture or drill down on transactions. With our investments in OpenTelemetry, we are starting to add support for vendor-agnostic tracing as well.
• Log analytics: The heart of Azure Monitor is our powerful centralized logs platform which stores together all logs in Azure across monitoring, security, and management. You can even send your own custom logs, define schemas and transforms, and take advantage of the full power of log analytics. You get a rich query language (KQL) capable of correlations, troubleshooting, analytics, and even AIOps. This platform also powers the security monitoring and SIEM scenarios in Azure with Microsoft Sentinel and Defender for Cloud.

Open and extensible platform for partners and customers to innovate

• Partners and integrations: While Azure Monitor provides rich observability and analytics capabilities, as a customer you have a choice to use any monitoring or analytics solution that suits your requirements, and we will fully support that from the platform. We integrate data from Azure resources with partner solutions like Datadog and Elastic (now natively available in Azure) using the same underlying platform that powers our experiences. We also provide hooks for you to export data, connect alerts with ITSM systems, or shift left by incorporating monitoring within your DevOps toolchains.
• Open-source solutions: If you need to use open-source metrics or logging solutions alongside Azure Monitor, we support multiple CNCF (Cloud Native Compute Foundation) projects. You can seamlessly scrape Prometheus metrics for Kubernetes clusters with Container insights, and there is a Logstash output plugin for sending custom logs to Azure Monitor.
• Operational dashboarding: Beyond the dashboarding and reporting capabilities natively available in Azure, Grafana provides a very rich single-pane-of-glass visualization solution for multi-cloud environments. Azure Monitor is natively part of the core Grafana software with a variety of dashboarding templates that you can build on.

Enterprise-ready for mission-critical scenarios

• Privacy: Azure Monitor is fully GDPR compliant and does not collect any PII out-of-the-box. We do support data purging on request. Lockbox protection allows you to control access to any data you are collecting requests during support incidents.
• Security: For all your logs in Azure Monitor, we provide data encryption at rest with customer-managed keys (CMK) in your Azure Key Vaults. There is even more security at the infrastructure level with 256-bit AES encryptions. You can take advantage of private links support to connect securely to any of your private network endpoints. To ensure further trust and data security, we recommend you configure agents to use at least Transport Layer Security (TLS) 1.2, and if needed, ingest Azure Active Directory authenticated logs into Azure Monitor.
• Compliance: We provide many capabilities to help you meet any compliance requirements in your organization, industry, or geography. You can leverage activity logs and audit logs for security compliance and retain or archive specific data for long durations (up to 7 years) as needed, with additional support for data immutability (in Azure Storage). Overall, we comply with most of the data residency and sovereignty requirements and are even targeting to support Schrems II very soon.

For a more in-depth look at Azure and observability, you can learn more about monitoring best practices for your cloud and edge environments with our dedicated guidance cookbooks; we welcome you to reach out to us with any questions or feedback on our Tech Community.