Guide to the future-proof Azure Monitoring

The adoption of Microsoft Azure and the public cloud, in general, grows quickly. There are obvious advantages from moving traditional servers to virtual machines in the Azure cloud, such as provisioning and administration of the servers.

Adopting microservices and elastic resources represent a material change from a traditional server-based or monolithic approach and with different benefits and advantages.

Get the page into your mailbox so you can easily access it at any time.

Lift & Shift Azure Adoption

With “lift-and-shift” approaches where the “servers,” applications, or services are not re-designed but “just moved” to a different infrastructure the monitoring challenge should not necessarily be very different.


You can likely apply the same application performance monitoring practice you have today by installing agents to collect data and real-time performance metrics from the VMs that you run in Azure and probably use your existing Azure monitoring tool in the traditional sense.


Alternatively, you can leverage the out-of-the-box performance Azure monitor metrics and analytics made available through the Azure Monitor API. The data from the Azure Monitor API can be consumed by a range of monitoring platforms including Microsoft’s own tools.

A list of Microsoft Partner tools that integrate with the Azure Monitor API is available here.

AIMS service providers 1

Microservices / Serverless Adoption

Many will be attracted to the serverless options that Azure (and other cloud services) provide. These microservices in Azure include services like Logic Apps, Event hub, Function apps, Web Sites, API management, and Service bus often have elastic options. Elastic means that the capacity of the services scales automatically with the processing need required.

Here is an example: a customer-facing application built on elastic services will auto-scale the service if customer consumption of the service increases beyond the capacity of the default set-up. So, as an IT admin, Ops, or developer team, you can disregard the necessary scaling administration required for VM-based or physical services.
However, keep in mind that the billing will also be elastic, but microservices do not need to be elastic. Microservices can also be configured for static capacity. In a static capacity configuration, traditional capacity management and performance monitoring is required.

Hybrid Challenges

A hybrid environment with resources across Azure and local data centers or a hybrid cloud set-up using Azure and other public clouds such as Google Cloud and AWS will be a valid option for many companies.

From an application monitoring perspective, it’s imperative to think through the current and future interaction of your infrastructure, applications, and services across technology silos and hybrid environments. Most will see that the dynamics and agility of modern development projects will mean that interaction between data center locations and cloud already happens, or is extremely likely in the future. The ONLY conclusion is that monitoring data needs to be unified across all the hybrid locations (local and cloud).

Typology discovery AIMS

Microsoft Azure subscription costs & billing

The cost of the cloud has been and will be a concern at a certain scale and usage. Dropbox chose to do a reverse migration from AWS to local data centers. The challenge with cloud costs has also been thoroughly covered by the venture firm Andreesen Horowitz - The Cost of Cloud - A Trillion Dollar Paradox.

“Now, there is a growing awareness of the long-term cost implications of cloud. As the cost of cloud starts to contribute significantly to the total cost of revenue (COR) or cost of goods sold (COGS), some companies have taken the dramatic step of “repatriating” the majority of workloads (as in the example of Dropbox) or in other cases adopting a hybrid approach (as with CrowdStrike and Zscaler). Those who have done this have reported significant cost savings: In 2017, Those who have done this have reported significant cost savings: In 2017, Dropbox detailed in its S-1 a whopping $75M in cumulative savings over the two years prior to IPO due to their infrastructure optimization overhaul, the majority of which entailed repatriating workloads from public cloud.”  Andreesen Horowitz”

Still, for many and probably most, cloud and Azure represent cost efficiency until a certain scale or usage kicks in. Most companies are not like Dropbox who has the skills, capacity, and ability to reverse migrate. Many companies take a hybrid approach to hedge their bets to not get too reliant on one cloud platform. What is abundantly clear is that controlling cost in Azure is important and should be a part of the cloud monitoring services setup.

Build confidence in your IT Operations

Get value out of AIMS in days - not months or years


Step 1: Connect
Connect Azure to AIMS to analyze data.

Step 2: Analyze and learn
Access anomalies and correlations within 2 weeks.

Step 3: Scale with confidence
Confidently scale your IT monitoring and operations with almost 0 maintenance as AIMS automatically discovers new resources and adjusts thresholds.

 

Complete Azure monitoring can be a massive scope

“Azure environment monitoring” sounds like a specific, contained scope. It can be, but often it's the contrary. Azure is a massive, and essentially a data center with the available technologies and resources quickly growing and changing. To benefit from services such as Azure, your usage will likely be similar (quickly growing and changing).

This is a list of default resources available. This list primarily includes Microsoft’s resources and does not include a significant list of other third-party resources. As such, in the broad sense, monitoring Azure, is more like monitoring of a data center than any narrow or defined resource set.

Depending on your situation, you should consider if the infrastructure monitoring need is specific and defined or organizational-wide. At AIMS, we will always argue that no modern services and applications live in a silo, hence, the answer to the question is that your organization needs an organizational-wide monitoring approach that covers all technologies and resources interacting.

Summary: The Azure challenges you need to consider

So, what are the Azure challenges you need to take into account for your organization?

  1. Hybrid scenarios: You should be prepared for comprehensive monitoring, which includes resources in Azure and outside of Azure (cloud and on-premises).

  2. Costs: As documented by Andreesen Horowitz and Dropbox, you need to be conscious of costs. Hence, cost/billing is an integral part of Azure monitoring.

  3. Scope of resources: Modern development is API-based and technology adoption (and service integration) is agile, frequent, and rapid. You need to be prepared with a monitoring tool that adapts to any Azure resource (and hence any Azure performance metric).

What are the requirements for modern third-party Azure monitoring tools?

The days of tech teams defining alerting based on selecting specific performance parameters and setting static alerting thresholds are gone. Traditional tools were built for monolith applications and not for highly integrated and agile cloud, microservice, and hybrid environments often spanning beyond the organization and industry supply chains. Hence, the dynamic nature of modern applications and services means you need another approach. This change and the consequential changed requirements to monitoring platforms are supported by the industry analysts:

 

“The complexity of IT systems has been exponentially increasing for the past several years. This has led to I&O teams that lack modernized skill sets, processes, and tools to struggle with achieving complete visibility into the digital services they provide to customers.”

Forrester logo

 

“By 2024, 30% of business leaders will rely on AI in IT Operations (AIOps) platforms for automated insights to drive business-related decisions, as compared to less than 3% today.”

209-2094877_gartner-logo-white-gartner-logo-white-png

 

For a free, open introduction to AIOps, you can check out Gartner & How to get started with AIOps.

Criteria for future-proofing (Azure) monitoring

So, what are the criteria you should evaluate when choosing a monitoring tool? The overall challenge is to implement a monitoring platform with a high degree of automation with support for any current and future technology. This will allow you to reduce the manual effort associated with customized monitoring, troubleshooting while ensuring you have a future-proof implementation. If you succeed, you will free up capacity in IT to deliver on business projects while maintaining the control and insight that your business expects. In the following points, we are digging a bit deeper into a few key monitoring capabilities and best practices that should be on your list of requirements from a vendor:
 
  1. The needle in the haystack and the unknowns: Neither you nor any monitoring vendor knows which technologies or resources are correct to monitor in a modern application.  Recent history is full of examples of massive IT downtime across industries. What is common among most cases is that the cause of the downtime was unexpected or previously unknown. Searching for an unknown in a modern application is often like the search for the needle in the haystack. 
    The learning is that you cannot select a few performance metrics. You need to get a comprehensive data capture of performance data across all layers and resources.  This is what is often referred to as Observability and for all relevant resources.

  2. When static thresholds no longer apply, what does?  You can safely assume that by going from manually selected metrics/ KPIs to harvesting all relevant performance data, you easily multiply the metrics and resources monitored by a factor of 100x. With this scope of metrics, it is easy to understand that static thresholds have zero chance of success (if static thresholds ever had any success at all). This is where machine learning and artificial intelligence come into play to help you understand the performance of your Azure applications.

  3. To succeed with modern applications, you rely on a solution that learns the behavior of each metric monitored. This means that instead of one flat static threshold, the system will have thousands of thresholds (typically one or more for every minute of the day) for each metric. And such a monitoring system will keep updating these thousands or millions of thresholds based on the new behavior. This means that a modern monitoring solution will have a rich digital DNA of how your IT systems should behave at any point in time, and for all relevant metrics across all relevant resources.

  4. Instrumentationless auto-discovery: This allows you to understand relationships and dependencies across your IT resources, without manual effort. What if you can reduce the time and dependency on manual documenting your enterprise architecture?  Automated monitoring with the additional capability of discovering relationships and dependencies is a force multiplier in your monitoring setup. 

  5. An alert should give you sufficient insight to i) understand what is happening ii) what triggered the alert iii) which systems are impacted / can be impacted.  The Discovery of relationships between resources is key to efficiently understanding the potential impact on your business from an alert. 

  6. Monitoring is no longer contained to tech teams: You need intuitive insight that is exposed to the business.  With IT driven by digitalization, success is measured by revenues and profitability.  A consequence of digitalization is that stakeholders across the organization require insight to help you monitor and to understand the performance of the “IT factory” that delivers business value.  

  7. Again, automation is key. Everyone with some experience in IT development or IT operations has felt the challenge of communicating the status of an application to non-tech stakeholders. The lack of common vocabulary to communicate should be resolved by intuitive live dashboards presenting the status of key business applications.

  8. Time to value in hours, not weeks or months: We should expect more from monitoring vendors. We all have gotten used to SaaS applications with quick time to value. The same should apply to monitoring vendors while complying with the needs outlined above. 

  9. Look for a provider that can satisfy automation, future-proofing with a low installation and onboarding friction that allows you to validate value with no more than a few hours of your time.  
 

What are your options? Microsoft, incumbents, and an innovative approach.

Microsoft obviously has its own solutions for monitoring Azure. We have written a separate blog on How to monitor Azure with Microsoft tools. Below, we have linked to a set of resources to look into if Microsoft is your go-to vendor:

Several companies have Azure integrations with various monitoring approaches. You can view the list of vendors integrating with Microsoft Azure Monitor to access performance metrics from Azure here. 

AIMS - A different approach to monitoring

If you recognize the challenges and needs we have outlined in this article, AIMS could be a perfect solution for you.


But, don’t take our word for it. Sign-up for the AIMS Community Edition and experience the AIMS Capabilities for Azure monitoring for free. We should be embarrassed if you spend more than 1 hour on the set-up :)


Verify for yourself how AIMS satisfies automated and future-proof monitoring:

AIMS AIOps

Automated Azure Monitoring with Artificial Intelligence

What if Microsoft Azure monitoring can be completely automated, removing the traditional monitoring manual labor of setting up and defining static thresholds and defining alerting scenarios that typically end up only covering a very limited part of the monitoring unit? See how it could be done in less than 15 minutes.

More about Azure Monitoring
Azure monitoring - everything you need to know AIMS

More about Microsoft Azure monitoring tools, use cases, tips, and "how-to" guides

Read about Azure Service Bus monitoring

How to monitor Azure API Management

Tips to Monitor Azure SQL Databas

How to monitor Redis / Cache in Azure