Guide to the future-proof Azure Monitoring

The adoption of Microsoft Azure and the public cloud, in general, grows quickly. There are obvious advantages from moving traditional servers to virtual machines in Azure, such as provisioning and administration of the servers.

Adopting microservices and elastic resources represent a material change from a traditional server-based or monolithic approach and with different benefits and advantages.

Get the page into your mailbox so you can easily access it at any time.

Lift & Shift Azure Adoption

With “lift-and-shift” approaches where the “servers,” applications, or services are not re-designed but “just moved” to a different infrastructure the monitoring challenge should not necessarily be very different. 

You can likely apply the same monitoring practice you have today by installing agents to collect performance metrics from the VMs that you run in Azure and probably use your existing monitoring solution in the traditional sense. 

Alternatively, you can leverage the out-of-the-box performance metrics made available through the Azure Monitor API. The data from the Azure Monitor API can be consumed by a range of monitoring solutions including Microsoft’s own tools.

A list of Microsoft Partner tools that integrate with the Azure Monitor API is available here.

AIMS service providers 1

Microservices / Serverless Adoption

Many will be attracted to the serverless options that Azure (and other clouds) provide. These microservices in Azure include services like Logic Apps, Event hub, Function apps, Web Sites, API management, and Service bus often have elastic options. Elastic means that the capacity of the services scales automatically with the processing need required.

Here is an example: a customer-facing application built on elastic services will auto-scale the service if customer consumption of the service increases beyond the capacity of the default set-up. So, as an IT admin, ops, or developer team, you can disregard the necessary scaling administration required for VM-based or physical services.

However, keep in mind that the billing will also be elastic, but microservices do not need to be elastic. Microservices can also be configured for static capacity. In a static capacity configuration, traditional capacity management and performance monitoring is required.

Hybrid Challenges

A hybrid environment with resources across Azure and local data centers or a hybrid multi-cloud set-up leveraging Azure and other public clouds such as Google Cloud and AWS will be a valid option for many companies.

From a monitoring perspective, it’s imperative to think through the current and future interaction of your infrastructure, applications, and services across technology silos and hybrid environments. Most will see that the dynamics and agility of modern development projects will mean that interaction between data center locations and cloud already happens, or is extremely likely in the future. The ONLY conclusion is that monitoring needs to be unified across all the hybrid locations (local and cloud).

Typology discovery AIMS

Azure costs & billing

The cost of the cloud has been and will be a concern at a certain scale and usage. Dropbox chose to do a reverse migration from AWS to local data centers. The challenge with cloud costs has also been thoroughly covered by the venture firm Andreesen Horowitz - The Cost of Cloud - A Trillion Dollar Paradox.

“Now, there is a growing awareness of the long-term cost implications of cloud. As the cost of cloud starts to contribute significantly to the total cost of revenue (COR) or cost of goods sold (COGS), some companies have taken the dramatic step of “repatriating” the majority of workloads (as in the example of Dropbox) or in other cases adopting a hybrid approach (as with CrowdStrike and Zscaler). Those who have done this have reported significant cost savings: In 2017, Those who have done this have reported significant cost savings: In 2017, Dropbox detailed in its S-1 a whopping $75M in cumulative savings over the two years prior to IPO due to their infrastructure optimization overhaul, the majority of which entailed repatriating workloads from public cloud.”  Andreesen Horowitz”

Still, for many and probably most, cloud and Azure represent cost efficiency until a certain scale or usage kicks in. Most companies are not like Dropbox who has the skills, capacity, and ability to reverse migrate. Many companies take a hybrid approach to hedge their bets to not get too reliant on one cloud platform. What is abundantly clear is that controlling cost in Azure is important and should be a part of the monitoring setup.

Sign up for AIMS and start monitoring your Azure environment for free

Step 1

Sign up for the free AIMS Community Edition

Step 2

You will be forwarded to the final confirmation & registration page. On completion, you will gain access to your personal AIMS environment. You will receive an email with further information shortly.

Step 3

Watch as AIMS AI learns your topology and normal behavior patterns.

Want to learn more? Schedule a demo.

Azure monitoring can be a massive scope

“Azure monitoring” sounds like a specific, contained scope. It can be, but often it's the contrary. Azure is a massive, and essentially a data center with the available technologies and resources quickly growing and changing. To benefit from Azure, your usage will likely be similar (quickly growing and changing).   

This is a list of default resources available. This list primarily includes Microsoft’s resources and does not include a significant list of other Microsoft resources and third-party resources. As such, in the broad sense, monitoring Azure, is more like monitoring of a data center than any narrow or defined resource set.

Depending on your situation, you should consider if the need is specific and defined or organizational-wide. At AIMS, we will always argue that no modern application lives in a silo, hence, the answer to the question is that your organization needs an organizational-wide monitoring approach that covers all technologies and resources interacting.

Summary: The Azure challenges you need to consider

So, what are the Azure challenges you need to take into account for your organization?

  1. Hybrid scenarios: You should be prepared for monitoring, which includes resources in Azure and outside of Azure (on-prem, other clouds).

  2. Costs: As documented by Andreesen Horowitz and Dropbox, you need to be conscious of costs.  Hence, cost/billing is an integral part of Azure monitoring.

  3. Scope of resources: Modern development is API-based and technology adoption (and service integration) is agile, frequent, and rapid. You need to be prepared with a monitoring solution that adapts to any Azure resource (and hence any performance metric). 

What are the requirements for modern monitoring solutions?

The days of tech teams defining alerting based on selecting specific performance parameters and setting static alerting thresholds are gone. Traditional tools were built for monolith applications and not for highly integrated and agile cloud, microservice, and hybrid environments often spanning beyond the organization and industry supply chains. Hence, the dynamic nature of modern applications means you need another approach.

This change and the consequential changed requirements to monitoring solutions are supported by the industry analysts:

 

“The complexity of IT systems has been exponentially increasing for the past several years. This has led to I&O teams that lack modernized skill sets, processes, and tools to struggle with achieving complete visibility into the digital services they provide to customers.”

Forrester logo

 

“By 2024, 30% of business leaders will rely on AI in IT Operations (AIOps) platforms for automated insights to drive business-related decisions, as compared to less than 3% today.”

209-2094877_gartner-logo-white-gartner-logo-white-png

 

For a free, open introduction to AIOps, you can check out Gartner & How to get started with AIOps.

Criteria for future-proofing (Azure) monitoring

So, what are the criteria you should evaluate when choosing a monitoring solution?

The overall challenge is to implement a monitoring solution with a high degree of automation with support for any current and future technology. This will allow you to reduce the manual effort associated with monitoring, troubleshooting while ensuring you have a future-proof implementation. If you succeed, you will free up capacity in IT to deliver on business projects while maintaining the control and insight that your business expects.

In the following, we are digging a bit deeper into a few key capabilities that should be on your list of requirements from a vendor:

  1. The needle in the haystack and the unknowns: Neither you nor any monitoring vendor knows which technologies or resources are correct to monitor in a modern application.  Recent history is full of examples of massive IT downtime across industries. What is common among most cases is that the cause of the downtime was unexpected or previously unknown. Searching for an unknown in a modern application is often like the search for the needle in the haystack. 

    The learning here is that you cannot select a few performance metrics. You need to get a comprehensive data capture of performance data across all layers and resources.  This is what is often referred to as Observability and for all relevant resources.

  2. When static thresholds no longer apply, what does?  You can safely assume that by going from manually selected metrics/ KPIs to harvesting all relevant performance data, you easily multiply the metrics and resources monitored by a factor of 100x. With this scope of metrics, it is easy to understand that static thresholds have zero chance of success (if static thresholds ever had any success at all). This is where machine learning and artificial intelligence come into play. 

    To succeed with modern applications, you rely on a solution that learns the behavior of each metric monitored. This means that instead of one flat static threshold, the system will have thousands of thresholds (typically one or more for every minute of the day) for each metric. And such solutions will keep updating these thousands or millions of thresholds based on the new behavior. This means that a modern monitoring solution will have a rich digital DNA of how your IT systems should behave at any point in time, and for all relevant metrics across all relevant resources.

  3. Instrumentationless auto-discovery: This allows you to understand relationships and dependencies across your IT resources without manual effort. What if you can reduce the time and dependency on manual documenting your enterprise architecture?  Automated monitoring with the additional capability of discovering relationships and dependencies is a force multiplier in your monitoring setup. 

    An alert should give you sufficient insight to i) understand what is happening ii) what triggered the alert iii) which systems are impacted / can be impacted.  The Discovery of relationships between resources is key to efficiently understand the potential impact on your business from an alert. 

  4. Monitoring is no longer contained to tech teams: You need intuitive insight that is exposed to the business.  With IT driven by digitalization, success is measured by revenues and profitability.  A consequence of digitalization is that stakeholders across the organization require insight to understand the performance of the “IT factory” that delivers business value. 

    Again, automation is key. Everyone with some experience in IT development or IT operations has felt the challenge of communicating the status of an application to non-tech stakeholders. The lack of common vocabulary to communicate should be resolved by intuitive live dashboards presenting the status of key business applications.

  5. Time to value in hours, not weeks or months: We should expect more from monitoring vendors. We all have gotten used to SaaS applications with quick time to value. The same should apply to monitoring vendors while complying with the needs outlined above. 

    Look for a provider that can satisfy automation, future-proofing with a low installation/onboarding friction that allows you to validate value with no more than a few hours of your time.  

What are your options? Microsoft, incumbents, and an innovative approach.

Microsoft obviously has its own solutions for monitoring Azure. We have written a separate blog on How to monitor Azure with Microsoft tools.  We will not go deeper into Microsoft’s tools for monitoring Azure than stating that it's manual and require significant effort. Below, we have linked to a set of resources to look into if Microsoft is your go-to vendor:

Several companies integrate with Azure with various monitoring approaches.  You can view the list of vendors integrating with Microsoft Azure Monitor to access performance metrics from Azure here

AIMS - A different approach to monitoring Azure

If you recognize the challenges and needs we have outlined in this article, AIMS could be a solution for you.

But, don’t take our word for it. Sign-up for the AIMS Community Edition and experience the AIMS Capabilities for Azure monitoring for free. You should be embarrassed if you spend more than 1 hour on the set-up :)

Verify for yourself how AIMS satisfies automated and future-proof monitoring:

Sign up for AIMS and start monitoring your Azure environment

Step 1

Sign up for the free AIMS Community Edition

Step 2

You will be forwarded to the final confirmation & registration page. On completion, you will gain access to your personal AIMS environment. You will receive an email with further information shortly.

Step 3

Watch as AIMS AI learns your topology and normal behavior patterns.

Learn more