The ultimate guide to Biztalk Monitoring

Microsoft BizTalk is one of the most robust and reliable integration engines which have been used by various businesses for almost 2 decades. A lot of organizations are relying on Microsoft BizTalk Server to handle their highly complicated environment. It is obvious that it is essential to monitor the BizTalk server applications and the infrastructure on a regular basis to make sure the issues are resolved timely and to keep the environment healthy and always accessible.  

BizTalk is at the heart of most integrations – sitting in the middle of the enterprise architecture for all internal and external transactions. As a result of its central role, BizTalk can often end up as a single point of failure, where a minor and unforeseeable issue can result in extended downtime with myriad impacts for your business.

Get the page into your mailbox so you can easily access it at any time.

First thing first, what is Biztalk Server?

According to the BizTalk Server documentation,“BizTalk Server is a publish and subscribe architecture that uses adapters to receive and send messages, implements business processes through orchestration, and includes management tracking of these different parts. BizTalk Server also includes trading partner management for business-to-business messaging, high availability to maximize uptime, a development platform to create your own components, an administration console to manage your artifacts, and business activity monitoring to manage aggregations, alerts, and profiles.”

BizTalk AIMS

The heart of the product is the BizTalk Server Engine and it consists of two main parts:

  • A messaging component that provides the ability to communicate with a range of other software. By relying on adapters for different kinds of communication, the engine can support a variety of protocols and data formats, including Web services and many others.

  • Support for creating and running graphically defined processes called orchestrations. Built on top of the engine’s messaging components, orchestrations implement the logic that drives all or part of a business process.

Several other BizTalk components can also be used in concert with the engine, including:

  • A Business Rule Engine that evaluates complex sets of rules.

  • A Group Hub that lets developers and administrators monitor and manage the engine and the orchestrations it runs.

  • An Enterprise Single Sign-On (SSO) facility that provides the ability to map authentication information between Windows and non-Windows systems.

Why is BizTalk (still) important?

Simply put, BizTalk allows applications to talk to each other. It’s used to share information or events between internal or external applications linked together to perform business processes. All the systems used by your enterprise can connect to BizTalk, as well as systems used by any other companies you need to communicate with.

Companies (like yours) are still investing heavily in BizTalk and will continue to do so for years. BizTalk is often used for companies’ most business-critical applications, and as companies move to a hybrid integration scenario (more on that below) those systems are often the last ones to be migrated and some never will.

AIMS service providers 1

Where do most BizTalk performance problems originate?

Any IT manager knows that performance issues can literally come from anywhere and can be difficult to troubleshoot without sufficient data and system oversight. But there are some common hotspots where mistakes often cause issues in your BizTalk environment:

  • Any critical load (and message growth) on hardware
  • Poor system design
  • Poor infrastructure setup Throttling on hosts
  • Human errors
  • Patching and configuration changes
  • Overloading the system by (manually) enabling BizTalk tracking
  • Issues with SQL Server performance
IIn addition to the technical blunders above, a flawed approach to monitoring and IT system governance can often cause unnecessary downtime (long troubleshooting). Fundamental flaws include:

  • Assuming your necessary monitoring scope is covered
  • Manual configuration of static thresholds
  • Reliance on limited data
  • Outsourcing the problem
  • Relying on technical staff to know the business
Monitoring challenges AIMS

Monitoring Biztalk

One of the primary concerns in Biztalk Monitoring is knowing that applications are online and healthy. In this context, healthy means that artifacts like Receive Locations, Send Ports, and Orchestrations are configured and enabled.  

However, component monitoring is not the only thing that we should take care of while monitoring Biztalk. This can be multifold as there are different components and criteria that you need to consider in order to monitor the Biztalk as a whole. Some of them are:

  • There can be issues in the system or infra layer of Biztalk. For eg. DTC issues, issues in the operating system, connectivity issues, etc.
  • There could be a possibility wherein some of the components in Biztalk like Receive Location, Send Port, Host instances, or Orchestration are not running. This will create a problem in the message flow, so you need to take care of that part also. 
  • Biztalk is not processing the message with its full capacity, some slowness in some systems. 

Furthermore, when a source or destination system is no longer available, it is crucial that a BizTalk team receives a notification. So, in nutshell, monitoring Biztalk can be grouped into different categories:

  1. Availability Monitoring 
  2. Health Monitoring
  3. Performance Monitoring
  4. Threshold Monitoring
Biztalk monitoring AIMS

1. Availability Monitoring

This helps us in notifying on the availability of the various components of BizTalk or even the whole application is down. This also includes the services running on the server, which are Enterprise Single Sign-on, Rule engine, and DTC (depending on the configuration).  

One of the important points is related to Single Sign-On Service availability. It doesn’t immediately result in a BizTalk outage, however, with the Single-Sign-On Master Secret Server offline, a simple activity like Windows Patching will render your BizTalk environment inoperable. Dealing with Single Sign-On Master Secret Server issues proactively is a good practice and will prevent unnecessary downtime.

2. Health Monitoring

Monitoring the health of BizTalk server environments helps us in identifying whether our applications and their components are in good or bad condition. For example, are any applications or their constituent artifacts experiencing exception conditions? Or are messages suspended because of invalid data?

Take a look at how to run a BizTalk health check.

3. Performance Monitoring

This helps us in identifying whether our application and its components are running efficiently. It means that it will tell you what the performance of your database is, what is the performance of your CPU while processing the data, or how much your host instance is utilizing while processing the messages.  

Monitoring databases

Monitoring your BizTalk databases is one of the most important aspects of Performance Monitoring. Execution of the BizTalk Server SQL Agent jobs is crucial for managing the BizTalk Server databases and for maintaining optimal performance. Even with the jobs running successfully, there can be more causes of the uncontrolled growth of the databases like:

  • Excessive suspended message or service instances
  • Disk failures
  • High levels of tracking
  • BizTalk Server throttling
  • Poor SQL Server performance
  • Network latency issues

4. Threshold Monitoring

This type of monitoring is based on the threshold rules you have created in your environment. The rules can be typically based on the requirement of the Biztalk application and its performance over time.

For example, BizTalk Throttling configuration is a protection mechanism that will significantly slow down the rate at which BizTalk is processing messages. BizTalk will do this to prevent an all-out outage. Throttling conditions can severely back up a BizTalk farm and will probably have a negative impact on real-time processing.  

Throttling issues usually need time to work themselves out, but proper tuning can often be prevented. You can set a limit on the threshold value on the different critical database connections, maximum throttling delay as per the analysis on the load of any component of Biztalk, and based on it the threshold monitoring configured.

Threshold monitoring aims biztalk icon

The role of BizTalk in Hybrid

Since Microsoft’s introduction of Microsoft Azure as the future of integration in the cloud, the firm has actively encouraged customers to begin (or at least begin planning) a migration of on-premise BizTalk integrations to the Azure cloud or look at extending BizTalk with cloud capabilities.  

Microsoft has had mixed success here, and while some new integrations may be purely cloud-based for existing BizTalk customers, many companies still depend on BizTalk to integrate their most critical, high-security applications safely behind their own firewalls.  

And while we still see companies starting new projects using BizTalk, most teams planning to migrate existing ones will do it bit-by-bit, creating a hybrid scenario, where BizTalk will continue to act as a service bus or gateway and connect the on-premise systems as well as the cloud apps. 

Using the BizTalk Adapter Service in BizTalk 2016 makes it easy to keep connecting to new solutions no matter where and how they are built, and many BizTalk users will undoubtedly take advantage of these features to futureproof their existing BizTalk integrations.

BizTalk monitoring Aims scheme

BizTalk monitoring checklist

The goal of monitoring BizTalk server environment is to minimize the amount of time that an anomaly goes undetected, and ideally to help proactively detect situations that might cause an exception before they occur.

Change & error log

  • Logs changes on added components 
  • Logs changes on deleted components 
  • Added applications 
  • Deleted applications 
  • Messaging pattern changes
  • Renaming of components Errors

Status Monitoring 

  • Stopped notifications on BizTalk hosts
  • Ports/receive locations 
  • Orchestrations 
  • Disconnect notifications from agent 
  • Reconnect notifications from agent

Server Parameters

  • CPU 
  • Memory 
  • Disk utilization 
  • Network BizTalk spool

Ports & orchestration parameters

  • Message count 
  • Message delay 
  • Message volume per port
  • Message volume per orchestration

Host Parameters

  • Active instances 
  • Data size Inbound latency 
  • Message delivery delay 
  • Message delivery income rate 
  • Message delivery outgoing rate 
  • Delivery throttling state 
  • Outbound latency 
  • Process memory usage 
  • Queue length 
  • Resident orchestration 
  • Suspended messages

SQL Parameters

All Stored Procedures, Databases, Memory usage stats, CPU usage, I/O activities, Server wait stats, Historical blocking information, Missing indexes, Logical disk volumes, Database parameters, Database sizes, Error captures, Maintenance plan jobs, Schedules jobs, OS build settings, SQL server settings and configuration, Memory settings, CPU processor/ parallelism settings, Virtual page size, Accounts, Remote servers, Database and user rights

Messaging patterns

  • Message count 
  • Delay 
  • Message volume

AIMS - an advanced application performance monitoring solution powered by AI

If you recognize the challenges and needs we have outlined in this article, AIMS could be a solution for you.

But, don’t take our word for it. Sign-up for the AIMS Community Edition and experience the AIMS Capabilities for BizTalk monitoring for free. Monitoring your Biztalk Server has never been easier. We should be embarrassed if you spend more than 1 hour on the set-up :)

Verify for yourself how AIMS satisfies automated and future-proof monitoring:

Let's get the conversation started

Achieve effortless IT monitoring with truly automated AIOps platform.