The future of integrations with Inglorious Integrator

February 14, 2022

In this instalment, I speak with #aimsperformancepro Ahmed Taha – Technical Architect at Link Development. Ahmed has over 10 years of hands-on experience with the Microsoft integration stack, including .NET, BizTalk Server, SQL and Azure, and is one of a select few proud owners of an exclusive Inglorious Integrator t-shirt!

What is your function and in what manner are you confronted with monitoring issues?

I provide consultation, support, and design for on-premises and cloud-based integration solutions in Microsoft stack.

In my support role for existing customers integration solutions, I rely on all the available tracking and monitoring tools. The lack of proper monitoring tools, or unconfigured built-in tracking feature makes it very difficult to debug encountered issues, and I may need to have to inspect the source code, or the intermediate messages in a wiretap fashion.

What do you often encounter in terms of monitoring / what do you notice when working with customers?

I noticed that monitoring is done either inadequately, or entirely overlooked in customers integration solutions. This makes the integration solution like a black box with no visibility into the technical or business aspects of the solution.

Customers usually realize this at a later stage, after the solution was deployed and started to have technical issues that are difficult to debug, or business users want to gain insights into their business processes going through that black box.

When I work with new customers during the design phase in our projects, I properly introduce monitoring as a crucial part of the integration solution, and they usually see the value right away.

What should every IT manager responsible for integration know within an organization (what do you often hear back from customers)?

I think IT managers should always be up to speed with all the integration services whether on-premises or on the cloud into their integration toolbox. This way they can assist their customers to achieve their needs using the right tools.

Customers are now more willing than ever before to learn more about cloud-based solutions, this is partly due to todays’ cloud technologies hype, the operational expenditure pricing model, and server-less cloud services concept.

With what purpose are monitoring tools often used (which customer requests are involved, is a goal always specific)?

Mainly it used for technical monitoring. Initially, a typical request from technical operators is to have centralized dashboards containing the health statuses of all the solution components in one place, in addition to alerting capabilities. Afterwards, they take their time to further explore it and customize it as needed.

In the cases where business users required business reports, these requests tend to be much more detailed, as they know exactly what they want to see in the business reports.

In what scenario is monitoring most often used, first-line monitoring (functional) vs. second-line monitoring (technical)?

In my experiences, it is mostly used for technical monitoring for each component, or system in the solution.

What is missing in the current monitoring solutions?

Having the capability to automatically, and correctly get to the root cause of encountered issues from all the ingested data & metrics across the different systems, without having an Administrators to make sense of all these metrics.

Can you share an example(s) of a downtime incident that caused problems?

It was a support account for a customer who had an existing non-optimized integration solution that had a huge memory footprint for a daily batch, and when the solution reached new peak loads, the solution crashed the servers, and caused downtime, it went unnoticed for some time, until it affected other far more important applications that were being consumed by external parties on the same servers. Only when the external parties started complaining, that the customer noticed the issue at hand. The critical applications deployed on the same servers were impacted as well, which disrupted the operations of several external service consumers. It caused delays in the impacted business process, and all the previous and current daily batches had to be entered by employees into the target DB.

Mainly, end-users were impacted due to delays, and errors in the data, not sure about an actual cost figure. A monitoring solution would have detected this performance issue much earlier, it would have raised alerts, and at least the problematic application would have been taken offline sooner, which could have provided the team with more time to find an alternative and could have spared the other deployed critical applications from being impacted for as long as it happened.

Do you have a customer that extracts business intelligence / analytics from their IT systems in general, and integrations environment specifically?

Yes, some customers will require crucial business reports to be on top of their business processes and make sure everything is going smoothly from a business side. Some will use custom DB with a structured format for analytical tools, as this provides them with the ultimate flexibility in terms of data insights and tools. Others are satisfied by what BAM has to offer, in BizTalk projects.

How do you see Azure and the current development towards the cloud? BizTalk vs. Logic Apps?

I always say that Logic Apps itself alone can’t be compared with BizTalk Server, BizTalk server is a full-fledged publish-subscribe platform with wide array of toolset and built-in features.

For the time being, I think BizTalk will still have its good share in developing integration solutions especially that today, a lot of the organization’s resources are still on-premises, some of which could never be lifted to the cloud due to policies and regulations.

How do you see the future of integration and how should monitoring solutions adapt to this?

When the customers’ IT landscape changes and contains merely SAAS applications, cloud-based integration solutions will be the obvious option to go for, especially with introduction of citizen integrators concept which empowers business users with powerful toolsets that can easily model and build business processes.

The IT world is getting more complex, and IT managers need to cope: How do you see the demands on IT systems getting more complex, and how are you coping with this?

Organizations are driven by competitors, and this in turn drives IT managers to move faster, by leveraging microservices architecture, robust cloud computing platforms, to decrease the time to market, and possibly the cost as well.

Cloud services, and new paradigms are being introduced in an extremely fast pace, keeping up with all that is indeed a challenge and can be overwhelming. You need to pick a technology specialization that you are most excited about, and you also should be aware of the other complementary services and how they are positioned.

I personally rely on reading books, it is my primary source of information, I like to get the information first-hand from MVPs and Microsoft product team.

Download the free community survey report now

Want to hear what Ahmed and 15 other Microsoft integration pros have to say about the future of integrations, key challenges and the critical role of monitoring in what's coming next? Download the free report – Monitoring the Future of Microsoft Integrations: A Community Survey today.

Monitoring the Future of Microsoft Integrations

Topics from this blog: Blog


BizTalk and Azure integration consultant, technical writer and #aimsperformancepro.

Eva De Jong

BizTalk and Azure integration consultant, technical writer and #aimsperformancepro.

More from the Author

May 31, 2019 1:19:12 PM
Tips to Monitor Azure SQL Databases
Mar 14, 2019 6:00:50 PM
How to monitor Logic Apps in Azure
Feb 8, 2019 12:08:00 PM
AIMS Azure Agent cost monitoring

Share this Post

Subscribe to our newsletter



A Beginner's Guide To AIOps

AIOps is the core of digital operations. It acquires data from different sources including existing IT monitoring tools and automates processes to the point where users are given only the truly...

Automated Azure Monitoring with Artificial Intelligence

Traditional monitoring fails in a modern IT environment when systems and IT applications become more complex, more agile, partly due to interconnect connectivity between applications and the external...

IT operations monitoring

IT operations monitoring tools will help you better understand and control all your IT data and information. AIMS automated monitoring solution is powered by AI, which gives you even greater...