In this instalment, I speak with #aimsperformancepro Ahmed Taha – Technical Architect at Link Development. Ahmed has over 10 years of hands-on experience with the Microsoft integration stack, including .NET, BizTalk Server, SQL and Azure, and is one of a select few proud owners of an exclusive Inglorious Integrator t-shirt!
What is your function and in what manner are you confronted with monitoring issues?
I provide consultation, support, and design for on-premises and cloud-based integration solutions in Microsoft stack.
In my support role for existing customers integration solutions, I rely on all the available tracking and monitoring tools. The lack of proper monitoring tools, or unconfigured built-in tracking feature makes it very difficult to debug encountered issues, and I may need to have to inspect the source code, or the intermediate messages in a wiretap fashion.
What do you often encounter in terms of monitoring / what do you notice when working with customers?
I noticed that monitoring is done either inadequately, or entirely overlooked in customers integration solutions. This makes the integration solution like a black box with no visibility into the technical or business aspects of the solution.
Customers usually realize this at a later stage, after the solution was deployed and started to have technical issues that are difficult to debug, or business users want to gain insights into their business processes going through that black box.
When I work with new customers during the design phase in our projects, I properly introduce monitoring as a crucial part of the integration solution, and they usually see the value right away.
What should every IT manager responsible for integration know within an organization (what do you often hear back from customers)?
I think IT managers should always be up to speed with all the integration services whether on-premises or on the cloud into their integration toolbox. This way they can assist their customers to achieve their needs using the right tools.
Customers are now more willing than ever before to learn more about cloud-based solutions, this is partly due to todays’ cloud technologies hype, the operational expenditure pricing model, and server-less cloud services concept.
With what purpose are monitoring tools often used (which customer requests are involved, is a goal always specific)?
Mainly it used for technical monitoring. Initially, a typical request from technical operators is to have centralized dashboards containing the health statuses of all the solution components in one place, in addition to alerting capabilities. Afterwards, they take their time to further explore it and customize it as needed.
In the cases where business users required business reports, these requests tend to be much more detailed, as they know exactly what they want to see in the business reports.
In what scenario is monitoring most often used, first-line monitoring (functional) vs. second-line monitoring (technical)?
In my experiences, it is mostly used for technical monitoring for each component, or system in the solution.
What is missing in the current monitoring solutions?
Having the capability to automatically, and correctly get to the root cause of encountered issues from all the ingested data & metrics across the different systems, without having an Administrators to make sense of all these metrics.
Can you share an example(s) of a downtime incident that caused problems?
It was a support account for a customer who had an existing non-optimized integration solution that had a huge memory footprint for a daily batch, and when the solution reached new peak loads, the solution crashed the servers, and caused downtime, it went unnoticed for some time, until it affected other far more important applications that were being consumed by external parties on the same servers. Only when the external parties started complaining, that the customer noticed the issue at hand. The critical applications deployed on the same servers were impacted as well, which disrupted the operations of several external service consumers. It caused delays in the impacted business process, and all the previous and current daily batches had to be entered by employees into the target DB.
Mainly, end-users were impacted due to delays, and errors in the data, not sure about an actual cost figure. A monitoring solution would have detected this performance issue much earlier, it would have raised alerts, and at least the problematic application would have been taken offline sooner, which could have provided the team with more time to find an alternative and could have spared the other deployed critical applications from being impacted for as long as it happened.
Do you have a customer that extracts business intelligence / analytics from their IT systems in general, and integrations environment specifically?
Yes, some customers will require crucial business reports to be on top of their business processes and make sure everything is going smoothly from a business side. Some will use custom DB with a structured format for analytical tools, as this provides them with the ultimate flexibility in terms of data insights and tools. Others are satisfied by what BAM has to offer, in BizTalk projects.
How do you see Azure and the current development towards the cloud? BizTalk vs. Logic Apps?
I always say that Logic Apps itself alone can’t be compared with BizTalk Server, BizTalk server is a full-fledged publish-subscribe platform with wide array of toolset and built-in features.
For the time being, I think BizTalk will still have its good share in developing integration solutions especially that today, a lot of the organization’s resources are still on-premises, some of which could never be lifted to the cloud due to policies and regulations.
How do you see the future of integration and how should monitoring solutions adapt to this?
When the customers’ IT landscape changes and contains merely SAAS applications, cloud-based integration solutions will be the obvious option to go for, especially with introduction of citizen integrators concept which empowers business users with powerful toolsets that can easily model and build business processes.
The IT world is getting more complex, and IT managers need to cope: How do you see the demands on IT systems getting more complex, and how are you coping with this?
Organizations are driven by competitors, and this in turn drives IT managers to move faster, by leveraging microservices architecture, robust cloud computing platforms, to decrease the time to market, and possibly the cost as well.
Cloud services, and new paradigms are being introduced in an extremely fast pace, keeping up with all that is indeed a challenge and can be overwhelming. You need to pick a technology specialization that you are most excited about, and you also should be aware of the other complementary services and how they are positioned.
I personally rely on reading books, it is my primary source of information, I like to get the information first-hand from MVPs and Microsoft product team.
Download the free community survey report now
Want to hear what Ahmed and 15 other Microsoft integration pros have to say about the future of integrations, key challenges and the critical role of monitoring in what's coming next? Download the free report – Monitoring the Future of Microsoft Integrations: A Community Survey today.
Topics from this blog: Blog
