In the next few weeks, we are going to do some interviews with Integration Specialists and IT managers to get to the bottom of what is important in Integration Monitoring. I have invited some of the best developers and managers to join me in my research. This first interview I did is the one most special to me, as Sandro Pereira has been one of my role models since I started working in integration.
For this interview, I met with Sandro in London while I was visiting Integrate 2018. We sat together right after his session on BizTalk lessons from the road, and I got to ask him the following questions.
What is your role and in what manner are you confronted with monitoring issues?
I am a Senior BizTalk Developer, working as a consultant at DevScope. I advise on architecture, development, design – a little of everything. I basically provide anything a customer needs, as best as I can.
The monitoring issues I encounter are different, and I believe there is not one tool that can do the trick. Using several tools together could give more insight in an environment.
At what point did you decide to start looking at monitoring software?
I have no need to monitor myself, but I am currently advising some clients. I started looking at monitoring on their behalf, but Portugal is different from for example the Netherlands or England with using monitoring tooling due to high cost of modern tooling; they are still mostly using PowerShell and SCOM. Logging is very important when working with different companies, message flows etc. I advise on different tools depending on customer needs.
In what ways are you using your monitoring solution at this point?
There are currently two clients that are still in orientation stage, which are currently using PowerShell. I think it is still better to use PowerShell than using nothing at all to monitor the solutions. Because this tool is public and free, but it takes a lot of work getting it right and you will still miss a lot due to static thresholds.
There is an article about monitoring using PowerShell written by Sandro here, clearly stating these are to be used in absence of a monitoring tool.
What would you like to be added to a monitoring solution?
There is not one party that has it all, but I would like to see more concern for GDPR. Tracking and auditing is being controlled but I would, for example, like to know that some fields can be hidden in messages, encrypted or deleted before transferring it to systems that don’t need those field. If not that, I would at least be able to warn users and readers of sensitive data in the file that they received. When these files contain something private we must register that the file was in someone’s care.
What would you say is the key value of having a monitoring solution within your organization?
It’s important, critical even, to have a monitoring solution. I am in favour of giving no one access to BizTalk Servers and SQL Servers. Not only for safety, but also for server usage as people will forget to log of and let their machines consuming memory and CPU. This causes unnecessary performance issues. Microsoft tents to give someone full admin right, or nothing at all, and it is not save.
Developers should log on to a BizTalk server for installation or configuration purpose only.
Can you share an example(s) of a downtime incident that caused problems?
The best (or worst, depending how you look at it) example I have is a total of 8 hours downtime for a company I worked for. There was no monitoring was in place, and when huge massaging volume was coming in BizTalk started throttling, backups failed, and at the end everything collapsed because there was no more disk space available. Finding out the reason was simple, but to recover everything took us about 4 to 8 hours with a total of 8 people, while no one else in the company could do their job.
Do you extract business intelligence / analytics from your IT systems in general, and integrations environment specifically?
We currently use BAM to extract some data, and I am always trying to advise customers to use BAM or PowerBI for KPI improvement, business needs and monitoring KPIs.
In what scenario is monitoring most often used, first-line monitoring (functional) vs. second-line monitoring (technical)?
Not one of my customers have monitoring in place. As I said, companies in Portugal do not use a lot of monitoring tools because of pricing. They don’t see the need for it – until they encounter a disaster.
How do you see Azure and the current development towards the cloud? Is your company going to use these new technologies?
The companies I work for are going to use stuff like PowerBI, CRM, BizTalk. I am trying to bring some Azure to customers, like LogicApps and API management, but most of the time we work in an on-prem environment using BizTalk Server. Azure opens door to new challenges as well as problems.
How do you see the future of integration and how should monitoring solutions adapt to this?
In the present, but also in the future as well as the past, we are in the middle of change. It is not easy being an Integration Specialist; we’ll never have an easy life with new challenges and technologies constantly coming our way. We need to be more clever in producing better solutions for our customers. The perfect scenarios like those brought to us in demos and presentations do not apply to real time scenarios, due to cost and other issues. We’ll first have to look to our clients and their needs, and with them in mind try to build a good solution. Monitoring solutions should be concentrating on BizTalk Server as well as cloud, for clients desire a single place for accessing monitoring info in one single tool. Hybrid is the future and in my opinion monitoring tools need to be hybrid also, making data accessible from one central place. Don’t monitor everything but do gather enough information.
How do you see the demands on IT systems getting more complex, and how are you coping with this?
It is complex and not complex at same time. Azure helps you out with building services and taking servers out of our hands. We need to keep an eye out for new development. It is hard to advise because there are not a lot of experts in the field, and what looks good on paper does not necessarily work good in real time. For API management, logic apps, different BizTalk features, there is too much on the market and hard to keep up and know about everything. We need field experts, test new software constantly and adapt tested software weekly to meet our requirements.
Download the free community survey report now
Want to hear what Sandro and 15 other Microsoft integration pros have to say about the future of integrations, key challenges and the critical role of monitoring in what's coming next? Download the free report – Monitoring the Future of Microsoft Integrations: A Community Survey today.
Topics from this blog: Blog