Applications and systems exist at the edge, in the data center, in a bevy of clouds, and in user departments. IT is called upon to manage all of them.
But is this management doable, and how can IT approach it for the best results?
First, let’s look at why enterprises use distributed systems and applications. Distributed systems and apps can deliver better performance when they are placed in the specific areas and departments of the company they serve. In such cases, they can use dedicated networks, storage and processing. If the systems are cloud-based, they can be readily scaled upward or downward in both cost and use, and there are cloud providers that maintain them.
At the same time, these distributed assets become interrelated in areas of data, processing, and governance. Systems and applications must be centrally managed when they come together.
Following are three challenges distributed systems and applications present for IT, and some ways IT can best address them.
1. Security and updates
Challenge: The IT attack surface is expanding exponentially as enterprises augment centralized IT systems with edge and cloud-based systems and applications. This compounds in difficulty when edge and mobile devices come in the door with lax or nonexistent security settings, making them easy attack targets. The promulgation of edge networks, mobile devices and cloud-based systems also increases the need for IT to apply software security and patch updates in a timely and consistent way.
Best practice: It’s no longer enough to use a standard monitoring system to track network and user activities throughout the enterprise. Tools like identity access management (IAM) can track a user’s activities and permission clearances across internal IT assets, but they provide limited visibility of what might be going on in the cloud. Tools like cloud infrastructure entitlement management (CIEM) can microscopically track user activities and permissions in the cloud, but not on prem. Identity governance and administration (IGA) can bring together both IAM and CIEM under one software umbrella, but its focus is still on the user and what the user does. CIEM can’t track malware in an embedded software routine that activates, or any other anomaly that could arise as data is moved among systems. For this, observability software is needed.
Observability tools can track every detail of what happens within each transaction as it moves through systems, and mobile device management (MDM) software can track the whereabouts of mobile devices. Meanwhile, security update software can be automated to push out software updates to all common computing platforms. Finally, there is the need to know when any addition, deletion or modification occurs to a network. Zero-trust networks are the best way to detect these changes.
The takeaway for IT is that it’s time to evaluate these different security and monitoring tools and defenses, and to create an architectural framework that identifies which tools are needed, how they fit with each other, and how they can end-to-end manage a distributed system and application environment. Best of breed IT departments are doing this today.
2. Data consistency
Challenge: Data across the enterprise must be accurate and consistent if everyone is to use a single version of the truth. When accuracy and consistency measures fail, different department managers get disparate information, which generates dissonance and delays in corporate decision making. Most enterprises report issues with data accuracy, consistency, and synchronization, often brought on by disparate, distributed systems and applications.
Best practice: The good news is that most organizations have installed tools such as ETL (extract, transform, and load) that have normalized and unified data that flows from variegated sources into data repositories. This has resulted in higher quality data for enterprise users.
Interestingly, a persistent problem when it comes to managing distributed systems is actually an “old school” problem. It’s how to manage intra-day batch and nightly batch processing.
Let’s say a company is in the US but has remote manufacturing facilities in Brazil and Singapore. At some point, the finished goods, inventory, work in process, and cost information from all these systems must coalesce into a consolidated corporate “view” of the data. It is also understood that these various facilities operate in different time zones and on different schedules.
Typically, you would batch together major system transaction updates during a normal nightly batch process, but nighttime in say, Philadelphia, is daytime in Singapore. When and how do you schedule the batch processing?
It’s cost prohibitive and, in some cases, impossible to perform transaction updating of all data in real time, so central IT must decide how to update. Does it do periodic intra-day “data bursts” of transactions between distributed systems, and then night process the other batches of transactions that are in less dissimilar time zones? Which batch update processes will deliver the highest degree of timely and quality data to users?
Optimized and orchestrated intra-day and nightly batch processing updates are a
60-year-old problem for IT. One reason it’s an old problem is that revising and streamlining batch processing schedules is one of IT’s least favorite projects. Nevertheless, best-of-class IT departments are paying attention to how and when they do their batch processing. Their ultimate goal is putting out the most useful, timely, highest quality information to users across the enterprise.
3. Waste management
Challenge: With the growth of citizen IT, there are numerous applications, systems, servers, and cloud services that users have signed up for or installed, but that end up either seldom or never used. The same can be said for IT, given its history of shelfware and boneyards. In other cases, there are system and application overlaps or vendor contracts for unused services that self-renew, and that nobody pays attention to. This waste is exacerbated with distributed systems and applications that may not have a central point of control.
Best practice: More IT departments are using IT asset management software and zero-trust networks to identify and track usage of IT assets across the enterprise and in the cloud. This helps them identify unused, seldom used or replicated assets that should be sunset or removed, with a corresponding cost savings.
Vendor contract management is a more complicated issue, because it is possible that individual user departments have contracts for IT products and services that IT may not know about. In this case, the matter should be raised to upper management. One possible solution is to have IT or an internal contract management or audit group go out to various departments in the enterprise to collect and review contracts.
Inevitably, some contracts will be found missing. In these cases, the vendor should be contacted so a copy of the contract can be obtained. The cost savings goal is to eliminate all services and products that the company isn’t actively using.
Final Remarks
Managing IT in a highly distributed array of physical and virtual facilities is a significant challenge for IT, but there are tools and methods that are fit for the task.
In some cases, such as batch processing, even an old school approach to batch management can work. In other cases, the tools for managing security, user access and data consistency are already in-house. They just have to be orchestrated into an overall architecture that everyone in IT can understand and work toward.
Because the tools and methods for distributed system and application management so clearly fall within IT’s wheelhouse, it’s the job of the CIO to educate others about IT’s security, governance, data consistency, and management needs to reduce corporate risk.