Data and Decision Making
Published 11-November-2020

How data continues to stifle agility & productivity ... and what you can do about it on a shoe-string.

In autumn last year McKinsey reported(1) that the successful use of data analytics was linked to the success of high performing organisation and was responsible for a widening gap between companies lacking the capability.

high-performing organizations are three times more likely than others to say their data and analytics initiatives have contributedat least 20 percent to earnings before interest and taxes (EBIT) over the past three years.

That was before the COVID19.

The COVID19 Grenade

COVID19 has ‘rolled a grenade into the room’ brining about the fastest and most dramatic enforced change since World War II.

Whilst the focus is currently on the immediate human tragedy, looking ahead the deepening economic impact will increasingly dominate the headlines.

To thrive and survive businesses will need to improve their agility, productivity and resilience.

Constraints on Agility

Agility is constrained by existing process, people and information(data). Lack of agility lengthens the time to identify and react to both problems and opportunities. Increasing agility means that not only must processes and behaviours (mind sets) be improved, but data used in decision making must me up-to-date, accessible, trusted and adaptable.

Headaches in the Wild West

Most organisations rely on enterprise IT systems to support a multitude of workflows and to gain critical operational and financial insights. The ways these systems collect, store and share data creates constraints. This is where the problems begin.

Each of these IT systems is a ‘data silo’. Each data silo is a fragment of a wider data landscape. Even after lengthy procurement cycles with good stakeholder engagement there are functional trade-offs. From day one, some of these systems just don’t work well enough for some populations. These groups often resort to, or continue using, Microsoft Excel® spreadsheets. This spreadsheet data landscape is essential to an organisation striving to operate at peak performance with the lowest risk.  At the same time this landscape is largely unregulated, unverified, uncontrolled and creates its own risks and challenges. It’s the ‘wild west’ in the world of data.

Much if not all of this Excel landscape can be considered ‘Dark Data’. Gartner(2) defines Dark Data as “the information assets organizations collect, process and store during regular business activities, but generally fail to use for other purposes”.  We can extend this term to include data that is known to be needed but is not yet collected at all.

The enterprise applications forming the remainder of the data landscape also present a tough challenge, both to those needing access to the data and those wanting to have an overall picture of the organisation. Data access is usually restricted due to license costs or training requirements. The most widespread workaround is to extract information and then manually copy& paste into Excel or export into data analysis tools like Microsoft PowerBI®.

However, this is slow due to the manual labour required. It’s error prone and presents difficulties in aligning data from multiple silos. The resulting ‘picture’ is usually incomplete, vulnerable to bias, is costly to maintain and somewhat unregulated (very difficult or impossible to audit).

The end result is limited data availability, lack of data provenance, low trust of the ‘understanding’ the data delivers and decision-making that is weeks or months behind actual events.

Into the Light

Assuming zero budget and a need to make improvements, what can be done?

Firstly, map the data landscape and discover what it REALLY looks like. I’d advocate a ‘spreadsheet amnesty’ to make it safe for people to share what they are doing. A good way to start the mapping is via small, focussed workshops, build ‘swim lane’ diagrams as stakeholders, flows and data silos are identified. The aim is to find out what data is used, where it comes from, how it’s being used, who is using it and where reporting is done. The output of this activity should provide:

  1. Identification of all the apps and data silos(including the spreadsheets)
  2. A map of pathways from data capture to data usage
  3. The points where data is manually manipulated (focal points for wasted time and increased risk)
  4. Identification of stakeholders and how they produce & interact with data

Along the way you will shine light on the Dark Data, find valuable creative thinking, uncover critical insights and unearth hidden gems. The next step is to ask the stakeholders what they need now and in the future. Ask them to share the problems and issues they face and the opportunities they could take advantage of if improvements were made. You’ll find most will be only to will to share!

Armed with this new understanding you have the core information needed to begin to improve the data landscape. Rather than the next step being the creation of an overarching strategy, it is often best to start with some tactical changes focused on doable improvements that can generate some quick wins.

Making small improvements can unlock significant, often unpredicted, value. Working in this way you’ll be able to make targeted, significant improvements faster and cheaper than most IT deployments. This approach not only generates value it attracts interest, builds momentum and creates a hunger for more. Who doesn’t want to see their work made easier with a few quick wins?!

From the outset, think how you can make this approach an easily repeatable process. Change is a guaranteed constant, so you, and your organisation need to work smart to keep pace. When you run out of quick wins, or the interest in more fundamental change arises, that is the time when a new or revised strategy is required. The development or revision of strategy will be faster, easier and will yield better results because you’ve already done much of the groundwork and learned more about your business. In short, you will be starting from a better-informed position. So, there is some light at the end of the tunnel, despite the best efforts of COVID19 to make it longer. The chance to emerge into a new world of opportunity may depend on the choices you make in the coming months about your data landscape. If you want some help or advice on that first step or any of the other issues raised in this article please feel to reach out to me here.

- Sean Blencowe

Connected Intelligence - Connecting you to Smarter

(1)  https://www.mckinsey.com/business-functions/mckinsey-analytics/our-insights/catch-them-if-you-can-how-leaders-in-data-and-analytics-have-pulled-ahead
(2)  https://www.gartner.com/en/information-technology/glossary/dark-data
(3)  https://www.splunk.com/en_us/blog/leadership/dark-data-has-huge-potential-but-not-if-we-keep-ignoring-it.html

BACK TO BLOGS