Diving Deeper

Make Big Data Smart Data (Part 2)

Part 2: The Key to Becoming a Data-driven Enterprise

Your competitors are working vigorously to execute a data management strategy that will enable them to dig deeper and pull out insights that enhance profitability, build new products, minimize exposure to risk, and deepen customer relationships. Data management on its own is probably not enough. It’s what you can do with data once you have a handle on it. Today’s small banks and startup FinTechs are also on the move, not hamstrung by twenty years of infrastructure accumulation. They will grow quickly because they are innovative, and they are nimble.

Your ability to compete for market share in this environment depends on how well you take your bank from being a legacy organization to an information-based, data-driven enterprise. How do you do that?

That’s the question I left you with in my last blog post. With a better understanding of the data management landscape – what it looks like today, where it’s going tomorrow – you probably have a clearer idea of how important it is to leverage the insights data provide if you’re going to outpace your competition in the financial services domain. So what can you do from a practical perspective to make sure your organization implements a data management strategy that provides a competitive edge?

Here are four areas to consider.

Foundational Readiness

Just because an organization has decided to invest in their data and capture the advantages it offers doesn’t mean it’s equipped to do so. The organization has to be foundationally ready.

What does this mean? For starters, being foundationally ready is a base-level position for your organization. It means you know where your data lives . . . how it’s named, how it works, what elements of it you can trust. Some refer to this discovery process as “master metadata management,” a four-letter word for some in the industry or an exercise that sounds expensive. It doesn’t have to be either. It’s really a process that involves getting to know the data landscape of your organization. That may mean inventorying your data, working with producers to identify metadata, and negotiating with consumers to determine what constitutes important information or “good” for their use cases.

Foundational readiness also requires understanding lineage. Where is your data coming from? Where is it going? While this groundwork assessment is a documentation-type exercise, it’s extremely important because it will enable your current processes to run more smoothly and accurately. You and your organization could get enormous lift simply by figuring out where data lives and which elements represent the correct version of the truth. The clarity that comes with knowing where data started, where it has been, and how it was transformed instills confidence when you use it to drive action. You could also identify data landmines – failure points that could derail product development, misrepresent profitability, and undermine dependability. Fuzzy processes and black box data journeys weaken confidence, complicate audits, and threaten regulatory compliance. 

As part of this process, it’s vital to integrate upstream and downstream data sources and users. Looking downstream, your data team can help establish this integration by finding out who in the organization is using these data sources, how they are using them, and which variables are important. Looking upstream? There needs to be a clear message to those who produce these data sources: Don’t turn them off. This is especially important when you’re dealing with user-defined data that lives outside the production space. For instance, an analyst may create a table or chart for personal use but later delete the source not realizing that a few others have come to depend on it for their own critical processes. One takeaway from this sort of experience is that it’s usually a good idea to rely exclusively on well-controlled sources with clear policies in place to protect the integrity of those processes.

Mapping out the ecosystem is also vitally important if you expect to achieve foundational readiness. Rather than requiring an expensive yearlong consulting engagement that culminates with a report, the right data team can often address this need in a matter of a few weeks or a month. In doing so, they can determine where data live and begin to evaluate the maturity of the ecosystem and then spend time strengthening it. Are there red flags flying, such as varying levels of legacy infrastructure, some on the cloud, some on individual laptops? Is data well governed? Standardized and documented? Is quality defined and then monitored? Who owns the data? Is data management more of a manual process rather than automated?

So the key indicators for gauging your foundational readiness include knowing where your data is and what’s actually in it, being able to distinguish between reliable and unreliable data, and knowing who uses the data and how.

Expert Pragmatism

As mentioned in Part 1 of this blog post, lots of consultants are classical data people – they’re smart – but they lack the real-world experience that comes from having worked in your space. It’s important to recognize the distinction between consultants who have academic credentials and those who have on-the-ground experience, understand industry-specific applications, and recognize the outcomes you aim to achieve.

To that end, it’s vital to consider the distinction between data management and data engineering. Standard thinking when it comes to data management is that the best solution is to have one centralized data ecosystem on the same tech stack that works the same way no matter what. In a perfect world, that’s ideal. But ideal is elusive in the real world. Enterprise solutions require enterprise sponsorship, and that doesn’t exist in most places. If you’re not in a position where you can immediately develop and implement a centralized data ecosystem and platform, the best and most timely solution is for your data management team to do what it can with what it has.

Data engineering can do just that. Data engineers who have gained real-world experience by working in your space can provide clever and pragmatic solutions in the way they approach your problem. They can enable effectiveness and efficiency by providing a solution that accommodates your existing technology. If you hear someone say, “this is a textbook solution, and we need to start by helping you with a data lake,” that’s a red flag. If someone purports to know your needs and problems with data management before digging in, they might know the Utopian solution, but they might not know what will really work for you. In other words, anyone who suggests a solution without first understanding the problem isn’t demonstrating expert pragmatism. “What are the challenges you’re facing?” That should be the first question you hear. Yes, data engineers understand the data management space, but they need to get to know your world and your organization before proposing solutions.

Engineering-based Solution

Whether designing the most effective analyses, modeling, automation, and – increasingly – machine learning solution, data engineers focus on the precision required to make their work extensible, scalable, and flexible. You get the best results when people and machines work together and when solutions to data management challenges are engineering-based. After all, that’s what makes the new players in your space so fast and nimble.

Even if your organization is one of the most advanced players in the financial services space and the most sophisticated teams of data analysts available are designing engineering-based solutions, tribal knowledge is critically important. There are always a few key players who can recall, for example, that pre-2005 data lives in cold storage on a particular server. A couple others may remember when the organization transitioned from one system to another, causing numbers to split and explaining the digital data shelf.

While tribal knowledge is invaluable, having to rely on a few people for knowledge regarding the location and origin of data is not sustainable. Documenting, remediating and ultimately automating out that sort of need-to-know information opens a lot of doors. When data is stored, organized, and presented in a way that makes it accessible to those who need it, you democratize innovation not just to machine learning and AI teams but every analyst looking to work smarter. Modelers, for instance, should not spend hours hunting for data. It should be at their fingertips. And the best way to support machine learning and other automated solutions is to build the foundational data on which those solutions rely.

Robust metadata supports a knowledge base to help analysts use data more effectively

Another hallmark of an engineering-based solution is that needs are not addressed locally but built with unlimited scalability in mind. For example, right now there are probably dozens or hundreds of analysts running SQL scripts that check data and develop a dashboard in every corner of your organization. Why limit the efficiency of the rest of the organization by restricting the reach of these scripts? Build toward the future where these can be leveraged across teams and silos. Engineering thrives on scalability and modularity. And these drive speed and innovation.

Collaborative Approach

While data management is certainly an issue that affects the technology-based facets of a business operation, business leaders are focused on outcomes: generating profitability and growth, securing new customers, extending credit to drive net present value. Your data management solutions should drive these outcomes and do so in a way that is incremental, innovative, and collaborative. It is tempting to treat data management as a check-the-box exercise for compliance people or as a new toy or tech people. Avoid this trap. Data is there to fuel business insights. Having everyone pointed in the same direction is key to success.

Moving everyone toward that success requires a collaboration. That begins the very first time you meet with data engineers and they ask you what you’re trying to accomplish. Grow the business? Limit losses? Are you looking to derive insights from your data, which you can then funnel back to the rest of your enterprise in an effort to inform policy and strategy?

A collaborative approach also helps cultivate a team culture as people from inside and outside the organization work together to assess challenges and design solutions. It’s important to remember that when you build something new, you inherently displace something old, something that may not only have reusable parts but also may have been developed by people within the organization. Reusing what you have, when feasible, is both efficient and financially smart. Moreover, being solicitous toward other team members who developed older practices cultivates trust and encourages their buy-in for the project.

With every step, it’s imperative to build trust. You can do that while implementing an innovative data management strategy by maintaining and promoting transparency. Let everyone in the organization see what you’re doing and what you’re producing. Be agile about it in every sense of the word. Data quality reports, data dictionaries, scratch notes, documentation – you generate these and other sources of information during the course of your work. Without spending any more time or money, you can provide people throughout your organization with a clear picture of what you’re doing simply by keeping them apprised of data they might find useful but didn’t know existed.Financial services organizations have spent years, perhaps decades, and a substantial amount of money collecting data on products, customers, and transactions, but they have put only fractional amounts of that information to work. They have been slow to leverage the massive volume of data they have at their fingertips to accelerate growth, strengthen profitability, develop products, and build market share. Too often, they’ve allowed data-driven insights to remain buried in siloed architectures, stored on drives and laptops, and rest undisturbed in the work experiences of various employees. Cloud migration devolves into lift and shift with similar structures and processes replicated with new platforms underneath. But that is a missed opportunity to really build for the future. Given the current landscape in the world of data management, as well as the need to establish a competitive edge in a rapidly disrupted market, now is the time to make big data smart data.