Of all recent developments in data management, the data fabric may be the most compelling for a number of reasons. For example, it promises the ability to connect all of an organization’s data in a single unified environment, so that any team–regardless of department or location–can collaborate on data projects. Additionally, it promises to do so without requiring ETL, or even its easier twin of ELT (extract, load, transform). The data can be located, integrated with other data sources, and analyzed without ever leaving its original source location. Furthermore, it promises to make the process of locating the right data sources and combining them into an analysis-ready dataset intuitive and accessible to even non-technical users.
If data fabrics are so great, why doesn’t everyone have one? This lack of adoption is at least partially due to a perception that a data fabric must be custom-built for an organization. Proponents of this point of view argue that, while there are plenty of solutions out there touting the moniker ‘data fabric’, most of them don’t yet provide the components of a true comprehensive fabric.
Is this a fair view? No, for reasons we’ll discuss shortly. But it has big implications, because currently many organizations are either engaged in a multi-year, multi-million dollar project to build their own data fabric, or they’re simply ignoring its potential and sticking with the insufficient data architecture they’ve currently got in place.
First, let’s talk about what building a data fabric would take. Then, let’s talk about the minimum requirements an off-the-shelf enterprise data fabric solution would require for it to be worth buying vs. building.
Critical components for a data fabric
A data fabric relies heavily on augmented data management and automation at all levels of design. Here are a few of the core components for any system that qualifies as a data fabric:
It’s clear from only a surface-level exploration that creating a data fabric, even in its MVP (minimally viable product) form, would require extensive resources and expertise, an astronomical budget, and an indefinite amount of time. The task of creating a system that intelligently handles multiple types of metadata, alone, would take years and cost $millions.
Assuming you’re not IBM, Google, or Facebook, you probably don’t have access to these resources or the kind of budget required. If, however, you were to explore the option of an off-the-shelf data fabric, what would it need to make real improvements to your ability to manage data? At what point would it cross over from simply being a glorified data catalog to being a true fabric that expands and grows with your organization’s data?
Minimum requirements for an off-the-self data fabric
For a data fabric to be viable for an organization, it must provide certain features and benefits.
In sum, most organizations will find that attempting to build a data fabric is a quagmire. But, commercial solutions may fit the bill if they meet, at minimum, the above requirements. Furthermore, a solution that fits these requirements will provide the following advantages over building a custom platform:
provides numerous advantages over attempting to internally build a data fabric, including:
If you’re interested in trying out a commercial data fabric solution that exceeds these requirements, read on.
SHARE THIS:
