Data virtualization as a solution — Part 4: virtual data modeling

The moment a data point is copied, the link between the information and its source is severed — losing real-time freshness, the source's permission model, performance, and automatable retrieval. The trigger is often something trivial: a spreadsheet, an import into a dashboard, a move to a shared drive or data lake. Preserving that link not only avoids the drawbacks but opens up entirely new modeling strategies.

A virtual data space allows for entirely new approaches and modeling strategies.

A different approach to modeling

Where a warehouse must define and load the maximum level of detail up front to build "the one" model, a virtual model is just a description of how to handle data — it contains no data itself, so it can be optimized per access. The rule of thumb is "as early as possible, as little as necessary": filter to the selection first, then add detail, letting the source system's performance build each data slice. Because users always need an aggregate or a subset, almost any use case can be served this way.

A platform offering such a space needs connectivity to the sources, fast end-user modeling, push-down of filters and parameters to the source, no caching, Single Sign-On everywhere without technical users, and access to the layer where the full business logic lives.

New operators — and data blending without boundaries

Comparing very large datasets (think system migrations) is often impossible by copying, because all data must be available at once. In the virtual space we work only with the slice needed right now: compare at a high level first, then drill down only where differences appear. A dedicated comparison operator takes two tables, compares the chosen figures and attributes, and returns only the deviations to filter the next, more granular query.

When the data sets are too large, performing a complete comparison becomes impractical or even impossible.

Blending changes too: there is no longer one system or network boundary. ESG reporting can pull live from supplier systems alongside HR, purchasing, logistics, accounting and production; supply-chain tracking can read partner companies' systems directly; and the impact of an acquisition or divestiture can be modeled by virtually integrating the affected unit's data — a holistic and sustainable way to handle enterprise data.