A Governed Data Mesh Needs (Open) Metadata

Data meshes are the data infrastructure of the future. A Data Mesh addresses many data management challenges that come from an ever more distributed landscape that many companies find themselves in. It allows for the movement of data to be automated across physical locations, for domains to set up their own data pipelines, and, when done right, managing data policies in a distributed way. One of the interesting challenges that comes with introducing a Data Mesh is understanding and governing data in a distributed set up. Central data governance is hard enough, but can be solved with the adoption of a single technology stack. But in a distributed world, we inevitably end up with a heterogeneous technology landscape, where different tools have their own data catalogs. To govern and trace data across this heterogeneous landscape, next to a data mesh, we also need a way to exchange meta data between the different data catalogs. Here a standard for metadata is required. ING is working with different technology partners like IBM, MS and SAS on ODPi Egeria, which allows for metadata exchange across different stacks. Data Meshes and Open Metadata need to go hand in hand to create a true governed data mesh.