Data Mesh: An Architectural Deep Dive

Data Mesh is a paradigm shift in how we imagine and build big data management solutions, and most importantly is a shift in how we form our teams around these solutions and govern them. Data Mesh departs from half a century old assumptions on how we need to manage analytical data to be useful. It shifts analytical data architecture from monolithic - such as lake and warehouse - to distributed; data ownership from centralized teams to decentralized owners; from data as an asset to hoard to a product to delight its users; from manual and centralized governance to a federated and computational one; and shifts from data platforms as a utility layer to a set of experience planes. This talk introduces Data Mesh foundations at the high level and deep dives into a proposed logical architecture to bring the paradigm to life. It introduces the architecture of new concepts such as data products - the quantum of Data Mesh architecture - and its affordances, as well as the planes of the data platform in support of computational governance and distribution. If you are a critical thinker with an architectural eye and keen to be part of the next generation solutions to democratize and decentralize big data, this talk is for you.