The Data Mesh concept is based on four main pillars, the first of which is an organisation divided into data domains. To be effective, this structure must reflect the business reality of the company and the interactions between its various entities. It therefore presupposes a high degree of proximity to businesses.
1- Data Mesh: the ultimate model for data-driven companies?
2- Data domains: Data Mesh gives business domains superpowers.
3- Data Mesh: data is a product
4- Data infrastructure self-service as the technological driving force behind Data Mesh
5- Data Mesh: federated governance to guarantee efficiency
Data domains: instructions for use
The use of data in business processes and decision-making is a real vector of change for creating value at all levels of the company. And in this respect, Data Mesh is an opportunity for business domains to drive their own transformation and improve all their decision-making through analytics.
Structuring data into domains that are as close as possible to the company’s organisation and processes has two advantages. Data Mesh helps to democratise data beyond circles of experts such as Data Scientists, Data Engineers, Data Architects, etc. This approach also seeks to ensure that data is fully appropriated by operational staff, as close as possible to their needs. However, this is precisely what organisations are looking to achieve in their quest for data-driven maturity.
With Data Mesh, responsibility for data, knowledge of it and its uses are fully in the hands of the businesses, which are the key players in optimising data. This is the principle of ‘Domain Ownership’, which is motivated by:
- The ability to evolve data sharing in line with organisational growth (number of data sources, number of consumers, etc.),
- Rapid and continuous data change cycles,
- The agility to reduce cross-team synchronisation and remove the bottlenecks of centralised governance,
- Improving data quality and resilience of analysis (BI, AI) by bringing producers closer to consumers (removing the complexity of intermediate data pipelines).
The data domain approach of Data Mesh is directly inspired by the Domain Driven Design (DDD) concept which is also applied to the design of software solutions.
What exactly does organisation into data domains mean?
The data domain approach of Data Mesh is directly inspired by the Domain Driven Design (DDD) concept, which is also applied to the design of software solutions. This axis of Data Mesh therefore consists of breaking down data (and its use) according to segmentation by domain.
This is a fundamental notion, at the heart of the Data Mesh concept. Great care must also be taken when dividing and distributing data domains. In some companies, this choice will be obvious and will be based on the main functions of the organisation chart (HR, operations, marketing, etc.). In other contexts, the breakdown may be by business line, by distribution channel or according to the specific needs of each entity. The domains of a single organisation may combine narrow scopes or broad, cross-cutting ones.
Whatever form it takes, special care must be taken with this breakdown. The granularity of the domains and their structure require an in-depth examination for each company to take into account their specific needs and the level of maturity of the various departments. In all cases, it is essential that the breakdown by data domain reflects the business reality of the company. It should never be the result of a purely technical approach or one that is too far removed from the reality on the ground.
The methodological approach is to map and describe the different types of data domain:
- Source-aligned data domains, which concentrate analytical data reflecting the business facts generated by operational systems and known as native data products.
- Aggregate data domains, which concentrate analytical data resulting from correlation and enrichment operations involving data originally from several source-aligned data domains (e.g. 360° vision construction).
- Consumer-aligned data domains, which concentrate analytical data transformed to meet the needs of one or more business use cases.
Once established, the domains drive the distribution and optimisation of data. They play a greater role in giving power (if not superpowers) back to the business domains, which can become fully autonomous in the use of data. To achieve this, domains are responsible for ALL data within their scope. This of course includes operational data (ERP, process tools, etc.), but also analytical data.
To this end, the domains conform to a set of roles and responsibilities:
- Data producers and managers of all data assets (analytical and operational).
- Co-pilots of data governance, through the application of common rules and standards defined at organisational level.
- Designers of data products according to common principles to enable exposure (e.g. via APIs) and consumption by the different domains.
- Participants in sharing data with other domains. The value generated is increased by scaling up, for example through the creation of a data exchange or data marketplace. This is a fundamental aspect of Data Mesh to which we will return in more detail.
Gartner estimates that poor data quality costs companies an average of $12.9 million per year.
Autonomous, agile domains
Data Mesh undoubtedly represents a paradigm shift, both in the distribution of skills and in responsibilities. With organisation by domain, data know-how is no longer centralised (within IT or in a data factory), but rather distributed.
There are many advantages of this approach:
- Overcome the bottleneck of a centralised team and develop data products that immediately address business priorities.
- Drastically improve data quality by bringing both the production and use of data closer to the business. These are also better able to document the data in a unified catalogue. The key to this is economies of scale. Gartner estimates that poor data quality costs companies an average of $12.9 million per year.
- Encourage cross-domain data exchange to break down silos and create company-wide value.
- Encourage a culture of internal collaboration and the development of skills in data.
Data Mesh and Data Domains: the right recipe to finally involve the business?
Moving from a world where data is driven centrally by the Data Office and IT, to a world where roles are distributed and shared with the domains, requires an in-depth review of the organisation and therefore very high-level sponsorship.
Data Mesh implies assigning expert data skills in the domains, as close as possible to the business, to create multidisciplinary teams compatible with an agile delivery method.
This coverage in terms of skills is also essential to deal with the various subjects related to the end-to-end management of data and data products. This will require a review of job descriptions to incorporate these new activities.
The deployment of organisation by data domains is carried out iteratively, starting as a minimum with a producer domain (aligned with the source) and a consumer domain (aligned with the consumer). This initiation is based on an enabling, motivated and mature business.
Less mature businesses can be supported and advised by the Data Office, which will provide methodology and resources. The data domains can then be rolled out with a gradual transfer of activity and responsibility. Decentralisation will then be accelerated as maturity progresses.
🔎Michelin & Data Mesh: from platform to data domains
Joris Nurit, chief data and analytics architect at Michelin, discovered the concept of Data Mesh in 2019. At that point, he was heading up the company’s data lake strategy. At the same time, Michelin had already been deploying a distributed architecture for five years. And this was not the only part of its IT to which the company had applied a distributed approach.
Michelin thus implemented one of the main principles of Data Mesh by setting the following objective: a single mesh to govern all data domains. The tyre manufacturer also looked at the other pillars. One of the CDO’s priorities was therefore to set up a global governance system covering and connecting all domains.
This federated governance is based on different roles such as the Data Owner, Data Steward and Data Administrator. The Data Owners of the domains are responsible for ensuring the proper management of the source and aggregate products.
Data domains that are independent and yet communicative
Companies very often come up against obstacles, in particular the difficulty of identifying relevant use cases that create value. This can often be explained by the low level of interest or acculturation of business entities with regard to data.
With Data Mesh, the businesses, or rather the data domains, more than ever become players in the data strategy implemented in the company. Moreover, this appropriation is carried out on a federated basis. Each domain thus participates in the management and optimisation of the global data assets at company level, thanks to standardised data sharing between the domains and common governance.
Data Mesh is about breaking with these practices. Through its global approach, it promotes the exchange of data between silos. Its solution: the creation of data domains that are both independent and interacting. The intrinsic value of the data products (another pillar of Data Mesh) exposed and shared by the source-aligned domains is intended to be leveraged through the uses that will be implemented by the consumer-aligned domains.
Things to remember
📌A necessary overhaul of the organisation thanks to a very high level of sponsorship
📌A data domain structure as close as possible to the business
📌Expert data skills required in the domains
📌A Data Officer facilitator and driving force behind Data Mesh
This article was written in collaboration with Christophe Auffray.