Data mesh: Airport’s perspectives on modern day data sharing’s challenges

Oslo airport serves as a hub for many other Norwegian airports. Data exchange and analysis is crucial for efficient operation. Credit: Avinor

Today’s situation: increasing availability and demand for data

Avinor, a publicly-owned company that runs 45 airports in Norway and operates the Norwegian ATM services, has seen an increasing availability and demand for accessing data. Internet of Things (IoT) devices, smart phone apps, connected units in the traffic and increased digitalisation of manual routines are all driving the increase of data-availability exponentially. The data science and business intelligence department, being the ‘man in the middle’, is constantly lagging and becomes a bottleneck for insights and analytics. The demand for data, both by internal parties and external parties, is seemingly without limits. Many of the challenges are described in the 2020 ‘Data Mesh Principles and Logical Architecture’ written by Zhamak Dehghani.

Figure 1 shows examples of parties exchanging data with Avinor.

Fig 1. Avinor data exchange. Credit: Avinor.

In most business initiatives, like Avinor, data is emphasised as high value assets. Most of the processes for making the operation more cost-efficient are data-driven, including remote towers, autonomous vehicles for snow removal, common operation centres, remote screening of passengers and baggage, and making passenger prediction more precise. The introduction of artificial intelligence (AI) and machine learning promises great benefits across many areas but requires high volumes of data with expected quality of data to maximise the benefits.

Historical evolution of data sharing

Historically, the use and sharing of data has often led to data being mined directly into the productions systems. When the reporting load made production systems unstable, this was solved by mirroring the systems to replica databases. Weaknesses with methods used earlier, was often related to solutions being dependent on specific persons.

The need for reporting across single sources created the area of the data-warehouses, which separated the data from their systems and enabled a more data-centric approach. Additionally, with the data lakes, unstructured data could be managed in a more consistent way.

Avinor has decided to use the data mesh principles to make data available internally as well as externally.

How does Avinor understand and implement data mesh?

According to Wikipedia, a data mesh is defined as: ‘a sociotechnical approach to building a decentralised data architecture, by leveraging a domain-oriented, self-serve design (in a software development perspective).’ This definition by itself does not give much guidance on how to approach and implement a data mesh, so we must look elsewhere for some guidance.

Avinor has found it is beneficial to relate to the four principles of data mesh:

1. Domain ownership
2. Data as a product
3. Self-serve data platform
4. Federated computational governance.

These principles mean we must define data in a broader way than earlier, but also that responsibility is transferred to domain specific teams, which historically has less technical competence and interest. The domain specific people are more interested in business goals than technical aspects.

All the principles need novel approaches and considerations. For us, the principle of ‘Federated computational governance’ was the most difficult to explain and toaddress.

Zhamak Dehghani explains this principle partly, as:

“There is a need for these independent data products to interoperate and “a data mesh implementation requires a governance model that embraces decentralisation and domain self-sovereignty, interoperability through global standardisation, a dynamic topology and most importantly automated execution of decisions by the platform.”

Winter operation is demanding and require cooperation between different resources. Credit: Avinor.

How Avinor understands the challenges related to the “Federated computational governance”

‘Federated computational governance’ introduces the idea of interoperability through global standardisation. This is a challenge because a global standard simply does not exist. Many areas do not have standardisation models on a data product level. For airports, this is something that may be forwarded to the ACI ACRIS community, but Avinor is exchanging data elements that originates in industries other than the airport industry, so the idea of global standardisation is so far no more than an idea.

For example, when AVINOR catches data for prediction passenger flow in security control, the data comes from sources outside the aviation industry, such as from train, bus, and road information sources.

From a practical view we need to incorporate three main perspectives:

• Interoperability and standardisation within our company
• Interoperability and standardisation within the national transport sector
• Interoperability and standardisation with global perspective (such as SWIM…), international sector standardisation and recommended practices.

Avinor’s challenges with ‘domain ownership’

One of the challenges with ‘domain ownership’ is that the people in different domains have different degrees of data knowledge. Our hypothesis is that the user interface needs to be highly self-explaining and the IT-tech language must be as simple as possible to make the domain resources feel in control of the processes and making them as independent as possible of central IT resources. The domain resources may need some help, but we think the involvement of work from a central IT organisation is too large and that the central IT will become a bottleneck.

Avinor’s challenges with ‘data as a product’

Having ‘data as a product’ means both having the technical infrastructure to deal with data as a product, the user interface, and the understanding of how to do this. Technically, Avinor has chosen a platform called CKAN which is a data management system (DMS) to support features related to ‘data as a product.’

Avinor’s challenges with ‘self-serve data platform’

Having users catering themselves to data introduces a lot of challenges. One is that Avinor may not even know all the users of different data products; there are open and restricted datasets. For restricted data sets, the users need to authenticate themselves, and the data owners need to make rules for authorising access.

Oslo airport serves as a hub for many other Norwegian airports. Data exchange and analysis is crucial for efficient operations. Credit: Avinor

Data mesh, our experiences so far

Our findings have identified that:

• It takes time to build a data sharing culture and set up data teams
• Communication strategy is important for onboarding and executive commitment, and must share a common understanding
• Framework building can be costly and exhausting:
  • It’s important to ramp up early with a value-driven approach and
  • High value data objects must be identified early in the journey
• The tech platform must facilitate a high degree of self service and user orientation.

We also find that commitments at the C level, as well as business and tech stakeholders, are important key elements.

Remote operation centres, including Remote Towers are dependent on data sharing.. Credit: Avinor.

Summary

Our experience is that implementing and using data mesh is related to several challenges, involving domain resources requiring better documentation; more easily understandable interfaces and a collective understanding of terms and processes, which do create some more overhead, compared to the traditional method of having central resources responsible for data-availability and reporting. Some of the requirements are difficult for Avinor to influence, such as global standardisation.

But overall, we find that the benefits of data mesh are considerable larger than the downsides. The benefits include introducing a common technical platform and involving domain resources more than previously, which is reducing dependency on central resources.

At the end of the day, we think the data mesh approach is catering for more scalability and is a better way to support data usage and a higher degree of data quality.

Avinor is working on making plow trucks autonomous, which requires efficient sharing of different datasets. Credit: Avinor.