Environmental research infrastructures building FAIR services

NORCE has participated in a large European project where 13 European environmental research infrastructures collaborated to make their datasets and services more compatible with the FAIR principles. The ENVRI-FAIR project ran from 2019 to 2023, and NORCE participated in a work package for infrastructures that deliver geoscience data.

How to help researchers find relevant data?

Although many datasets are available on the internet, it can be difficult for a researcher to identify relevant data and then use them in their research. Reasons for this could be, for example, that different data sources describe and categorize their data with different terms and use different formats to store the data. It can also be difficult to find out which permissions and license conditions are linked to the use of the data.

That's why the FAIR principles came about

As an answer to the need, a publication entitled FAIR Guiding Principles for scientific data management and stewardship was published in 2016.

FAIR is an abbreviation for Findability, Accessibility, Interoperability and Reusability.

The idea was that by following the FAIR principles, data providers can make it possible for researchers to find relevant data sets for their research, understand which licenses regulate the use of the data sets, gain access to data and reuse them. There are several initiatives to implement and spread knowledge about the FAIR principles, see e.g. https://force11.org/info/the-f... and https://www.go-fair.org/go-fai....

In ENVRI-FAIR, 13 environmental research infrastructures worked together to make their datasets and services more compatible with the FAIR principles. This would also prepare these infrastructures to connect to the European Open Science Cloud (EOSC, https://eosc-portal.eu/). One of the infrastructures was the European Plate Observing System (EPOS) which is a comprehensive e-infrastructure with a data portal where researchers can search for and download geoscientific data, especially related to plate tectonics. EPOS includes data from various disciplines, including seismic data, geodetic data and data from volcano observatories.

In ENVRI-FAIR, EPOS participated in the work package for geoscience infrastructures. Many European research institutions were involved, including the Department of Geosciences at the University of Bergen and NORCE. UiB and NORCE have previously been central to the development of the Norwegian node of the EPOS infrastructure.

NORCE's role in ENVRI-FAIR

NORCE's role in ENVRI-FAIR was linked to the Enlighten-web software. Enlighten-web is a web application for interactive visual analysis developed by NORCE. Namely, EPOS wants to make various external analysis tools available in its data portal, for example for visual analysis of geoscientific data.

NORCE's role in the ENVRI-FAIR project was therefore to look at how Enlighten can be made available from the data portal in EPOS in a way that follows the FAIR principles. In EPOS, such tools and resources are called Integrated Core Services - Distributed, or ICS-D for short.

Besides Enlighten, ICS-D in this project included two other components; SWIRRL is a framework developed by KNMI (The Royal Netherlands Meteorological Institute) for defining and executing workflows. In ENVRI-FAIR, this applies to workflows for preparing data and starting Jupyter notebooks and Enlighten. The last component is Jupyter notebooks, a tool for LiterateProgramming. Together, these services form a Virtual Research Environment (VRE) for data analysis and visualization.

Work done

A gap analysis was made of the ICS-D components with regard to the FAIR principles. The gap analysis resulted in three main activities being defined to make these components more compatible with the FAIR principles.

The first was to define metadata for the ICS-D components. This metadata should then be included in the EPOS portal, to give users of the portal the opportunity to discover and use ICS-D components that are relevant to their needs. NORCE defined metadata that describes Enlighten web and what it can be used for. The metadata was defined using the SHAPEness METADATA EDITOR which represents such data in a standardized way (Resource Description Framework), and EPOS-DCAT-AP metadata model and interoperability.

The next main activity was to facilitate the creation and storage of provenance data from ICS-D. Provenance data is important, for example, for users to know to what extent they can trust services and data. Specifically, NORCE has created templates that SWIRRL can use to register provenance data for the Enlighten web service. These templates use so-called Provenance Notation, PROV-N. We have also arranged for provenance data to be stored for Enlighten web visualizations.

The third and last main activity was the necessary extensions and adaptations of the ICS-D software. A workflow was defined in SWIRRL to prepare data from the portal for display in Enlighten web.

NORCE created a program for conversion from various data formats used in the EPOS portal to a format that Enlighten-web can read. This program is run as part of the aforementioned workflow in SWIRRL. One reason why we chose this solution is the consideration of the provenance. NORCE also had to make some minor adjustments to Enlighten web so that it can be launched from the portal using SWIRRL.

The SWIRRL framework is made available for testing the ICS-D components Jupyter Notebook and Enlighten-web. The work in this project has made these components more ready to be connected to EOSC through the use of standardized metadata and provenance data.

The results are illustrated in a use case called Geo-scientific data around Vesuvius volcano. This includes data from historical and modern earthquakes, satellite data, data on faults and geological maps. A screenshot from Enlighten web showing these data types is shown above.

EPOS is well advanced in implementing the FAIR principles.