ecosystem (un)certainties
From the start of the Copernicus Data Space Ecosystem project, we knew that it was going to be a challenging task. Setting up a new ecosystem, almost from scratch, requires quite some elements with first and foremost the element of trust. Trust in a longer-term perspective, trust in maturity of operations, trust in the quality of archives and trust in the ability to scale. Trust needs to be earned and does not happen overnight. This was certainly an element that was not there in the early days of the DIAS platforms and kept many communities in limbo of where to process their project in the abundance of platforms.
This trust needed to be built upon an ever-evolving environment which includes numerous challenging aspect such as a wide array of official and community standards, multiple processing baselines, non-stop technological advances in the field of Artificial Intelligence, geopolitical influences, scarcity of professionals with specialized knowledge, etc., all in a setting where many companies are just embarking Earth observation data and services as integral components to optimize their business processes and foster a commitment to align the sustainability goals.
Despite the inherent challenges, the Copernicus Data Space Ecosystem is not only an ambitious service but also an extraordinary opportunity to elevate European Earth observation platforms and increase its incredible value to support pressing needs as set out on a global, continental, national and regional scale towards a sustainable world.
state of play today
So, where is the platform today? The existing ESA distribution hubs have seized to exist since early November 2023, which resulted a in huge surge in the number of registrations and active usage. In terms of data, the platform already holds petabytes of instantly accessible to the whole archive of Earth observation data including:
- Sentinel-1 SLC, GRD, OCN data
- Sentinel-2 L1C and L2A
- Sentinel-3 OLCI, SLSTR, SYN, SRAL data
- Sentinel-5p L1B and L2 data
- A Selection of the Copernicus Services data
This is a game changer from the previous data hubs and European DIAS platforms, which all applied a rolling archive, imposing a restriction to the readily available data. In the near future additional datasets will be integrated in the offer, such as the Copernicus Contributing Missions and Federated datasets.
The Copernicus Data Space Ecosystem roadmap is regularly updated and gives you an overview of which data and services are already available or upcoming. On top of this massive amount of invaluable data, a wide variety of access methods is orchestrated to facilitate seamless interaction. These methods include curated catalogs (utilizing standards such as STAC, OData, and OpenSearch), user-friendly interfaces exemplified by the Copernicus Browser and Data Workspace, and streamlined access through Application Programming Interfaces (APIs). This comprehensive approach supports users to efficiently access and harness the wealth of available data tailored to their preferences and requirements.
Copernicus Data Space Ecosystem roadmap as published in December 2023.
Click here to discover the full service roadmap.
The Application Programming Interfaces (APIs) are extremely valuable. They provide a whole different way of interacting with the data. Unlike conventional data downloading methods, these APIs enable comprehensive data analysis seamlessly spanning from local to continental and even global scales. This approach enhances the versatility and efficiency of data utilization, allowing users to conduct analyses directly without the need for extensive downloads.
- openEO API: these APIs enable users to access, retrieve, and analyze geospatial data in a consistent and interoperable manner.
- Sentinel Hub APIs: provide a comprehensive set of tools for accessing and processing satellite imagery, particularly from the Sentinel satellite constellation
These APIs have their own way of interacting with the data and can be used by anyone who has limited programming knowledge. Each API is metered but is supplied with respectively sufficient credits or processing units for regular usage. Eligible users with more extensive requirements can submit an application to increase their quota. This request can be made via the service desk, specifically for Copernicus Service or EU granted projects, or users can also explore the possibility of obtaining a higher quota by submitting their ideas on creodias.eu, or by seeking sponsorship through ESA's Network of Resources, or purchasing credit packages from the commercial offerings within the Ecosystem (under construction). However, the free tier credit and processing unit packages are replenished on a monthly basis to ensure seamless and continuous usage for our valued users.
At VITO, we are mainly working on the openEO component with continuous development and improvement of the service in terms of maturity, interoperability with other openEO back-ends, the addition of new features, improve user experience and finally updating the last bits and pieces in user documentation. Our openEO team aims to finetune all different topics to be able to support the growth of a thriving community and ecosystem.
The openEO web editor is the main graphical interface and serves as a useful tool for monitoring running jobs, execute predefined analytics algorithms and quickly see the results.
A growing ecosystem
The whole vision of the data space ecosystem is to establish a system where components can be easily reused, thereby fostering the seamless creation of added value on top of the Copernicus Data Space Ecosystem. Through collaborative ecosystem governance involving the European Commission and ESA, anyone can join the Copernicus Data Space Ecosystem. This could be the integration of the identity provision component, billing component, processing added value services using streamline data access API’s, use of IaaS for service provision, federating data sets into the Ecosystem using an openEO back-end, they all count as valid forms of being part of the growing Ecosystem. This way companies can advertise their offering and focus on their unique capabilities instead of handling each component themselves. The whole governance part is being worked on and clear conditions will be further detailed in the coming months.
the future of terrascope and other collaborative ground segments
With all these capabilities of the centralized Copernicus Data Space Ecosystem, where does it leave the decentralized ‘connected’ EO platforms, such as the collaborative ground segments and other data hubs? In my humble opinion, they are essential to engage local and regional users and communities, and support them through a hands-on experience to truly showcase the potential of Earth observation data and services and guide them in the uptake. It’s like enlarging the web around a spider. The larger the web, the more opportunities can be caught, but only if they are connected to the broader environment. There is a plethora of reasons to build such a web in a federated approach:
- Increased resilience: The ecosystem gives higher resilience when specific nodes in the network experience disruptions or go down.
- Enhanced spatial and temporal coverage: Federated nodes contribute to an expanded spatial and temporal coverage by incorporating new, regional Earth Observation (EO) datasets into the network.
- Data sovereignty and privacy: Nodes can be specifically designed to uphold national data sovereignty and individual privacy. This includes the capability to process and store data locally, minimizing the necessity for international data transfers and ensuring compliance with local regulations. An important aspect in this day and age of geopolitical tensions.
- Innovation and scalability: The decentralized nature of the network fosters innovation by allowing multiple stakeholders to independently develop and deploy new technologies. Moreover, the ecosystem is highly scalable, as the addition of new nodes, such as sensors or satellites, does not necessitate a reconfiguration of the entire system.
In Belgium for example, we operate Terrascope, the Belgian collaborative ground segment to provide all Belgian stakeholders easy access to open-source satellite images and earth observation services, free of charge. I'm also very pleased to announce that Terrascope will provide open-source satellite data until at least 2028. On behalf of the Belgian Science Policy Office (BELSPO), State Secretary for Science Policy Thomas Dermine signed an agreement with VITO (Vlaamse Instelling voor Technologisch Onderzoek) to secure continuous access to satellite data and derived products from the Sentinel constellation, and related image processing services to continue supporting all Belgian users for at least another 5 more years.
The natural evolution of the Terrascope platform will be making optimal use of the central Copernicus Data Space Ecosystem infrastructure and federate as one of the decentralized nodes. The focus will lie even more on local user enablement and cooperation with federal scientific institutes as stakeholders to shape EO-enabled data to our societal needs, and the realization of the Green Transition as set out by the European Commission.
Furthermore, we will maintain the added value and national heritage data sets, link into national and regional data spaces, explore the state-of-the art technology such as AI foundation models and tailor use cases according to national priorities.
Terrascope and the openEO platform
Let me end this blog with an update on the future of the openEO platform. Terrascope is one of the leading federated back-ends of this state-of-the-art ESA platform. It was pushed forward by the bright minds in the consortium and ESA in many ways, from openEO API maturation to additional functions, from use cases to global awareness of this open source technology that is behind it. It also laid the groundwork for openEO federation, a concept that is now further adopted in the Copernicus Data Space Ecosystem, opening up to other back-ends of which Terrascope will be one. Needless to say that it makes a lot of sense that the openEO platform will continue to exist as one of the ecosystem members, integrating with the Copernicus Data Space Ecosystem. This way it can benefit from user federation, the future billing engine and high visibility towards the rapidly growing community. Most of all, this will bring a more consolidated approach to users instead of using different platforms and ways of working. More details about these exciting developments are being finalized at he moment and will be shared soon.
Looking forward to support you further, because it is clear that we are facing an exciting era in the world of Earth observation platforms. Discover more at Copernicus Data Space Ecosystem and Terrascope, and stay tuned.