Multi-cloud and DIAS secured platform for high-performance deployment of services

11 May 2020

Yasmine BOULFANI, CS GROUP (Space Business Unit) France
With my thanks to the CS GROUP team for their contribution to this article

 

EO DATA INTERPRETATION AND ACCESS CHALLENGES

Earth Observation (EO) satellite missions provide an unprecedented increase volume of data production leading to new capabilities to monitor & analyze the health of our planet. However, these massive volumes of data are complex to analyze and are not accessible. In fact:

  • EO Data access remains complicated due to heterogeneous and various data sources and data formats;
  • EO Data interpretation/semantization is a complex task requiring high level scientific skills;
  • Processing of this voluminous data requires high-performance computing centres;
  • EO data correlation with other business domain is difficult because EO data is not indexed by general purpose search engines.

To address these issues, SNAPEARTH offers:

  • a one-stop-shop solution;
  • a semantized EO database continuously updated;
  • a familiar entry point powered by QWANT search engine, to open up EO data to a wide audience;
  • and finally, the ability to process this EO data with the cloud. SNAPEARTH is proposing a Cloud agnostic solution, allowing a holistic access to any cloud provider including DIAS, the major cloud & European cloud providers.
     

SNAPEARTH PLATFORM USERS

The SNAPEARTH platform addresses 3 kinds of users:

  1. IT professionals: users wishing to build fully customized services, either created for themselves or made available to other users of the platform. These users are looking for computing power and simplified access to data.
  2. EO service creators (hobbyists): users who are not IT professionals but who have good ideas and want to implement them by combining services. They are looking for services and a way they can combine them.
  3. Students and Researchers: users from higher education and scientific research who are looking for simplified access to data.
     

SNAPEARTH PLATFORM OFFERS

The SNAPEARTH platform offers 3 main services (Figure 1), named: EarthSignature, EarthSearch and EarthSelf Service.

4 pilot projects are planned as first users of the SNAPEARTH platform to provide very advanced use cases linked to several EO vertical markets, such as climate (EarthClimate), press (EarthPress), agriculture (EarthAgriculture) and food security (EarthFood-Security).

Figure 1: SNAPEARTH service offers overview

SNAPEARTH aims to create a virtuous circle in terms of understanding the ecosystem in relation to EO data. For this purpose, two services for access to complex semantic data are offered:

  • EarthSignature service: a unique database of semantic satellite products continuously updated. EarthSignature is based on artificial intelligence (deep learning models) designed to extract biophysical land cover semantic information from satellite imagery. It automatically and accurately maps the images provided by satellites for indexation. The EarthSignature service is designed mainly for “IT professionals” and possibly “students and researchers”, users who are already familiar with EO lexicon.
  • EarthSearch service: all data produced by EarthSignature dataset will be automatically indexed and accessible with the EarthSearch Portal dedicated for natural language queries on images and which will also benefit from a strong integration in the QWANT search engine. Qwant user will be able to express questions in natural language on images e.g. “Wildfire in Corsica , Roads in Ivory Coast , Evolution of forest resources in France , Climate data record in Italy over year 2017, Atmospheric composition in Berlin over the year, Urban zones in Senegal…” and get either a text response or an imagery response. The EarthSearch service is intended mainly for “EO service creators” users, these users can perform searches relevant to their field of application and access results extracted from satellite data. EarthSearch service makes sure that their searches concerning their field of application are linked to the semantics of the satellite products.  This allows users who do not have knowledge of these semantics to discover them in relation to their business domain.

To valorize this data at large scale, SNAPEARTH proposes EarthSelf service. EarthSelf service is a set of services dedicated to data processing with several levels of integration (Figure 2 in the following section). The solution has been designed to ensure a high level of portability, flexibility, scalability and security. The platform is powered by SAFESCALE, an open source multicloud secured management platform battle proven on various successful European projects.
 

SNAPEARTH INTEGRATION SERVICES

Integration with the SNAPEARTH platform is intended to be accessible to users with various degrees of tech-savviness in the fields of infrastructure, resources provisioning, application deployment and data or storage management. To this end, SNAPEARTH proposes three built-in services offering progressive integration levels.

Figure 2: SNAPEARTH platform integration services

IaaS integration

 “IaaS” stands for Infrastructure as a Service. A first level of integration, designed for “IT professionals”. At this level of integration users access directly to the virtual infrastructure (such as virtual networks, virtual machines or storage partitioning) and can build fully customized services.
 

KaaS integration

“Kaas” stands for Kubernetes as a Service. A second level of integration, positioned on top of the IaaS layer, which includes all the functionalities offered by this layer but by providing an abstraction layer over virtualized computing resources and, a fortiori, over hardware resources. SNAPERATH users can then deploy and manage their applications using Kubernetes management tools they are the most familiar with by focusing on building, deploying and operating their applications without worrying about the underlying architecture.

The KaaS layer is also intended for “IT professionals” users, but optimizing Time To Market and Total Cost of Operation. The KaaS integration solution offers them the following services:

  • Orchestration of micro services: To accelerate the Time To Market by simplify service deployments
  • Observability and security enhancing:  tackling the Total Cost of Operation by offering ergonomic solution to monitor performance and security and to understand and troubleshot applications.
     
WaaS integration

“WaaS” stands for Workflow as a Service. The last level of integration, positioned on top of the KaaS and IaaS layers, includes all the functionalities offered by the layers underneath it in order to allow users to quickly create operational and secure services without requiring any knowledge of neither the underlying container orchestration mechanism nor its underlying infrastructure. This layer is intended for users, who are not IT professionals but who have good ideas and want to implement them by combining existing services to produce added value. This layer, allow them to focus on developing their individual processing tasks, using whichever technology they are the most familiar with; then to push them through Git repositories which will be managed through a workflow management layer.  Thus, users will be able to access all the services on top of this layer (such as access to the EarthSignature database and the other SNAPEARTH services). In addition, they will not have to worry about the large-scale processing, (e.g. to reach national, continental or even global land cover computing areas).

Overall, it has to be noted that, for any application, the SNAPEARTH platform takes into account the specific needs of every integration use case and allows to mix and match integration features spanning multiple integration solution levels.

0 Comments
Leave a comment