## Summary Related documentation and projects: - https://marbleclimate.com/ - https://pavics.ouranos.ca/citation.html - https://climatedata.ca/about/ Publication: https://www.researchgate.net/publication/359624740 ## Description We present here the recent progress of the DACCS (Data Analytics for Canadian Climate Services) project [1], funded by the Canadian Foundation for Innovation (FCI) and various provincial partners in Canada, in response to a national cyberinfrastructure challenge. The project’s development phase is set to finish in 2023, while funding for maintenance is planned over ten years. DACCS leverages previous development efforts of PAVICS [7], a virtual laboratory facilitating the analysis of climate data, without the need to download it. A key aspect of DACCS is to develop platforms and applications that combine both climate science and Earth observation, in order to stimulate the creation of architectures and cyber infrastructures shared by both. In climate sciences, it is now common to manipulate very large and highly dimensional volumes of data derived from climate models such as the CIMIP5 and soon CIMIP6 datasets. In Earth Observation (EO), the concepts of time series and ARD data also lead us to manipulate multidimensional data cubes. Consequently, the two domains share a common set of requirements, especially in terms of visualization services and deployment of applications close to the data. The overall philosophy of the platform is to provide a robust backend with interoperable services with the ability to deploy processing applications close to the data (A2D). The proposed platform architecture adopts some of the characteristics of the EO Exploitation Platform Common Architecture (EOEPCA) [12], including user access, data discovery, user workspaces, process execution, data ingestion and interactive development. This architecture describes A2D use cases where a user can select, deploy and run applications on distant infrastructures, usually where the data resides. An Execution Management Service (EMS) transmits such a request to an Application Deployment and Execution Service (ADES) [8]. Python notebooks are offered as the main user interaction tool. This way, advanced users can keep the agility of Python objects, all while offering the option to call demanding A2D tasks via widgets. Beyond the technical contribution, the platform offers an opportunity to explore synergies between the two scientific domains, for example to create hybrid machine learning models [6]. The platform ingests Sentinel-1 & 2 and Radarsat Constellation Mission (RCM) images in order to create multi-temporal and multi-sensor datacubes. The EO ingestion is performed by Sentinel Application Platform (SNAP) [13] workflows packaged as application inside Docker containers [11]. The goal is to provide researchers the ability to create data cubes on the fly via python scripts, with an emphasis on replicability and traceability. Resulting data cubes will be stored in NetCDF format along with the appropriate metadata to be able to document data provenance and guarantee replicability of the processing. Workflows and derived products will be indexed in a STAC catalog. The platform is aiming at facilitating the deployment of typical processing tasks including standard ingest processing (w.g. calibration, mosaicking, etc.), but also advanced Machine Learning tasks. Aside from EO use cases, DACCS project should advance PAVICS climate analytics capabilities. The platform readily allows access to several data collections ranging from observations, climate projections and reanalyses. PAVICS relies on the Birdhouse Framework [9], a key component of the Copernicus Climate Data Store [10]. PAVICS is the backbone of the analytical capabilities of ClimateData.ca, a portal created to support and enable Canada’s climate change adaptation planning, and improve access to relevant climate data. Climatedata.ca provides access to a number of climate variables and indices, either computed from CMIP5 and CMIP6 projections downscaled to Canada [3], or from historical gridded datasets [4]. Variables can be visualized on maps or on graphs, either on the basis of their grid cell element or by spatial aggregates, such as administrative regions, watersheds or health regions. Climatedata.ca also provides up-to-date meteorological station data, standardized precipitation evapotranspiration index (SPEI) [5], as well as sea-level rise projections. The portal’s Analyze page [14] allows users to select their grid cells or regions, and to set the thresholds, climate models, RCPs and percentiles they would like to use for analysis. User queries are then sent to the PAVICS node through OGC API - Processes. Computations are realized by xclim[2], a library of functions to compute climate indices from observations or model simulations. The library is built using xarray and benefits from the parallelization handling provided by dask. ## References 1. Landry, T. (2018). "Bridging Climate and Earth Observation in AI-Enabled Scientific Workflows on Next Generation Federated Cyberinfrastructures", AI4EO session, the ESA Earth Observation Φ-week, November 2018, ESA-ESRIN, Frascati, Italy. 2. Logan, T. et al. (2021): xclim - a library to compute climate indices from observations or model simulations. Online. https://xclim.readthedocs.io/en/stable/ 3. Cannon, A.J., S.R. Sobie, and T.Q. Murdock (2015): “Bias Correction of GCM Precipitation by Quantile Mapping: How Well Do Methods Preserve Changes in Quantiles and Extremes?”, Journal of Climate, 28(17), 6938-6959, doi:10.1175/JCLI-D-14-00754.1. 4. McKenney, Daniel W., et al. (2011): "Customized spatial climate models for North America." Bulletin of the American Meteorological Society, 92.12: 1611-1622. 5. Tam BY, Szeto K, Bonsal B, Flato G, Cannon AJ, Rong R (2018): “CMIP5 drought projections in Canada based on the Standardised Precipitation Evapotranspiration Index”, Canadian Water Resources Journal 44: 90-107. 6. Requena-Mesa, Christian, et al.(2020) "EarthNet2021: A novel large-scale dataset and challenge for forecasting localized climate impacts." arXiv preprint arXiv:2012.06246, https://www.climatechange.ai/papers/neurips2020/48 7. The PAVICS platform. Online. https://pavics.ouranos.ca/ 8. Landry et al. (2020): “OGC Earth Observation Applications Pilot: CRIM Engineering Report”, OGC 20-045, http://www.opengis.net/doc/PER/EOAppsPilot-CRIM 9. C. Ehbrecht, T. Landry, N. Hempelmann, D. Huard, and S. Kindermann (2018): “Projects Based in the Web Processing Service Framework Birdhouse”, Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., XLII-4/W8, 43–47 10. Kershaw, Philip, et al. (2019): "Delivering resilient access to global climate projections data for the Copernicus Climate Data Store using a distributed data infrastructure and hybrid cloud model." Geophysical Research Abstracts. Vol. 21. 11. Gonçalves et al. (2021): “OGC Best Practice for Earth Observation Application Package”, OGC OGC 20-089, Candidate Technical Committee Vote Draft, Unpublished 12. Beavis P., Conway, R. (2020), A Common Architecture Supporting Interoperability in EO Exploitation, ESA EO Phi-Week 2020, Oct. 1st 2020. https://eoepca.org/articles/esa-phi-week-2020/ 13. Zuhlke et al. (2015), “SNAP (Sentinel Application Platform) and the ESA Sentinel 3 Toolbox”, Sentinel-3 for Science Workshop, Proceedings of a workshop held 2-5 June, 2015 in Venice, Italy. 14. ClimateData.ca Analyze page: https://climatedata.ca/analyze/