In each webinar, it will be presented the state-of-the-art in methods and tools for setting-up and maintaining HPC hardware and software infrastructures. The duration of each talk will be around 30-40 minutes, followed by a 10–15-minute moderated discussion with the audience.

There are already 4 webinars scheduled:

• Developing complex workflows that include HPC, Artificial Intelligence and Data Analytics | February 22

• A roadmap to quantum computing integration into HPC infrastructures | March 15

• Improving energy-efficiency of High-Performance Computing clusters | April 26

• Addressing the challenges of scientific visualization in the exascale age | May 31

The post RISC2 webinar series aims to benefit HPC research and industry in Europe and Latin America first appeared on RISC2 Project.

Date: May 31, 2023 | 4 p.m. (UTC+1)

The post Webinar: Addressing the challenges of scientific visualization in the exascale age first appeared on RISC2 Project.

However, to realize the full potential of this synergy, ML models (or models for short) must be built, combined and ensembled, which can be very complex as there can be many models to select from. Furthermore, they should be shared and reused, in particular, in different execution environments such as HPC or Spark clusters.

To address this problem, we proposed Gypscie [Porto 2022, Zorrilla 2022], a new framework that supports the entire ML lifecycle and enables model reuse and import from other frameworks. The approach behind Gypscie is to combine several rich capabilities for model and data management, and model execution, which are typically provided by different tools, in a unique framework. Overall, Gypscie provides: a platform for supporting the complete model life-cycle, from model building to deployment, monitoring and policies enforcement; an environment for casual users to find ready-to-use models that best fit a particular prediction problem, an environment to optimize ML task scheduling and execution; an easy way for developers to benchmark their models against other competitive models and improve them; a central point of access to assess models’ compliance to policies and ethics and obtain and curate observational and predictive data; provenance information and model explainability. Finally, Gypscie interfaces with multiple execution environments to run ML tasks, e.g., an HPC system such as the Santos Dumont supercomputer at LNCC or a Spark cluster. 

Gypscie comes with SAVIME [Silva 2020], a multidimensional array in-memory database system for importing, storing and querying model (tensor) data. The SAVIME open-source system has been developed to support analytical queries over scientific data. Its offers an extremely efficient ingestion procedure, which practically eliminates the waiting time to analyze incoming data. It also supports dense and sparse arrays and non-integer dimension indexing. It offers a functional query language processed by a query optimiser that generates efficient query execution plans.

References

[Porto 2022] Fabio Porto, Patrick Valduriez: Data and Machine Learning Model Management with Gypscie. CARLA 2022 – Workshop on HPC and Data Sciences meet Scientific Computing, SCALAC, Sep 2022, Porto Alegre, Brazil. pp.1-2. 

[Zorrilla 2022] Rocío Zorrilla, Eduardo Ogasawara, Patrick Valduriez, Fabio Porto: A Data-Driven Model Selection Approach to Spatio-Temporal Prediction. SBBD 2022 – Brazilian Symposium on Databases, SBBD, Sep 2022, Buzios, Brazil. pp.1-12. 

[Silva 2020] A.C. Silva, H. Lourenço, D. Ramos, F. Porto, P. Valduriez. Savime: An Array DBMS for Simulation Analysis and Prediction. Journal of Information Data Management 11(3), 2020. 

By LNCC and Inria 

The post Managing Data and Machine Learning Models in HPC Applications first appeared on RISC2 Project.

Date: September 22, 2022 | 4 p.m. (UTC+1)

Speaker’s presentation is available here.

The post Webinar: Interactive High-Performance Computing with JupyterLab first appeared on RISC2 Project.

The posters “Developing Efficient Scientific Gateways for Bioinformatics in Supercomputing Environments Supported by Artificial Intelligence” and “Scalable Numerical Method for Biphasic Flows in Heterogeneous Porous Media in High-Performance Computational Environments” are part of the activities of the LNCC RISC2 projects.

According to Carla Osthoff (LNCC) , former poster presents a collaboration project that aims to develop green and intelligent scientific gateways for bioinformatics supported by high-performance computing environments (HPC) and specialized technologies such as scientific workflows, data mining, machine learning, and deep learning.  The efficient analysis and interpretation of Big Data open new challenges to explore molecular biology, genetics, biomedical, and healthcare to improve personalized diagnostics and therapeutics; then, it becomes necessary to availability of new avenues to deal with this massive amount of information. New paradigms in Bioinformatics and Computational Biology drive the storing, managing, and accessing of data. HPC and Big Data advances in this domain represent a vast new field of opportunities for bioinformatics researchers and a significant challenge. The Bioinfo-Portal science gateway is a multiuser Brazilian infrastructure for bioinformatics applications, benefiting from the HPC infrastructure. We present several challenges for efficiently executing applications and discussing the findings on how to improve the use of computational resources. We performed several large-scale bioinformatics experiments that are considered computationally intensive and time-consuming. We are currently coupling artificial intelligence to generate models to analyze computational and bioinformatics metadata to understand how automatic learning can predict computational resources’ efficient use. The computational executions are carried out at Santos Dumont Supercomputer. This is a multi-disciplinary project requiring expertise from several knowledge areas from four research institutes (LNCC, UFRGS, INRIA Bordeaux, and CENAT in Costa Rica). Finally, Brazilian funding agencies (CNPQ, CAPES) and the RISC-2 project from the European Economic and Social Committee (EESC) support the project.

Latter poster presents a project that aims to develop a scalable numerical approach for biphasic flows in heterogeneous porous media in high-performance computing environments based on the high-performance numerical methodology. In this system, an elliptical subsystem determines the velocity field, and a non-linear hyperbolic equation represents the transport of the flowing phases (saturation equation). The model applies a locally conservative finite element method for the mixing speed. Furthermore, the model employs a high-order non-oscillatory finite volume method, based on central schemes, for the non-linear hyperbolic equation that governs phase saturation. Specifically, the project aims to build scalable codes for a high-performance environment. Identified the bottlenecks in the code, the project is now working in four different research areas. Parallel I/O routines and high-performance visualization to decrease the I/O transfers bottleneck, Parallel programming to reduce code bottlenecks for multicore and manycore architectures. and Adaptive MPI  to decrease the message communication bottleneck. The poster presents the first performance evaluation results used to guide the project research areas. This endeavor is a multi-disciplinary project requiring expertise from several knowledge areas from four research institutes (LNCC, UFRGS, UFLA in Brazil, and CENAT in Costa Rica). Finally, Brazilian funding agencies (CNPQ, CAPES) and the RISC-2 project.

The post National Laboratory for Scientific Computing participated in the ISC2021 first appeared on RISC2 Project.