Given the nature of the project, RISC2 could not fail to be represented through its partners and with a strong presence in the event’s program. Carlos J. Barrios, researcher from the Universidad Industrial de Santander and RISC2 partner was responsible for opening CARLA, with his address setting the tone for the conference, emphasizing the importance of collaborative efforts and knowledge sharing in furthering the frontiers of HPC.

Fabrizio Gagliardi, the coordinator of RISC2, also took center stage with a special talk that introduced the audience to the mission and objectives of the RISC2 project. The presentation shed light on the pivotal role that RISC2 plays in advancing HPC research and development of the cooperation between the two continents in this field. Gagliardi participated in the EuroHPCLatam panel: Policy and Global Actions, which included representatives from Red Clara, CAF and the Ministry of CyT Colombia. This panel explored the policies and global actions required to propel HPC forward in Latin America, emphasizing collaboration between key stakeholders.

Another highlight of CARLA 2023 was the tribute to Mateo Valero, one of the promoters of RISC2. Valero’s dedication and contributions to the field were celebrated through an award with his name and one he was the first recepient, underscoring the lasting impact of his work on the entire HPC community.

This event was particularly important as it coincided with the end of the RISC2 project and the presentation of its results. Over the course of three years, the initiative has strengthened contacts and promoted the exchange of knowledge between researchers from Latin America and Europe through the organization of nine webinars, the support of several schools, workshops, and other training events in the field for young students and researchers. During this period, RISC2 partners also participated in several conferences and ceremonies with policymakers to raise awareness of the importance of continuing to support and prioritize this area of research in the future.

F6blmDGWoAAP0O6

20230920 082437

20230920 082527

20230920 180527

20230920 104714

20230920 103331

The post CARLA 2023: RISC2 Results Presented at Largest HPC Conference in Latin America first appeared on RISC2 Project.

As the coordinator of the High Performance Computing Center (Nacad) at COPPE/UFRJ, Alvaro Coutinho, presented advances in AI in Engineering and the importance of multidisciplinary research networks to address current issues in Scientific Machine Learning. Alvaro took the opportunity to highlight the need for Brazil to invest in high performance computing capacity.

The country’s sovereignty needs autonomy in producing ML advances, which depends on HPC support at the Universities and Research Centers. Brazil has nine machines in the Top 500 list of the most powerful computer systems in the world, but almost all at Petrobras company, and Universities need much more. ML is well-known to require HPC, when combined to scientific computer simulations it becomes essential.

The conventional notion of ML involves training an algorithm to automatically discover patterns, signals, or structures that may be hidden in huge databases and whose exact nature is unknown and therefore cannot be explicitly programmed. This field may face two major drawbacks: the need for a significant volume of (labelled) expensive to acquire data and limitations for extrapolating (making predictions beyond scenarios contained in the trained data difficult).

Considering that an algorithm’s predictive ability is a learning skill, current challenges must be addressed to improve the analytical and predictive capacity of Scientific ML algorithms, for example, to maximize its impact in applications of renewable energy. References [1-5] illustrate recent advances in Scientific Machine Learning in different areas of engineering and computer science.

References:

[*] https://www.coppe.ufrj.br/pt-br/planeta-coppe-noticias/noticias/coppe-e-sociedade-especialistas-debatem-os-reflexos-da-inteligencia

[1] Baker, Nathan, Steven L. Brunton, J. Nathan Kutz, Krithika Manohar, Aleksandr Y. Aravkin, Kristi Morgansen, Jennifer Klemisch, Nicholas Goebel, James Buttrick, Jeffrey Poskin, Agnes Blom-Schieber, Thomas Hogan, Darren McDonaldAlexander, Frank, Bremer, Timo, Hagberg, Aric, Kevrekidis, Yannis, Najm, Habib, Parashar, Manish, Patra, Abani, Sethian, James, Wild, Stefan, Willcox, Karen, and Lee, Steven. Workshop Report on Basic Research Needs for Scientific Machine Learning: Core Technologies for Artificial Intelligence. United States: N. p., 2019. Web. doi:10.2172/1478744.

[2] Brunton, Steven L., Bernd R. Noack, and Petros Koumoutsakos. “Machine learning for fluid mechanics.” Annual Review of Fluid Mechanics 52 (2020): 477-508.

The post Scientific Machine Learning and HPC first appeared on RISC2 Project.

The goal of the investigation is to provide users with simplified access to computing structures through scientific solutions that represent significant developments in their fields. In the case of this project, it is intended to develop intelligent green scientific solutions for BioinfoPortal (a multiuser Brazilian infrastructure)supported by High-Performance Computing environments.

Technologically, it includes areas such as scientific workflows, data mining, machine learning, and deep learning. The outlook, in case of success, is the analysis and interpretation of Big Data allowing new paths in molecular biology, genetics, biomedicine, and health— so it becomes necessary tools capable of digesting the amount of information, efficiently, which can come.

The team performed several large-scale bioinformatics experiments that are considered to be computationally intensive. Currently, artificial intelligence is being used to generate models to analyze computational and bioinformatics metadata to understand how automatic learning can predict computational resources efficiently. The workshop was held from April 10th to 11th, and took place in the University of Sao Paulo.

RISC2 Project, which aims to explore the HPC impact in the economies of Latin America and Europe, relies on the interaction between researchers and policymakers in both regions. It also includes 16 academic partners such as the University of Buenos Aires, National Laboratory for High Performance Computing of Chile, Julich Supercomputing Centre, Barcelona Supercomputing Center (the leader of the consortium), among others.

The post Latin American researchers present greener gateways for Big Data in INRIA Brazil Workshop first appeared on RISC2 Project.

The post Inria Brasil Workshops first appeared on RISC2 Project.

In the European and Latin American (EuroLatAm) HPC ecosystems, it tis possible to identify actors in different domains: industry, academy, research, society, and government. Each of them, at different levels, has a group of demands or interactions, depending on the interests. I.e., the industry demands capabilities to have HPC solutions for productivity and wants skills from the academy to perform development actors to build applications to use solutions. Another example could be the relationship between research and the government. In the HPC Ecosystem, collaborations allow synergies to face common interests. Still, it demands policies and coordinated roadmaps to support long-term projects and activities with a clear impact on society.

Of course, a historical relationship exists between Latin America and Europe from colonial history. In the case of advanced computing projects, it is possible to identify, from the first EuroLatAm Grid Computing projects more than twenty years ago until the real supercomputing projects such as RISC and RISC2. Still, now, more with shared interests and the different EuroLatAm HPC projects improve competitiveness and collaboration. Competitiveness for industrial and productive business, partnership (and competitiveness) in science and education goals, and human wellness. So paraphrasing Mateo Valero “who does not compute does not compete”, I would add “who does not collaborate does not survive”.

Taking collaboration and competitiveness, the RISC2 project allows identifying sustainability elements and sustainable workflows for different projects. The impressive interaction between the actors of the HPC EuroLatAm ecosystem has not only given scientific results but also policies, recommendations, best practices, and new questions. For these outcomes, in the past 2022 Supercomputing Conference, RISC2 was awarded the 2022 HPCWire Editors’ Choice Award as the Best HPC Collaboration.

Sustainable advanced computing ecosystems and their growth are evident with the knowledge of the results of projects such as RISC2. Collaboration, interaction, and competitiveness build human development and guarantee development, technological diversification, and peer-to-peer relationships to attack common interests and problems. So, RISC2 is a crucial step to advance to a RISC3 as it was at the time of the previous RISC.

By Universidad Industrial de Santander

The post Advanced Computing Collaboration to Growth Sustainable Ecosystems first appeared on RISC2 Project.

However, to realize the full potential of this synergy, ML models (or models for short) must be built, combined and ensembled, which can be very complex as there can be many models to select from. Furthermore, they should be shared and reused, in particular, in different execution environments such as HPC or Spark clusters.

To address this problem, we proposed Gypscie [Porto 2022, Zorrilla 2022], a new framework that supports the entire ML lifecycle and enables model reuse and import from other frameworks. The approach behind Gypscie is to combine several rich capabilities for model and data management, and model execution, which are typically provided by different tools, in a unique framework. Overall, Gypscie provides: a platform for supporting the complete model life-cycle, from model building to deployment, monitoring and policies enforcement; an environment for casual users to find ready-to-use models that best fit a particular prediction problem, an environment to optimize ML task scheduling and execution; an easy way for developers to benchmark their models against other competitive models and improve them; a central point of access to assess models’ compliance to policies and ethics and obtain and curate observational and predictive data; provenance information and model explainability. Finally, Gypscie interfaces with multiple execution environments to run ML tasks, e.g., an HPC system such as the Santos Dumont supercomputer at LNCC or a Spark cluster. 

Gypscie comes with SAVIME [Silva 2020], a multidimensional array in-memory database system for importing, storing and querying model (tensor) data. The SAVIME open-source system has been developed to support analytical queries over scientific data. Its offers an extremely efficient ingestion procedure, which practically eliminates the waiting time to analyze incoming data. It also supports dense and sparse arrays and non-integer dimension indexing. It offers a functional query language processed by a query optimiser that generates efficient query execution plans.

References

[Porto 2022] Fabio Porto, Patrick Valduriez: Data and Machine Learning Model Management with Gypscie. CARLA 2022 – Workshop on HPC and Data Sciences meet Scientific Computing, SCALAC, Sep 2022, Porto Alegre, Brazil. pp.1-2. 

[Zorrilla 2022] Rocío Zorrilla, Eduardo Ogasawara, Patrick Valduriez, Fabio Porto: A Data-Driven Model Selection Approach to Spatio-Temporal Prediction. SBBD 2022 – Brazilian Symposium on Databases, SBBD, Sep 2022, Buzios, Brazil. pp.1-12. 

[Silva 2020] A.C. Silva, H. Lourenço, D. Ramos, F. Porto, P. Valduriez. Savime: An Array DBMS for Simulation Analysis and Prediction. Journal of Information Data Management 11(3), 2020. 

By LNCC and Inria 

The post Managing Data and Machine Learning Models in HPC Applications first appeared on RISC2 Project.

RISC2 was responsible for the organization of the “HPC and Data Sciences meet Scientific Computing” workshop, on September 26, which gathered 15 participants. This workshop discussed different topics, such as Scientific Machine Learning, High Performance Scientific Computing, and Data Science. Álvaro Coutinho, Marta Mattoso (from COPPE/Federal University of Rio de Janeiro), Frédéric Valentin (from the National Laboratory for Scientific Computing), Luc Giraud, Stéphane Lanteri, and Patrick Valduriez (from Inria) were the organizers of the workshop.

RISC2 also organized a tutorial about physics-informed neural networks. Our partners from Brazil, Álvaro Coutinho and Romulo Montalvão, from the Federal University of Rio de Janeiro, António Tadeu Gomes and Frédéric Valentin, from the National Laboratory for Scientific Computing, were the instructors of the session.

Our partners Carlos Barrios, from the Universidad Industrial de Santander, was one of the General Chairs of the Conference. “With 130 participants from all over the world, CARLA 2022 was a space of “rediscover” (to rediscover us) after two years in virtual mode. More than the scientific tracks and the panels, CARLA 2022 allowed us to discuss the challenges and the strengthening of collaboration between the partners (old and new)”, says Carlos Barrios.

Various RISC2 members also gave different presentations. Alba Cervera-Lierta, from the Barcelona Supercomputing Center, was one of the Keynote Speakers of the CARLA Conference, with a presentation about Quantum Computing. Esteban Meneses, from CeNAT, participated in a presentation about “Implementing a GPU-Portable Field-Line Tracing Application with OpenMP Offload”. Pablo Mininni, from the University of Buenos Aires, was responsible for one of the invited talks about “Multi-level parallelisation of computational fluid dynamics codes using CUDA, MPI and OpenMP.”

CARLA is an international conference that provides a forum to foste the growth and strength of the HPC community in Latin America through the exchange and dissemination of new ideas, techniques, and research in HPC and its application areas.

Also during the conference, the RISC2 members had a networking meeting with the SCALAC members, reinforcing the partnership with the SCALAC network.

About CARLA 2022:

“CARLA 2022 was a space of “rediscover” (to rediscover us) after two years in virtual mode. More than the scientific tracks and the panels, CARLA 2022 allowed us to discuss the challenges and the strengthening of collaboration between the partners (old and new)”.

Carlos Barrios Hernandez,  Universidad Industrial de Santander

“Having the RISC2 project supporting a networking dinner in CARLA was crucial in building up the next research collaboration we want to have in the region. I am thoroughly satisfied with the experience of connecting with European and Latin American peers”.

Esteban Meneses, CeNAT

“Among the most important elements, I can highlight the quality and variety of paper presented. This indicates to me that the Latin American HPC community is growing and getting stronger. In addition, I was able to notice efforts to generate relations between Europe and Latin America through the RISC2 project”.

Elvis Rojas Ramírez, CeNAT

The post RISC2 highly represented at CARLA 2022 first appeared on RISC2 Project.

As the utility and usage of HPC infrastructures increases, more computational and storage power is required to efficiently handle the amount of targeted applications. In fact, many HPC centers are now aiming at exascale supercomputers supporting at least one exaFLOPs (1018 operations per second), which represents a thousandfold increase in processing power over the first petascale computer deployed in 2008 [RD+15]. Although this is a necessary requirement for handling the increasing number of HPC applications, there are several outstanding challenges that still need to be tackled so that this extra computational power can be fully leveraged. 

Management of large infrastructures and heterogeneous workloads: By adding more compute and storage nodes, one is also increasing the complexity of the overall HPC distributed infrastructure and making it harder to monitor and manage. This complexity is increased due to the need of supporting highly heterogeneous applications that translate into different workloads with specific data storage and processing needs [ECS+17]. For example, on the one hand, traditional scientific modeling and simulation tasks require large slices of computational time, are CPU-bound, and rely on iterative approaches (parametric/stochastic modeling). On the other hand, data-driven Big Data applications contemplate shorter computational tasks, that are I/O bound and, in some cases, have real-time response requirements (i.e., latency-oriented). Also, many of the applications leverage AI and machine learning tools that require specific hardware (e.g., GPUs) in order to be efficient.

Support for general-purpose analytics: The increased heterogeneity also demands that HPC infrastructures are now able to support general-purpose AI and BigData applications that were not designed explicitly to run on specialised HPC hardware [KWG+13]. Therefore, developers are not required to significantly change their applications so that they can execute efficiently at HPC clusters.

Avoiding the storage bottleneck: By only increasing the computational power and improving the management of HPC infrastructures it may still not be possible to fully harmed the capabilities of these infrastructures. In fact, Big Data and AI applications are data-driven and require efficient data storage and retrieval from HPC clusters. With an increasing number of applications and heterogeneous workloads, the storage systems supporting HPC may easily become a bottleneck [YDI+16, ECS+17]. Indeed, as pointed out by several studies, the storage access time is one of the major bottlenecks limiting the efficiency of current and next-generation HPC infrastructures. 

In order to address these challenges, RISC2 partners are exploring: New monitoring and debugging tools that can aid in the analysis of complex AI and Big Data workloads in order to pinpoint potential performance and efficiency bottlenecks, while helping system administrators and developers on troubleshooting these [ENO+21].

Emerging virtualization technologies, such as containers, that enable users to efficiently deploy and execute traditional AI and BigData applications in an HPC environment, without requiring any changes to their source-code [FMP21].  

The Software-Defined Storage paradigm in order to improve the Quality-of-Service (QoS) for HPC’s storage services when supporting hundreds to thousands of data-intensive AI and Big Data applications [DLC+22, MTH+22].  

To sum up, these three research goals, and respective contributions, will enable the next generation of HPC infrastructures and services that can efficiently meet the demands of Big Data and AI workloads. 

References

[DLC+22] Dantas, M., Leitão, D., Cui, P., Macedo, R., Liu, X., Xu, W., Paulo, J., 2022. Accelerating Deep Learning Training Through Transparent Storage Tiering. IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing (CCGrid)  

[ECS+17] Joseph, E., Conway, S., Sorensen, B., Thorp, M., 2017. Trends in the Worldwide HPC Market (Hyperion Presentation). HPC User Forum at HLRS.  

[FMP21] Faria, A., Macedo, R., Paulo, J., 2021. Pods-as-Volumes: Effortlessly Integrating Storage Systems and Middleware into Kubernetes. Workshop on Container Technologies and Container Clouds (WoC’21). 

[KWG+13] Katal, A., Wazid, M. and Goudar, R.H., 2013. Big data: issues, challenges, tools and good practices. International conference on contemporary computing (IC3). 

[NCR+18] Netto, M.A., Calheiros, R.N., Rodrigues, E.R., Cunha, R.L. and Buyya, R., 2018. HPC cloud for scientific and business applications: Taxonomy, vision, and research challenges. ACM Computing Surveys (CSUR). 

[MTH+22] Macedo, R., Tanimura, Y., Haga, J., Chidambaram, V., Pereira, J., Paulo, J., 2022. PAIO: General, Portable I/O Optimizations With Minor Application Modifications. USENIX Conference on File and Storage Technologies (FAST). 

[OG+15] Osseyran, A. and Giles, M. eds., 2015. Industrial applications of high-performance computing: best global practices. 

[RD+15] Reed, D.A. and Dongarra, J., 2015. Exascale computing and big data. Communications of the ACM. 

[ENO+21] Esteves, T., Neves, F., Oliveira, R., Paulo, J., 2021. CaT: Content-aware Tracing and Analysis for Distributed Systems. ACM/IFIP Middleware conference (Middleware). 

[YDI+16] Yildiz, O., Dorier, M., Ibrahim, S., Ross, R. and Antoniu, G., 2016, May. On the root causes of cross-application I/O interference in HPC storage systems. IEEE International Parallel and Distributed Processing Symposium (IPDPS). 

By INESC TEC

The post HPC meets AI and Big Data first appeared on RISC2 Project.

The workshop aimed to exchange experiences, results, and best practices of collaboration initiatives between Europe and Latin America, in which HPC was essential, and to discuss how to work towards sustainability by reinforcing the bridges between the HPC communities in both regions. The workshop was organized by our partners Esteban Meneses from CeNAT, Fabrizio Gagliardi from BSC, Bernd Mohr from JSC, Carlos J. Barrios H. from UIS, and Rafael Mayo-Gacía from CIEMAT.

The workshop was opened with a keynote by Daniele Lezzi from BSC who reviewed the EU-LATAM collaboration on HPC. Six more presentations highlighted research work from Latin America and collaborative work between organizations on both continents. More information about the workshop including a detailed program can be found here.

The RISC2 project supported the IEEE Cluster Conference, a major international forum for presenting and sharing recent accomplishments and technological developments in the field of cluster computing, as well as the use of cluster systems for scientific and commercial applications by organizing a networking event at the end of the workshop day.

Our partner Esteban Meneses, from National High Technology Center in Costa Rica and one of the RISC2 partners, was one of the Publicity Co-Chairs of the IEEE Cluster 2022 Conference.

The post RISC2 organized a workshop co-located with IEEE Cluster 2022 first appeared on RISC2 Project.