The ENERXICO Project focused on developing advanced simulation software solutions for oil & gas, wind energy and transportation powertrain industries.  The institutions that collaborated in the project are for México: ININ (Institution responsible for México), Centro de Investigación y de Estudios Avanzados del IPN (Cinvestav), Universidad Nacional Autónoma de México (UNAM IINGEN, FCUNAM), Universidad Autónoma Metropolitana-Azcapotzalco, Instituto Mexicano del Petróleo, Instituto Politécnico Nacional (IPN) and Pemex, and for the European Union: Centro de Supercómputo de Barcelona (Institution responsible for the EU), Technische Universitäts München, Alemania (TUM), Universidad de Grenoble Alpes, Francia (UGA), CIEMAT, España, Repsol, Iberdrola, Bull, Francia e Universidad Politécnica de Valencia, España.  

The Project contemplated four working packages (WP): 

WP1 Exascale Enabling: This was a cross-cutting work package that focused on assessing performance bottlenecks and improving the efficiency of the HPC codes proposed in vertical WP (UE Coordinator: BULL, MEX Coordinator: CINVESTAV-COMPUTACIÓN); 

WP2 Renewable energies:  This WP deployed new applications required to design, optimize and forecast the production of wind farms (UE Coordinator: IBR, MEX Coordinator: ININ); 

WP3 Oil and gas energies: This WP addressed the impact of HPC on the entire oil industry chain (UE Coordinator: REPSOL, MEX Coordinator: ININ); 

WP4 Biofuels for transport: This WP displayed advanced numerical simulations of biofuels under conditions similar to those of an engine (UE Coordinator: UPV-CMT, MEX Coordinator: UNAM); 

For WP1 the following codes were optimized for exascale computers: Alya, Bsit, DualSPHysics, ExaHyPE, Seossol, SEM46 and WRF.   

As an example, we present some of the results for the DualPHYysics code. We evaluated two architectures: The first set of hardware used were identical nodes, each equipped with 2 ”Intel Xeon Gold 6248 Processors”, clocking at 2.5 GHz with about 192 GB of system memory. Each node contained 4 Nvidia V100 Tesla GPUs with 32 GB of main memory each. The second set of hardware used were identical nodes, each equipped with 2 ”AMD Milan 7763 Processors”, clocking at 2.45 GHz with about 512 GB of system memory. Each node contained 4 Nvidia V100 Ampere GPUs with 40 GB of main memory each. The code was compiled and linked with CUDA 10.2 and OpenMPI 4. The application was executed using one GPU per MPI rank. 

In Figures 1 and 2 we show the scalability of the code for the strong and weak scaling tests that indicate that the scaling is very good. Motivated by these excellent results, we are in the process of performing in the LUMI supercomputer new SPH simulations with up to 26,834 million particles that will be run with up to 500 GPUs, which is 53.7 million particles per GPU. These simulations will be done initially for a Wave Energy Converter (WEC) Farm (see Figure 3), and later for turbulent models. 

Figure 1. Strong scaling test with a fix number of particles but increasing number of GPUs.

Figure 2. Weak scaling test with increasing number of particles and GPUs.

Figure 3. Wave Energy Converter (WEC) Farm (taken from https://corpowerocean.com/)

As part of WP3, ENERXICO developed a first version of a computer code called Black Hole (or BH code) for the numerical simulation of oil reservoirs, based on the numerical technique known as Smoothed Particle Hydrodynamics or SPH. This new code is an extension of the DualSPHysics code (https://dual.sphysics.org/) and is the first SPH based code that has been developed for the numerical simulation of oil reservoirs and has important benefits versus commercial codes based on other numerical techniques.  

The BH code is a large-scale massively parallel reservoir simulator capable of performing simulations with billions of “particles” or fluid elements that represent the system under study. It contains improved multi-physics modules that automatically combine the effects of interrelated physical and chemical phenomena to accurately simulate in-situ recovery processes. This has led to the development of a graphical user interface, considered as a multiple-platform application for code execution and visualization, and for carrying out simulations with data provided by industrial partners and performing comparisons with available commercial packages.  

Furthermore, a considerable effort is presently being made to simplify the process of setting up the input for reservoir simulations from exploration data by means of a workflow fully integrated in our industrial partners’ software environment.  A crucial part of the numerical simulations is the equation of state.  We have developed an equation of state based on crude oil data (the so-called PVT) in two forms, the first as a subroutine that is integrated into the code, and the second as an interpolation subroutine of properties’ tables that are generated from the equation of state subroutine.  

An oil reservoir is composed of a porous medium with a multiphase fluid made of oil, gas, rock and other solids. The aim of the code is to simulate fluid flow in a porous medium, as well as the behaviour of the system at different pressures and temperatures.  The tool should allow the reduction of uncertainties in the predictions that are carried out. For example, it may answer questions about the benefits of injecting a solvent, which could be CO2, nitrogen, combustion gases, methane, etc. into a reservoir, and the times of eruption of the gases in the production wells. With these estimates, it can take the necessary measures to mitigate their presence, calculate the expense, the pressure to be injected, the injection volumes and most importantly, where and for how long. The same happens with more complex processes such as those where fluids, air or steam are injected, which interact with the rock, oil, water and gas present in the reservoir. The simulator should be capable of monitoring and preparing measurement plans. 

In order to be able to perform a simulation of a reservoir oil field, an initial model needs to be created.  Using geophysical forward and inverse numerical techniques, the ENERXICO project evaluated novel, high-performance simulation packages for challenging seismic exploration cases that are characterized by extreme geometric complexity. Now, we are undergoing an exploration of high-order methods based upon fully unstructured tetrahedral meshes and also tree-structured Cartesian meshes with adaptive mesh refinement (AMR) for better spatial resolution. Using this methodology, our packages (and some commercial packages) together with seismic and geophysical data of naturally fractured reservoir oil fields, are able to create the geometry (see Figure 4), and exhibit basic properties of the oil reservoir field we want to study.  A number of numerical simulations are performed and from these oil fields exploitation scenarios are generated.

Figure 4. A detail of the initial model for a SPH simulation of a porous medium.

More information about the ENERXICO Project can be found in: https://enerxico-project.eu/

By: Jaime Klapp (ININ, México) and Isidoro Gitler (Cinvestav, México)

The post Subsequent Progress And Challenges Concerning The México-UE Project ENERXICO: Supercomputing And Energy For México first appeared on RISC2 Project.

Humans are particularly efficient at recognising objects – think about what surrounds us: one recognises the object where one is reading the text from as a screen, the place where one sits as a chair, the utensil in which one drinks coffee as a cup, and one does all of this extremely quickly and virtually automatically. One is able to do all this despite the fact that 1) one holds large amounts of information about each object (if one is asked to write down everything you know about a pen, you would certainly have a lot to say); and that 2) there are several exemplars of each object type (a glass can be tall, made out of glass, metal, paper or plastic, it can be different colours, etc. – but despite that, any of them would still be a glass). How does one do this? How one is able to store and process so much information in the process of recognising a glass, and generalise all the different instances of a glass to get the concept “glass”? The goal of the ContentMAP is to understand the processes that lead to successful object recognition.

The answer to these question lies in better understanding of the organisational principles of information in the brain. It is, in fact, the efficient organisation of conceptual information and object representations in the brain that allows one to quickly and efficiently recognise the keyboard that is in front of each of us. To study the neuronal organisation of object knowledge, the project collects large sets of fMRI data from several participants, and then try to decode the organisational principles of information in the brain.

Given the amount of data and the computational requirements of this type of data at the level of pre-processing and post processing, the use of HPC is essential to enable these studies to be conducted in a timely manner. For example, at the post-processing level, the project uses whole brain Support Vector Machine classification algorithms (searchlight procedures) that require hundreds of thousands of classifiers to be trained. Moreover, for each of these classifiers one needs to compute a sample distribution of the average, as well as test the various classifications of interest, and this has to be done per participant.

Because of this, the use of HPC facilities of of the Advanced Computing Laboratory (LCA) at University of Coimbra is crucial. It allows us to actually perform these analyses in one to two weeks – something that on our 14-core computers would take a few months, which in pratice would mean, most probably, that the analysis would not be done. 

By Faculty of Psychology and Educational Sciences, University of Coimbra

Reference 

ProAction Lab http://proactionlab.fpce.uc.pt/ 

The post Mapping human brain functions using HPC first appeared on RISC2 Project.

However, to realize the full potential of this synergy, ML models (or models for short) must be built, combined and ensembled, which can be very complex as there can be many models to select from. Furthermore, they should be shared and reused, in particular, in different execution environments such as HPC or Spark clusters.

To address this problem, we proposed Gypscie [Porto 2022, Zorrilla 2022], a new framework that supports the entire ML lifecycle and enables model reuse and import from other frameworks. The approach behind Gypscie is to combine several rich capabilities for model and data management, and model execution, which are typically provided by different tools, in a unique framework. Overall, Gypscie provides: a platform for supporting the complete model life-cycle, from model building to deployment, monitoring and policies enforcement; an environment for casual users to find ready-to-use models that best fit a particular prediction problem, an environment to optimize ML task scheduling and execution; an easy way for developers to benchmark their models against other competitive models and improve them; a central point of access to assess models’ compliance to policies and ethics and obtain and curate observational and predictive data; provenance information and model explainability. Finally, Gypscie interfaces with multiple execution environments to run ML tasks, e.g., an HPC system such as the Santos Dumont supercomputer at LNCC or a Spark cluster. 

Gypscie comes with SAVIME [Silva 2020], a multidimensional array in-memory database system for importing, storing and querying model (tensor) data. The SAVIME open-source system has been developed to support analytical queries over scientific data. Its offers an extremely efficient ingestion procedure, which practically eliminates the waiting time to analyze incoming data. It also supports dense and sparse arrays and non-integer dimension indexing. It offers a functional query language processed by a query optimiser that generates efficient query execution plans.

References

[Porto 2022] Fabio Porto, Patrick Valduriez: Data and Machine Learning Model Management with Gypscie. CARLA 2022 – Workshop on HPC and Data Sciences meet Scientific Computing, SCALAC, Sep 2022, Porto Alegre, Brazil. pp.1-2. 

[Zorrilla 2022] Rocío Zorrilla, Eduardo Ogasawara, Patrick Valduriez, Fabio Porto: A Data-Driven Model Selection Approach to Spatio-Temporal Prediction. SBBD 2022 – Brazilian Symposium on Databases, SBBD, Sep 2022, Buzios, Brazil. pp.1-12. 

[Silva 2020] A.C. Silva, H. Lourenço, D. Ramos, F. Porto, P. Valduriez. Savime: An Array DBMS for Simulation Analysis and Prediction. Journal of Information Data Management 11(3), 2020. 

By LNCC and Inria 

The post Managing Data and Machine Learning Models in HPC Applications first appeared on RISC2 Project.

In Life Sciences, HPC advancements can improve different areas:

• a reduced time to scientific discovery;
• the ability of generating predictions necessary for precision medicine;
• new healthcare and genomics-driven research approaches;
• the processing of huge datasets for deep and machine learning;
• the optimization of modeling, such as Computer Aided Drug Design (CADD);
• enhanched security and protection of healthcare data in HPC environments, in compliance with European GDPR regulations;
• management of massive amount of data for example for clinical trials, drug development and genomics data analytics.

The outbreak of COVID-19 has further accelerated this progress from different points of view. Some European projects aim at reusing known and active ingredients to prepare new drugs as contrast therapy against COVID disease [Exscalate4CoV, Ligate], while others focus on the management and monitoring of contagion clusters to provide an innovative approach to learn from SARS-CoV-2 crisis and derive recommendations for future waves and pandemics [Orchestra].

The ability to deal with massive amounts of data in HPC environments is also used to create databases with data from nucleic acids sequencing and use them to detect allelic variant frequencies, as in the NIG project [Nig], a collaboration with the Network for Italian Genomes. Another example of usage of this capability is the set-up of data sharing platform based on novel Federated Learning schemes, to advance research in personalised medicine in haematological diseases [Genomed4All].

Supercomputing is widely used in Drug Design (the process of finding medicines for disease for which there are no or insufficient treatments), with many projects active in this field just like RISC2.

Sometimes, when there is no previous knowledge of the biological target, just like what happened with COVID-19, discovering new drugs requires creating from scratch new molecules [Novartis]. This process involves billion dollar investments to produce and test thousands of molecules and it usually has a low success rate: only about 12% of potential drugs entering the clinical development are approved [Engitix]. The whole process from identifying a possible compound to the end of the clinical trial can take up to 10 years. Nowadays there is an uneven coverage of disease: most of the compounds are used for genetic conditions, while only a few antiviral and antibiotics have been found.

The search for candidate drugs occurs mainly through two different approaches: high-throughput screening and virtual screening. The first one is more reliable but also very expensive and time consuming: it is usually applied when dealing with well-known targets by mainly pharmaceutical companies. The second approach is a good compromise between cost and accuracy and is typically applied against relatively new targets, in academics laboratories, where it is also used to discover or understand better mechanisms of these targets. [Liu2016]

Candidate drugs are usually small molecules that bind to a specific protein or part of it, inhibiting the usual activity of the protein itself. For example, binding the correct ligand to a vial enzyme may stop viral infection. In the process of virtual screening million of compounds are screened against the target protein at different levels: the most basic one simply takes into account the shape to correctly fit into the protein, at higher level also other features are considered as specific interactions, protein flexibility, solubility, human tolerance, and so on. A “score” is assigned to each docked ligand: compounds with highest score are further studied. With massively parallel computers, we can rapidly filter extremely large molecule databases (e.g. billions of molecules).

The current computational power of HPC clusters allow us to analyze up to 3 million compounds per second [Exscalate]. Even though vaccines were developed remarkably quickly, effective drug treatments for people already suffering from covid-19 were very fresh at the beginning of the pandemic. At that time, supercomputers around the world were asked to help with drug design, a real-world example of the power of Urgent Computing. CINECA participates in Exscalate4cov [Exscalate4Cov], currently the most advanced center of competence for fighting the coronavirus, combining the most powerful supercomputing resources and Artificial Intelligence with experimental facilities and clinical validation. 

References

[Engitix] https://engitix.com/technology/

[Exscalate] https://www.exscalate.eu/en/projects.html

[Exscalate4CoV] https://www.exscalate4cov.eu/

[Genomed4All] https://genomed4all.eu/

[Ligate] https://www.ligateproject.eu/

[Liu2016] T. Liu, D. Lu, H. Zhang, M. Zheng, H. Yang, Ye. Xu, C. Luo, W. Zhu, K. Yu, and H. Jiang, “Applying high-performance computing in drug discovery and molecular simulation” Natl Sci Rev. 2016 Mar; 3(1): 49–63.

[Nig] http://www.nig.cineca.it/

[Novartis] https://www.novartis.com/stories/art-drug-design-technological-age

[Orchestra] https://orchestra-cohort.eu/

By CINECA

The post Using supercomputing for accelerating life science solutions first appeared on RISC2 Project.

The NLHPC staff  set their  fundamental pillar to focus  efforts on the scientific community and show HPC as an essential tool for country development by getting users from diverging scientific areas, industry and public sector. This entails breaking access barriers to this kind of technology. NLHPC faces this challenge by making training for the basic use of HPC  and scientific software optimization;  which is key in order to make a good use of resources.

The training was carried out within a framework of computational thinking, being the process by which an individual, through his professional experience and acquired knowledge, manages to face problems of different kinds. This could be evidenced in our active participation in the resolution of the proposed activities, which enhanced our abstraction and engineering thinking. We will certainly take this vision of education and collaborative work to our professional environment, in the different roles we play as HPC administrators, teachers and students.

The proper use of computing services involves efforts to perform monitoring, control and infrastructure management tasks. With the help of the tools reviewed during our visit, we will be able to provide our users with the highest standards of quality, security and accessibility.

The joint effort of the RISC2 and EU-CELAC ResInfra projects made it possible for engineers from Colombia, Mexico and Peru to participate in this HPC management course, learn about Chilean culture, gain knowledge and valuable contacts for our profession.

After living this great experience, we hope that in the near future other supercomputing centers replicate this type of initiatives in other parts of the world, thus increasing the communication bridges between HPC administrators from different places, sharing knowledge and experiences.

We are left with the milestone of being part of the First School of HPC Administrators of Latin America and the Caribbean, with experiences that made us grow in professional, academic, and human aspects. As well as with alliances among colleagues and now friends, a network of support as brothers of the same region.

We conclude by thanking Rafael Mayo of CIEMAT for the initiative; Ginés Guerrero, Pedro Schürmann, Eugenio Guerra, Pablo Flores, Angelo Guajardo, Esteban Osorio, José Morales for the knowledge and experiences shared; RISC2 and EU-CELAC ResInfra for providing us with this learning opportunity, supporting the scholarship grant.

By:

Miguel Angel Barrera Arbelaez, Universidad de los Andes, Colombia

Carlos Enrique Mosquera Trujillo, Centro de bioinformática y biología computacional de Colombia BIOS, Colombia

César Alexander Bernal Díaz, Universidad Industrial de Santander, Colombia.

Eduardo Romero Arzate, Universidad Autónoma Metropolitana, México.

Ronald Darwin Apaza Veliz, Universidad Nacional de San Agustín, Perú.

Joel Gonzalez Lara, Centro de Análisis de Datos y Supercómputo, México

The post First School of HPC Administrators in Latin America and the Caribbean: A space for the formation of computational thinking first appeared on RISC2 Project.

Several factors contributed to the catastrophic outcomes of the Black Death. The crises was boosted by the lack of two important components: knowledge and technology. There was no clue about the spread dynamics of the disease, and containment policies were desperately based on assumptions or beliefs. Some opted for self-isolation to get away from the “bad air” that was believed to be the cause of the illness [2]. Others thought the plague was a divine punishment and persecuted the heretics in order to “appease the heavens” [3]. Though the first of these two strategies was actually very effective, the second one only increased the tragedy of that scenario. 

The bubonic plague of the 14th century is a great example of how unfortunate ignorance can be in the context of epidemics. If the transmission mechanisms are not well-understood, we are not able to design productive measures against them. We may end up −such as our medieval predecessors− making things much more worse. Fortunately, the advances in science and technology have provided humanity with powerful tools to comprehend infectious diseases and rapidly develop response plans. In this particular matter, epidemic models and simulations have become crucial. 

In the recent COVID-19 events, many public health authorities relied on the outcomes of models, so as to determine the most probable paths of the epidemic and make informed decisions regarding sanitary measures [4]. Epidemic models have been around for a long time, and have become more and more sophisticated. One reason is the fact that they feed on data that has to be collected and processed, and which has increased in quantity and variety.  

Data contains interesting patterns that give hints about the influence of apparently non-epidemiological factors such as mobility and interaction type [5]. This is how, in the 19th century, John Snow managed to discover the cause of a cholera epidemic in Soho. He plotted the registered cholera cases in a map and saw they clustered around a water pump that he presumed was contaminated [6]. Thanks to Dr. Snow’s findings, water quality started to be considered as an important component of public health. 

As models grow in intricacy, the demand for more powerful computing systems also increases. In advanced approaches such as agent-based [7] and network (graph) models [8], every person is represented inside a complex framework in which the infection spreads according to specific rules. These rules could be related to the nature of the relations between individuals, their number of contacts, the places they visit, disease characteristics, and even stochastic influences. Frameworks are commonly composed of millions of individuals too, because we often want to analyze countrywide effects. 

In brief, to unravel epidemic dynamics we need to process and produce a lot of accurate information, and we need to do it fast. High-performance computing (HPC) systems provide high-spec hardware and support advanced techniques such as parallel computing, which accelerate calculation by using several resources at a time to perform one or different tasks concurrently. This is an advantage for stochastic epidemic models that require hundreds of independent executions to deliver reliable outputs. Frameworks with millions of nodes or agents need several GB of memory to be processed, which is a requirement that can be met only by HPC systems. 

Based on the work of Cruz et al. [9], we developed a model that represents the spread dynamics of COVID-19 in Costa Rica [10]. This model consists of a contact network of five million nodes, in which every Costa Rican citizen has a family, school, work, or random connection with their neighbors. These relations impact the probability of getting infected, as well as the “infection status” of the neighbors. The infection status varies with time, as people evolve from not having symptoms to have mild, severe, or critical conditions. People may be asymptomatic as well. The model also addresses variations in location, school and workplace sizes, age, mobility, and vaccination rates. In addition, some of these inputs are stochastic. 

Such model takes only a few hours to be simulated in an HPC cluster, when normal systems would require much more time. We managed to evaluate scenarios in which different sanitary measures were changed or eliminated. This analysis brought interesting results, such as that going to a meeting with our family or friends could be as harmful as attending a concert with dozens of strangers, in terms of the additional infections that these activities would generate. Such findings are valuable inputs for health authorities, because they demonstrate that preventing certain behaviors in the population can delay the peak of infections and give them more time to save lives. 

Even though HPC has been fundamental in computational epidemiology to give key insights into epidemic dynamics, we still have to leverage this technology in some contexts. For example, we must first strengthen health and information systems in developing countries to get the maximum advantage of HPC and epidemic models. The above can be achieved through inter–institutional and international collaboration, but also through national policies that support research and development. If we encourage the study of infectious diseases, we benefit from this knowledge in a way that we can approach other pandemics better in the future. 

References

[1] Encyclopedia Britannica. n.d. Crisis, recovery, and resilience: Did the Middle Ages end?. [online] Available at: <https://www.britannica.com/topic/history-of-Europe/Crisis-recovery-and-resilience-Did-the-Middle-Ages-end> [Accessed 13 September 2022]. 

[2] Mellinger, J., 2006. Fourteenth-Century England, Medical Ethics, and the Plague. AMA Journal of Ethics, 8(4), pp.256-260. 

[3] Carr, H., 2020. Black Death Quarantine: How Did We Try To Contain The Deadly Disease?. [online] Historyextra.com. Available at: <https://www.historyextra.com/period/medieval/plague-black-death-quarantine-history-how-stop-spread/> [Accessed 13 September 2022]. 

[4] McBryde, E., Meehan, M., Adegboye, O., Adekunle, A., Caldwell, J., Pak, A., Rojas, D., Williams, B. and Trauer, J., 2020. Role of modelling in COVID-19 policy development. Paediatric Respiratory Reviews, 35, pp.57-60. 

[5] Pasha, D., Lundeen, A., Yeasmin, D. and Pasha, M., 2021. An analysis to identify the important variables for the spread of COVID-19 using numerical techniques and data science. Case Studies in Chemical and Environmental Engineering, 3, p.100067. 

[6] Bbc.co.uk. 2014. Historic Figures: John Snow (1813 – 1858). [online] Available at: <https://www.bbc.co.uk/history/historic_figures/snow_john.shtml> [Accessed 13 September 2022]. 

[7] Publichealth.columbia.edu. 2022. Agent-Based Modeling. [online] Available at: <https://www.publichealth.columbia.edu/research/population-health-methods/agent-based-modeling> [Accessed 13 September 2022]. 

[8] Keeling, M. and Eames, K., 2005. Networks and epidemic models. Journal of The Royal Society Interface, 2(4), pp.295-307. 

[9] Cruz, E., Maciel, J., Clozato, C., Serpa, M., Navaux, P., Meneses, E., Abdalah, M. and Diener, M., 2021. Simulation-based evaluation of school reopening strategies during COVID-19: A case study of São Paulo, Brazil. Epidemiology and Infection, 149. 

[10] Abdalah, M., Soto, C., Arce, M., Cruz, E., Maciel, J., Clozato, C. and Meneses, E., 2022. Understanding COVID-19 Epidemic in Costa Rica Through Network-Based Modeling. Communications in Computer and Information Science, pp.61-75. 

By CeNAT

The post Leveraging HPC technologies to unravel epidemic dynamics first appeared on RISC2 Project.

As the utility and usage of HPC infrastructures increases, more computational and storage power is required to efficiently handle the amount of targeted applications. In fact, many HPC centers are now aiming at exascale supercomputers supporting at least one exaFLOPs (1018 operations per second), which represents a thousandfold increase in processing power over the first petascale computer deployed in 2008 [RD+15]. Although this is a necessary requirement for handling the increasing number of HPC applications, there are several outstanding challenges that still need to be tackled so that this extra computational power can be fully leveraged. 

Management of large infrastructures and heterogeneous workloads: By adding more compute and storage nodes, one is also increasing the complexity of the overall HPC distributed infrastructure and making it harder to monitor and manage. This complexity is increased due to the need of supporting highly heterogeneous applications that translate into different workloads with specific data storage and processing needs [ECS+17]. For example, on the one hand, traditional scientific modeling and simulation tasks require large slices of computational time, are CPU-bound, and rely on iterative approaches (parametric/stochastic modeling). On the other hand, data-driven Big Data applications contemplate shorter computational tasks, that are I/O bound and, in some cases, have real-time response requirements (i.e., latency-oriented). Also, many of the applications leverage AI and machine learning tools that require specific hardware (e.g., GPUs) in order to be efficient.

Support for general-purpose analytics: The increased heterogeneity also demands that HPC infrastructures are now able to support general-purpose AI and BigData applications that were not designed explicitly to run on specialised HPC hardware [KWG+13]. Therefore, developers are not required to significantly change their applications so that they can execute efficiently at HPC clusters.

Avoiding the storage bottleneck: By only increasing the computational power and improving the management of HPC infrastructures it may still not be possible to fully harmed the capabilities of these infrastructures. In fact, Big Data and AI applications are data-driven and require efficient data storage and retrieval from HPC clusters. With an increasing number of applications and heterogeneous workloads, the storage systems supporting HPC may easily become a bottleneck [YDI+16, ECS+17]. Indeed, as pointed out by several studies, the storage access time is one of the major bottlenecks limiting the efficiency of current and next-generation HPC infrastructures. 

In order to address these challenges, RISC2 partners are exploring: New monitoring and debugging tools that can aid in the analysis of complex AI and Big Data workloads in order to pinpoint potential performance and efficiency bottlenecks, while helping system administrators and developers on troubleshooting these [ENO+21].

Emerging virtualization technologies, such as containers, that enable users to efficiently deploy and execute traditional AI and BigData applications in an HPC environment, without requiring any changes to their source-code [FMP21].  

The Software-Defined Storage paradigm in order to improve the Quality-of-Service (QoS) for HPC’s storage services when supporting hundreds to thousands of data-intensive AI and Big Data applications [DLC+22, MTH+22].  

To sum up, these three research goals, and respective contributions, will enable the next generation of HPC infrastructures and services that can efficiently meet the demands of Big Data and AI workloads. 

References

[DLC+22] Dantas, M., Leitão, D., Cui, P., Macedo, R., Liu, X., Xu, W., Paulo, J., 2022. Accelerating Deep Learning Training Through Transparent Storage Tiering. IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing (CCGrid)  

[ECS+17] Joseph, E., Conway, S., Sorensen, B., Thorp, M., 2017. Trends in the Worldwide HPC Market (Hyperion Presentation). HPC User Forum at HLRS.  

[FMP21] Faria, A., Macedo, R., Paulo, J., 2021. Pods-as-Volumes: Effortlessly Integrating Storage Systems and Middleware into Kubernetes. Workshop on Container Technologies and Container Clouds (WoC’21). 

[KWG+13] Katal, A., Wazid, M. and Goudar, R.H., 2013. Big data: issues, challenges, tools and good practices. International conference on contemporary computing (IC3). 

[NCR+18] Netto, M.A., Calheiros, R.N., Rodrigues, E.R., Cunha, R.L. and Buyya, R., 2018. HPC cloud for scientific and business applications: Taxonomy, vision, and research challenges. ACM Computing Surveys (CSUR). 

[MTH+22] Macedo, R., Tanimura, Y., Haga, J., Chidambaram, V., Pereira, J., Paulo, J., 2022. PAIO: General, Portable I/O Optimizations With Minor Application Modifications. USENIX Conference on File and Storage Technologies (FAST). 

[OG+15] Osseyran, A. and Giles, M. eds., 2015. Industrial applications of high-performance computing: best global practices. 

[RD+15] Reed, D.A. and Dongarra, J., 2015. Exascale computing and big data. Communications of the ACM. 

[ENO+21] Esteves, T., Neves, F., Oliveira, R., Paulo, J., 2021. CaT: Content-aware Tracing and Analysis for Distributed Systems. ACM/IFIP Middleware conference (Middleware). 

[YDI+16] Yildiz, O., Dorier, M., Ibrahim, S., Ross, R. and Antoniu, G., 2016, May. On the root causes of cross-application I/O interference in HPC storage systems. IEEE International Parallel and Distributed Processing Symposium (IPDPS). 

By INESC TEC

The post HPC meets AI and Big Data first appeared on RISC2 Project.

The RISC2 project supported the participation of five Latin American students, boosting the exchange of experience and knowledge between Europe and Latin America on the HPC fields. After the Summer School, the students whose participation supported by RISC2 wrote on a blog post: “We have brought home a new vision of the world of computing, new contacts, and many new perspectives that we can apply in our studies and share with our colleagues in the research groups and, perhaps, start a new foci of study”.

Distinguished scientists in the HPC field gave lectures and tutorials addressing architecture, software stack and applications for HPC and AI, invited talks, a panel on The Future of HPC and a final keynote by Prof Mateo Valero. On the last day of the week, the ACM School merged with MATEO2022 (“Multicore Architectures and Their Effective Operation 2022”), attended by world-class experts in computer architecture in the HPC field.

The ACM Europe Summer School joined 50 participants, from 28 different countries, from young computer science researchers and engineers, outstanding MSC students, and senior undergraduate students.

The post RISC2 supported ACM Europe Summer School 2022 first appeared on RISC2 Project.

The BCS-CNS hosted the ACM Summer School 2022. From 29 August to 2 September 2022, students, researchers, and professors from all over the world gathered to discuss High-Performance Computing (HPC), Artificial Intelligence (AI) and Machine Learning.

The RISC2 project supported the participation of Latin American students. We had the opportunity to travel from Argentina, Brazil, Chile, Colombia, and Costa Rica to connect with leading researchers in HPC at the ACM Summer School and boost our professional careers. For some of us, it was our first time in Europe. For others, it was the first time we had the chance to visit a research centre that hosts a TOP500 supercomputer such as Mare Nostrum. We shared our latent curiosity to learn, meet, and relate to people from all over the world.

We were welcomed to the ACM School by a legend in the world of HPC, Professor Mateo Valero, director of the BSC. World-class lecturers and researchers introduced us to topics that we had only read about in scientific articles, like specialized processors for machine learning, neuromorphic engineering, technical software development for new architectures, and vector accelerators. We could delve into the state-of-the-art of many lines of study, opening our minds in countless ways. We faced new challenges and found new perspectives that would allow us to advance our research projects and complete our graduate degrees.

Throughout the week, we met colleagues from all over the world with different lines of research, projects, and fields of study. This opportunity allowed us to create new relationships, nurtured us at a cultural level, and built new ties of friendship and possible professional contributions in the future, connecting Europe with Latin America. Likewise, we strengthened relations between Latin Americans, usually separated despite being neighbours. Conversations that initially arose with academic topics ended with more trivial issues, all accompanied by a cup of coffee or even a mate brought directly from Argentina. These conversations go hand in hand with great minds and unique people.

Professors like Valerie Taylor from the Argonne National Laboratory, Charlotte Frenkel from the Delft University of Technology, Luca Benini from the Università di Bologna and ETHZ, and Jordi Torres from Universitat Politècnica de Catalunya, among many others, allowed us to be part of a world that, in many cases, is hard to reach for many students in Latin America. Thanks to the RISC2 project, we had the opportunity to be part of this process, learn and bring back to our countries the knowledge about state of the art in HPC architectural trends and a new vision of the world of computing.

At the end of an intense week of study and conversations, of new knowledge and new friends, we returned to our countries of origin. Together, we have brought a new vision of the world of computing, new contacts, and many new perspectives that we can apply in our studies and share with our colleagues in the research groups and, perhaps, start new foci of study.

Finally, we hope to return and meet again, make new friends, share the knowledge acquired and our experiences, and further deepen the ties within Latin America and between Europe and Latin America. We hope that other fellow Latin Americans will also benefit from similar opportunities and that they can live these kinds of experiences. The RISC2 project gave us a unique opportunity, so we want to thank them and all of those who made it possible.

By:

• Claudio Aracena, University of Chile
• Christian Asch, CeNAT, Costa Rica
• Luis Alejandro Torres Niño, UIS, Colombia
• Matías Mazzanti, UBA, Argentina
• Matheus Borges Seidel, UFRJ, Brazil

The post ACM Summer School as a meeting point for Latin American young researchers first appeared on RISC2 Project.

In accordance with all of the above, it seems appropriate to consider the launch of coordination actions between both domains at both the European and Latin American levels to support the development of adjusted data models and simulation codes for energy thematic areas, while making use of the latest technology in HPC.

With different countries and regions in the world trying to win the race for the development of the most efficient Computing and Artificial Intelligence technology, it seems equally logical to support the development of high-performance computing research infrastructure. Scientific communities can now access the most powerful computing resources and use them to run simulations focused on energy challenges. Simulation enables planning and working towards tomorrow’s clean energy sources in a digital framework, greatly reducing prototyping costs and waste.

As examples, there are several fields in which the advances that the exploitation of digital methodologies (HPC jointly with data science and artificial intelligence) can bring will produce the resolution of more ambitious problems in the energy sector:

• Improvement in the exploitation of energy sources
  • Weather forecast or turbines in off-and on-shore wind energy
  • Design of new devices such as wind turbines, solar thermal plants, collectors, solid state batteries, etc
  • Computational Fluid Dynamics (CFD) analysis of heat transfer between solar radiation, materials, and fluids
• Design of advanced materials of energy interest
  • Materials for innovative batteries via accurate molecular dynamics and/or ab initio simulations to design and characterize at the atomic-scale new cathodes and electrolytes
  • Materials for photovoltaic devices via multiscale simulations where atomic-scale ab-initio simulations are combined with mesoscale approaches to design efficient energy harvesting devices
• Energy distribution
  • Integrated energy system analysis, optimization of the energy mix and smart grids fed with renewable energies, and further distribution in the electricity grid
  • Economic energy models
  • Smart meters and sensor deployment and further application to energy efficiency in buildings, smart cities, etc Exploitation on additional infrastructures such as fog/edge computing
  • New actors (prosumers) in a distributed electricity market, including energy systems integration

Behind all the previous items, there is a solid track of research that forms the foundations of this effort in reinforcing research on digital topics. Some examples are:

• Design of Digitalized Intelligent Energy Systems, for example, their application to cities in which zero-emissions buildings or intelligent power systems are pursued
• Deeper understanding of the physics behind the energy sources, for example, multiscale simulation or of the atmospheric flow for wind farm operation through CFD–RANS or LES simulation coupled to mesoscale models taking advantage of the capabilities offered by exascale computing
• New designs of Fluids Structure Interactions (FSI), for example, for full rotor simulations coupled to Computational Fluid Dynamics (CFD) simulations. Structural dynamics (fatigue) in different devices
• Optimization of codes by the means of new mathematical kernels, not simply computational porting
• Integration of different computing platforms seamlessly combining HPC, HTC, and High-Performance Data Analytics methodologies

Moreover, the advanced modeling of energy systems is allowed thanks to the tight synergy with other disciplines from mathematics to computer science, from data science to numerical analysis. Among the other high-end modeling requires:

• Data Science, as handlina g large volume of data is key to energy-focused HPC and HTC simulations and data-driven workflow
• Designs customized machine and deep learning techniques for improved artificial intelligence approaches
• Efficient implementation of digital platforms, their interconnections and interoperability

By CIEMAT

The post Supercomputing as a great opportunity for the clean energy transition first appeared on RISC2 Project.