As the coordinator of the High Performance Computing Center (Nacad) at COPPE/UFRJ, Alvaro Coutinho, presented advances in AI in Engineering and the importance of multidisciplinary research networks to address current issues in Scientific Machine Learning. Alvaro took the opportunity to highlight the need for Brazil to invest in high performance computing capacity.

The country’s sovereignty needs autonomy in producing ML advances, which depends on HPC support at the Universities and Research Centers. Brazil has nine machines in the Top 500 list of the most powerful computer systems in the world, but almost all at Petrobras company, and Universities need much more. ML is well-known to require HPC, when combined to scientific computer simulations it becomes essential.

The conventional notion of ML involves training an algorithm to automatically discover patterns, signals, or structures that may be hidden in huge databases and whose exact nature is unknown and therefore cannot be explicitly programmed. This field may face two major drawbacks: the need for a significant volume of (labelled) expensive to acquire data and limitations for extrapolating (making predictions beyond scenarios contained in the trained data difficult).

Considering that an algorithm’s predictive ability is a learning skill, current challenges must be addressed to improve the analytical and predictive capacity of Scientific ML algorithms, for example, to maximize its impact in applications of renewable energy. References [1-5] illustrate recent advances in Scientific Machine Learning in different areas of engineering and computer science.

References:

[*] https://www.coppe.ufrj.br/pt-br/planeta-coppe-noticias/noticias/coppe-e-sociedade-especialistas-debatem-os-reflexos-da-inteligencia

[1] Baker, Nathan, Steven L. Brunton, J. Nathan Kutz, Krithika Manohar, Aleksandr Y. Aravkin, Kristi Morgansen, Jennifer Klemisch, Nicholas Goebel, James Buttrick, Jeffrey Poskin, Agnes Blom-Schieber, Thomas Hogan, Darren McDonaldAlexander, Frank, Bremer, Timo, Hagberg, Aric, Kevrekidis, Yannis, Najm, Habib, Parashar, Manish, Patra, Abani, Sethian, James, Wild, Stefan, Willcox, Karen, and Lee, Steven. Workshop Report on Basic Research Needs for Scientific Machine Learning: Core Technologies for Artificial Intelligence. United States: N. p., 2019. Web. doi:10.2172/1478744.

[2] Brunton, Steven L., Bernd R. Noack, and Petros Koumoutsakos. “Machine learning for fluid mechanics.” Annual Review of Fluid Mechanics 52 (2020): 477-508.

The post Scientific Machine Learning and HPC first appeared on RISC2 Project.

Date: April 26, 2023 | 3 p.m. (UTC+1)

The post Webinar: Improving energy-efficiency of High-Performance Computing clusters first appeared on RISC2 Project.

The Call for High Performance Computing (HPC) Centers aims to support the acquisition of high performance computing equipment that can provide computational infrastructure to conduct research in all areas of knowledge that are intensive in computing resources. The resources necessary for the development of the infrastructure of the facilities to receive the high performance computing equipment are considered to be the responsibility of the proponent institutions and constitute the required counterpart for the presentation of the proposal. In addition, proposers must demonstrate a proven track record as an HPC center.

This program has the nature of creating infrastructure and is not intended to provide conventional funding for research projects that will eventually take advantage of the infrastructure supported here, and the support for the realization of such research projects should be sought in the lines of funding for research.

A portion of the maintenance costs of the equipment to be purchased may be requested in this Call. However, it is expected that proposals submitted to this Call will also propose other ways to cover equipment maintenance costs. No funds may be requested to cover costs for the maintenance of the building infrastructure and support for computer equipment, such as air conditioning and the like, which should be covered by funds contributed by the proponent institutions or from other sources. Furthermore, the costs of salaries and other charges related to the support staff that this Call for Proposals foresees should be available for the operation of the center cannot be requested in the proposals submitted to this Call for Proposals and are the sole responsibility of the proposing institutions. The proponents may foresee, in their business plan, charging for the provision of the services, provided that some level of gratuity is offered to users from academic institutions.

WHO?

This Call is open to Education or Research Institutions from all over Brazil, consortium or not, to support 1 center in the state of São Paulo and 1 or 2 centers in other Brazilian states, in a total amount of up to R$ 100 million. The center based in São Paulo may receive, in this Call, resources of up to R$ 50 million, and must meet the demand for high performance computing services within the entire state of São Paulo. The centers located in other states may receive resources of up to R$ 25 million, in the case of non-consortium projects, or up to R$ 50 million in the case of a consortium of several institutions that meet the demand for high performance computing services nationwide.

This Call is launched in the scope of FAPESP’s Multiuser Equipment Program – EMU and has an infra-structural nature.

Know more about this call here. 

The post Call for Proposals to Support High Performance Computing Centers FAPESP-MCTI-MCom-CGI.br first appeared on RISC2 Project.

The NLHPC staff  set their  fundamental pillar to focus  efforts on the scientific community and show HPC as an essential tool for country development by getting users from diverging scientific areas, industry and public sector. This entails breaking access barriers to this kind of technology. NLHPC faces this challenge by making training for the basic use of HPC  and scientific software optimization;  which is key in order to make a good use of resources.

The training was carried out within a framework of computational thinking, being the process by which an individual, through his professional experience and acquired knowledge, manages to face problems of different kinds. This could be evidenced in our active participation in the resolution of the proposed activities, which enhanced our abstraction and engineering thinking. We will certainly take this vision of education and collaborative work to our professional environment, in the different roles we play as HPC administrators, teachers and students.

The proper use of computing services involves efforts to perform monitoring, control and infrastructure management tasks. With the help of the tools reviewed during our visit, we will be able to provide our users with the highest standards of quality, security and accessibility.

The joint effort of the RISC2 and EU-CELAC ResInfra projects made it possible for engineers from Colombia, Mexico and Peru to participate in this HPC management course, learn about Chilean culture, gain knowledge and valuable contacts for our profession.

After living this great experience, we hope that in the near future other supercomputing centers replicate this type of initiatives in other parts of the world, thus increasing the communication bridges between HPC administrators from different places, sharing knowledge and experiences.

We are left with the milestone of being part of the First School of HPC Administrators of Latin America and the Caribbean, with experiences that made us grow in professional, academic, and human aspects. As well as with alliances among colleagues and now friends, a network of support as brothers of the same region.

We conclude by thanking Rafael Mayo of CIEMAT for the initiative; Ginés Guerrero, Pedro Schürmann, Eugenio Guerra, Pablo Flores, Angelo Guajardo, Esteban Osorio, José Morales for the knowledge and experiences shared; RISC2 and EU-CELAC ResInfra for providing us with this learning opportunity, supporting the scholarship grant.

By:

Miguel Angel Barrera Arbelaez, Universidad de los Andes, Colombia

Carlos Enrique Mosquera Trujillo, Centro de bioinformática y biología computacional de Colombia BIOS, Colombia

César Alexander Bernal Díaz, Universidad Industrial de Santander, Colombia.

Eduardo Romero Arzate, Universidad Autónoma Metropolitana, México.

Ronald Darwin Apaza Veliz, Universidad Nacional de San Agustín, Perú.

Joel Gonzalez Lara, Centro de Análisis de Datos y Supercómputo, México

The post First School of HPC Administrators in Latin America and the Caribbean: A space for the formation of computational thinking first appeared on RISC2 Project.

This school aimed to train HPC sysadmins with the latest technologies in supercomputing in a two-week training program, and discussed different topics, such as compilations, visualization and monitoring tools, networking, security tools, and installation, configuration and use of SLURM and EasyBuild, among many others.

According to Ginés Guerrero, the Executive Director of the NLHPC and one of the organizers of this training, “the NLHPC team wanted to pass on the knowledge gained for more than a decade to other administrators, so they can benefit from our experience. This has involved a great effort by a team of 7 engineers, putting aside all their tasks for several weeks to prepare an intensive 64-hour school from scratch. In addition, this process has been tailor-made, since the students indicated their own interests through a form.”

In total, the event had 8 participants from various countries: 2 from Mexico, 3 from Colombia, 2 from Chile, and 1 from Peru, leveraging the international networking opportunities and promoting closer relations between the administrators of various supercomputing centers in Latin America, the main goal of the RISC2 project. A team of 7 engineers (Guinés Guerrero, Pedro Schürmann, Eugenio Guerra, Pablo Flores, Ángelo Guajardo, Esteban Osorio, and José Morales) from NLHPC was responsible for delivering all the 35 lectures.

The post RISC2 supported the first school of HPC Administrators in Latin America and Caribe first appeared on RISC2 Project.

As the utility and usage of HPC infrastructures increases, more computational and storage power is required to efficiently handle the amount of targeted applications. In fact, many HPC centers are now aiming at exascale supercomputers supporting at least one exaFLOPs (1018 operations per second), which represents a thousandfold increase in processing power over the first petascale computer deployed in 2008 [RD+15]. Although this is a necessary requirement for handling the increasing number of HPC applications, there are several outstanding challenges that still need to be tackled so that this extra computational power can be fully leveraged. 

Management of large infrastructures and heterogeneous workloads: By adding more compute and storage nodes, one is also increasing the complexity of the overall HPC distributed infrastructure and making it harder to monitor and manage. This complexity is increased due to the need of supporting highly heterogeneous applications that translate into different workloads with specific data storage and processing needs [ECS+17]. For example, on the one hand, traditional scientific modeling and simulation tasks require large slices of computational time, are CPU-bound, and rely on iterative approaches (parametric/stochastic modeling). On the other hand, data-driven Big Data applications contemplate shorter computational tasks, that are I/O bound and, in some cases, have real-time response requirements (i.e., latency-oriented). Also, many of the applications leverage AI and machine learning tools that require specific hardware (e.g., GPUs) in order to be efficient.

Support for general-purpose analytics: The increased heterogeneity also demands that HPC infrastructures are now able to support general-purpose AI and BigData applications that were not designed explicitly to run on specialised HPC hardware [KWG+13]. Therefore, developers are not required to significantly change their applications so that they can execute efficiently at HPC clusters.

Avoiding the storage bottleneck: By only increasing the computational power and improving the management of HPC infrastructures it may still not be possible to fully harmed the capabilities of these infrastructures. In fact, Big Data and AI applications are data-driven and require efficient data storage and retrieval from HPC clusters. With an increasing number of applications and heterogeneous workloads, the storage systems supporting HPC may easily become a bottleneck [YDI+16, ECS+17]. Indeed, as pointed out by several studies, the storage access time is one of the major bottlenecks limiting the efficiency of current and next-generation HPC infrastructures. 

In order to address these challenges, RISC2 partners are exploring: New monitoring and debugging tools that can aid in the analysis of complex AI and Big Data workloads in order to pinpoint potential performance and efficiency bottlenecks, while helping system administrators and developers on troubleshooting these [ENO+21].

Emerging virtualization technologies, such as containers, that enable users to efficiently deploy and execute traditional AI and BigData applications in an HPC environment, without requiring any changes to their source-code [FMP21].  

The Software-Defined Storage paradigm in order to improve the Quality-of-Service (QoS) for HPC’s storage services when supporting hundreds to thousands of data-intensive AI and Big Data applications [DLC+22, MTH+22].  

To sum up, these three research goals, and respective contributions, will enable the next generation of HPC infrastructures and services that can efficiently meet the demands of Big Data and AI workloads. 

References

[DLC+22] Dantas, M., Leitão, D., Cui, P., Macedo, R., Liu, X., Xu, W., Paulo, J., 2022. Accelerating Deep Learning Training Through Transparent Storage Tiering. IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing (CCGrid)  

[ECS+17] Joseph, E., Conway, S., Sorensen, B., Thorp, M., 2017. Trends in the Worldwide HPC Market (Hyperion Presentation). HPC User Forum at HLRS.  

[FMP21] Faria, A., Macedo, R., Paulo, J., 2021. Pods-as-Volumes: Effortlessly Integrating Storage Systems and Middleware into Kubernetes. Workshop on Container Technologies and Container Clouds (WoC’21). 

[KWG+13] Katal, A., Wazid, M. and Goudar, R.H., 2013. Big data: issues, challenges, tools and good practices. International conference on contemporary computing (IC3). 

[NCR+18] Netto, M.A., Calheiros, R.N., Rodrigues, E.R., Cunha, R.L. and Buyya, R., 2018. HPC cloud for scientific and business applications: Taxonomy, vision, and research challenges. ACM Computing Surveys (CSUR). 

[MTH+22] Macedo, R., Tanimura, Y., Haga, J., Chidambaram, V., Pereira, J., Paulo, J., 2022. PAIO: General, Portable I/O Optimizations With Minor Application Modifications. USENIX Conference on File and Storage Technologies (FAST). 

[OG+15] Osseyran, A. and Giles, M. eds., 2015. Industrial applications of high-performance computing: best global practices. 

[RD+15] Reed, D.A. and Dongarra, J., 2015. Exascale computing and big data. Communications of the ACM. 

[ENO+21] Esteves, T., Neves, F., Oliveira, R., Paulo, J., 2021. CaT: Content-aware Tracing and Analysis for Distributed Systems. ACM/IFIP Middleware conference (Middleware). 

[YDI+16] Yildiz, O., Dorier, M., Ibrahim, S., Ross, R. and Antoniu, G., 2016, May. On the root causes of cross-application I/O interference in HPC storage systems. IEEE International Parallel and Distributed Processing Symposium (IPDPS). 

By INESC TEC

The post HPC meets AI and Big Data first appeared on RISC2 Project.

Mexico has a solid tradition in scientific research and education and has excellent human resources in the computer science field. Participants focused on promoting cooperation between supercomputing centers across the country. With that aim, they focused on the need to improve connectivity and engage on networks that allow for efficient and effective coordination among centers.

Leveraging from other cooperation-focused initiatives, like the Red Española de Supercomputación (RES) in Spain and PRACE at the European level, the Mexican HPC community is seeking to kick-off a nationwide collaboration network.

The post CINVESTAV hosts a meeting with representatives of supercomputing centers in Mexico first appeared on RISC2 Project.

The fund, granted by the Chilean National Agency for Research and Development, provides the possibility to finance and maintain the center’s team of engineers for the next five years, with the chance of renewal. If everything is going well and according to plan, the Center will be granted funding for another five years. The fund gives NLHPC the confidence to continue to grow and reach their objectives.

“Collaboration with international networks was important in the evaluation of this call. Undoubtedly, the fact of being part of RISC2 has been very relevant for us to be awarded the project”, stated Ginés Guerrero, NLHPC Director.

The NLHPC is the national supercomputing center in Chile, with specialization in high performance computing, that manages Guacolda-Leftraru, the most powerful supercomputer in South America. This infrastructure appeared in November in the IO500 list, a ranking of the fastest supercomputing (HPC) storage systems in the world, where it was placed in 39th place.

During the last year, more than 400 users from different areas of knowledge, more than 45 institutions have used the NLHPC. The center is at the service of the national scientific community, industry, and State and its main goal is to satisfy the demand for high performance computing.

Photo: Comunicaciones FCFM- U. de Chile

The post The National Laboratory for High Performance Computing awarded funding for 5 years first appeared on RISC2 Project.

This funding will help continue the work of the NLHPC, while strengthening its supercomputing infrastructures, which have been crucial to the national scientific community.

It is important to mention that the submission to the contest was supported by the RISC2 project, since the NLHPC’s team received support letters from the RISC2 consortium, including BSC, RedCLARA, RICAP, and SCALAC.

According to ANID, “the II Contest for Major Scientific and Technological Equipment Fondequip seeks to install scientific capacity, covering the country’s need for a larger and more extensive infrastructure, which houses sophisticated and innovative scientific equipment, promoting and facilitating the development of research excellence in the national territory with this scientific and technological resources. It also seeks to position Chile at the forefront of research excellence and frontier research at the international level”.

About NLHPC

The NLHPC specializes in HPC and manages Guacolda-Leftraru, the most powerful supercomputer in Chile and one of the most powerful in South America.

The NLHPC is at the service of the national scientific community, the State, and the industry that requires HPC services. Its main mission is to meet the national scientific demand for high performance computing, providing high-quality services and promoting its use in both basic and applied research problems.

The center began to take shape in 2009 as an initiative of the Center for Mathematical Modeling (CMM) at the Faculty of Physical and Mathematical Sciences of the University of Chile, which invited different institutions to join efforts towards the creation of a national supercomputing laboratory. The response of these institutions was very enthusiastic, and their support allowed the center to start operating in 2011. Since then, most research institutions in the country are part of the center, constituting the largest scientific network in Chile sharing an infrastructure. The laboratory has also signed collaboration agreements with several state institutions, international networks, and supercomputing centers around the world.

Photo: Guacolda-Leftraru, by National Laboratory for High Performance Computing (NLHPC)

The post National Laboratory for High Performance Computing awarded Fondequip Major funding first appeared on RISC2 Project.