How HLRS Empowers Computational Engineering

This regional strength in engineering has also played an important role in shaping HLRS’s evolution. Based on the campus of the University of Stuttgart — itself a center for research and higher education with an emphasis on engineering and the applied sciences — HLRS has for decades maintained a state-of-the-art infrastructure for computational simulation, visualization, and data analytics. Such tools have become essential for creating models of natural phenomena that are so complex or that occur at such small scales that they are difficult to study in any other way. In engineering, the design of many precision products for which Germany is renowned has become unimaginable without such computational models.

HLRS has long provided essential supercomputing resources for scientists in disciplines such as computational fluid dynamics, physics, chemistry, and materials science, among other fields. This includes not only academic researchers, but also scientists in industrial research and development — for example, in the automotive, energy, and aerospace industries. In fact, approximately 10% of HLRS computing hours are dedicated to industrial users, a figure that stands out as particularly noteworthy in Germany’s academic supercomputing landscape.

Bridging the academic and industrial worlds in this way has enabled HLRS to develop expertise in applying high-performance computing (HPC) to many kinds of real-world computational engineering problems. This knowledge has, in turn, guided its strategy for growth. HLRS’s goal of providing resources that optimally serve the needs of computational engineers has been a key driver of investments in its new computing infrastructure, professional training offerings, and in-house computer science research. As HLRS lays the groundwork for ever-larger supercomputers in the future, the center is also focused on providing the tools and services the engineering community will need to take full advantage of these resources as they become available.

For Stefan Hildenbrand at chemicals manufacturer Pfinder KG, HLRS has become an important partner, enabling the company to develop new services for clients in the automotive industry. Pfinder supplies wax-based anticorrosive coatings that are typically applied by assembly line robots during car manufacture. As team leader for digital engineering, Hildenbrand performs computational simulations to help optimize this process. Specifically, he uses OpenFOAM, a widely used software package for computational fluid dynamics, to predict how the coatings will behave when injected by a nozzle, the paths the droplets will follow through the air, and how they will form films on surfaces. These models are used to optimize the amount of fluid that needs to be sprayed for a specific application, to identify the best nozzle layout, and to ensure they are fully protective and durable.

Pfinder has a small parallel computing system in-house, but when complex questions arise or when results need to be delivered quickly, Hildenbrand turns to HLRS. “When we are deep in a project for a client that requires a large-scale simulation,” Hildenbrand explained, “we can’t wait four weeks to run the computation on our own system. HLRS makes it possible for us to quickly run a job on many hundreds of cores as opposed to on the 16 we have here. Instead of waiting for up to 4 weeks, this can deliver a result in as little as 3 to 4 days.”

Being able to call on HLRS also has other advantages over installing its own larger HPC system. Because Pfinder’s computational workload fluctuates widely, it would not make economic sense to install a large system of its own, only to have it sit idle for multiple months of the year. Moreover, Hildenbrand can be confident that HLRS uses reliable, state-of-the-art hardware that continually grows in power as HPC systems evolve. Keeping up with changes in HPC technologies would be impossible for a small company.

Having access to this kind of computing power became particularly important for Hildenbrand in 2018, when for the first time his team simulated the application of Pfinder coatings not just for one component, but for an entire automobile simultaneously. This jump in scale required a leap in computing power that was far beyond the reach of its own system. He anticipates continuing to utilize the HLRS supercomputer in the future, as this ability to model entire systems could offer a new market opportunity for his firm to provide complete coating application solutions.

Users of HLRS’s HPC systems will soon have even more power at their fingertips. In November 2018, HLRS and Hewlett Packard Enterprise announced a joint collaboration to build a next-generation supercomputer that will be 3.5 times faster than Hazel Hen, HLRS’s current flagship system. The upcoming supercomputer, called Hawk, will be the fastest in the world for industrial production, powering computational engineering and research across science and industrial fields to advance applications in energy, climate, mobility, and health. The 5,000-node Hawk system will have a theoretical peak performance of 24 petaFLOPs.

An important feature of Hawk is that its technical specifications were chosen with the specific needs of computational engineers in mind. Hawk’s architecture is based on chip manufacturer AMD’s next-generation EPYC processor, code-named Rome. HLRS picked the Rome processors because they utilize a memory subsystem that makes them particularly well suited for simulation applications in fields such as computational fluid dynamics, molecular dynamics, and other research areas in which many of HLRS’s users are engaged. In addition, the AMD EPYC processors complement the use of competing processors at the other two centers in the Gauss Centre for Supercomputing — the alliance of Germany’s three national supercomputing centers. The choice supports GCS’s goal of offering its users a diverse set of computing architectures.

“We are excited that Hawk constitutes a sizable increase in the performance of our flagship supercomputing system,” said Prof. Dr. Michael M. Resch, Director of HLRS. “But the real winners will be our user community of computational engineers in academic research and industry who will benefit from the ability to run much more complex simulations.”

There is no doubt that larger supercomputers enable larger simulations and increase the number of core hours available for the research community. At the same time, however, it has become clear that building hardware alone is not enough to meet the growing needs of computational engineering.

Writing software for large-scale parallel computing systems like Hazel Hen or Hawk is much more complex than running the same algorithm on a desktop computer or a small computing cluster. Indeed, developing and implementing effective and efficient codes for HPC systems requires specialized knowledge, and will be especially important as supercomputers approach the exascale, the next major plateau in computing power over that of current petascale systems. Optimizing codes for such extremely parallelized systems will be necessary to accelerate individual computational scientists’ research and to make the operation of supercomputers more efficient — this will, in turn, help save energy and make HPC resources available to the largest number of potential users.

For many years, HLRS has recognized that the future of high-performance computing lies in developing and increasing access to state-of-the-art codes capable of scaling to ever-larger systems. For these reasons, the center has undertaken several projects aimed at preparing computational engineers for the future.

In 2018, for example, HLRS became the lead coordinating center of a new Horizon 2020-funded research program called EXCELLERAT (European Centre of Excellence for Engineering Applications). A collaborative network including 13 partners from across Europe, EXCELLERAT will provide the HPC and engineering communities access to scalable applications and facilitate their availability through technology transfer.

Recognizing the need for such resources and expertise, companies including Porsche, Ansys, and Festo have expressed support for EXCELLERAT. Aircraft manufacturer Airbus has also been an early participant in the project, as EXCELLERAT will enable academic knowledge to be connected with industrial needs.

In addition to EXCELLERAT, HLRS participates in projects focused on identifying and addressing opportunities for supercomputing in specific industries. For more than 10 years it has been a key partner in the Automotive Solution Center Stuttgart, an alliance of companies that use simulation in automotive development and design. In 2018, it also joined the Hochschule der Medien und the Center for Art and Media in Karlsruhe (ZKM) in founding the Media Solution Center, an alliance of organizations interested in exploring opportunities for high-performance computing in the media arts. In coming years, HLRS intends to implement this solution center model to focus on increasing access to high-performance computing for health research and other areas.

Providing HPC training has long been a key activity for HLRS. In 2018, however, the center opened a new continuing education program designed with the needs of computational engineers in industry in mind. To make these training activities as accessible as possible, the new program, called the Supercomputing-Akademie, uses a “blended learning” format. This means that only a small part of the training actually takes place in a classroom. Approximately 80% of a participant’s time spent learning can be done online, making it easy for someone with a full-time job to join in the course alongside his or her normal responsibilities.

In July 2018, the Supercomputing-Akademie — which is supported by the Baden-Württemberg Ministry for Social Affairs and Integration through the European Social Fund, and by the Baden-Württemberg Ministry of Science, Research, and Art — completed its first 15-week course on parallel programming. In 2019, it began offering a second course, focusing on simulation, and will soon launch a new course addressing the planning, building, and running high-performance computing cluster systems. In the coming years, the curriculum will grow to include additional modules on visualization; performance optimization; cluster, cloud, and high-performance computing; ecology and economics of supercomputing; and data management — topics that are of great interest to system administrators, programmers, and HPC software users in industry.

The Supercomputing-Akademie is just one component of an ongoing dialogue between HLRS and computational engineers in industry. Working with the nonprofit organization SICOS-BW, the center has for the last two years also organized the Industrial HPC User Roundtable (iHURT), a full-day event designed to promote information exchange in industrial R&D. This dialogue also enables HLRS to understand the most important challenges that HPC users in industry face, and will continue to guide its development of new infrastructure and services in the coming years.

In the coming decade, simulation, visualization, data analytics, and machine learning will remain critical for research and development, and will play important roles in addressing some of our greatest challenges. By identifying and focusing on the needs of engineers and applied scientists, HLRS aims to provide the widest possible access to such tools.

Installing the next-generation Hawk supercomputer will be one important step toward this goal. Additionally, HLRS’s ambitious training program and its outreach and training programs focused on industry are important parts of this effort. Through such activities, HLRS is positioned to be a key partner for innovation in Baden-Württemberg, in Germany, in Europe, and beyond.

— Christopher Williams