NCSA director Bill Gropp on ‘imagining an HPC utopia’

Editor’s note: This is part of a series of virtual visits with NCSA thought leaders on current topics impacting the field of high-performance computing.

Reminiscing and Imagining an HPC Utopia
By Bill Gropp, NCSA Director

There are certain moments both in life and in a career that inspire reflection. I’m currently experiencing one of those times. After nearly a decade serving as NCSA’s director, in a few short weeks, I’ll be handing off the expansive responsibilities of leading a supercomputing and research center to focus on some more specific scientific ventures.

What a journey it has been.

Back in college, I switched to computing during my senior year, when I took both a numerical analysis and a symbolic computing course. That’s when I really felt a sense of, “Yes, this is what I want to be doing.” I discovered how to develop software that can accomplish interesting tasks. The numerical analysis course taught me how to solve problems with a computer and helped me understand how things work. It’s not magic, it’s mathematics. Then the symbolic computing course showed that computers can do more than just work with numbers. I remember building a system to symbolically differentiate expressions. It was one of the homework assignments I found really stimulating. It was exciting to build things quickly that could do enormous amounts of work. By building software, you can have a significant impact, far beyond what you can achieve yourself.

One of the highlights of my career followed that exact philosophy. Early in the history of the Message Passing Interface (MPI), the United States Air Force issued a call for proposals that required the use of MPI. They essentially wanted applications that wouldn’t tie them to particular hardware. And that’s when I knew that we probably had a lasting impact because that was part of the goal: to provide that transportability of codes between systems. I also recall when someone dropped a paper on my desk detailing the use of software we’d written to plan brain surgery. That was quite the moment of “Oh my goodness, people are really taking this seriously.”

Everlasting NCSA Memories
It’s those creative strategies and scaling approaches that made the National Center for Supercomputing Applications such an excellent fit for me, and why I will miss so much from this place. Working with the teams here has really been awesome. I’m continuing as principal investigator of Delta and DeltaAI, so I still get to do some of that. Still, there are so many creative and innovative people at NCSA who really want to make a difference. And working with them daily, I’m going to miss that.

But I’m proud of what we’ve accomplished together, especially in demonstrating that NCSA is more than just running big systems. Important as that still is, the shift in perception set a major direction and has been crucial to our growth. We took advantage of new opportunities – particularly with U.S. National Science Foundation-funded Category One systems – to deploy GPU-heavy resources and be more adventurous with the hardware, which was critical because of the explosive emergence of artificial intelligence.

And it’s more than hardware. Early in my tenure, we set out to fully implement software and applications in our strategies. Working together, we prioritized a year-long reorganization of NCSA to emphasize that software, research consulting and partnerships are as important as hardware. More recently, launching Illinois Computes and securing commitments from University of Illinois Urbana-Champaign leaders to support research computing were additional, significant steps forward.

Best Version of HPC
We’ll continue to evolve as the world changes around us. I’ve always felt, as I did when I chaired SC13, that high-performance computing is everywhere. I define HPC as “computing where you care about performance.” We use computing to solve problems, giving us a greater ability to interact with the world. An HPC utopia would be one where it’s easy to access powerful computing resources, use them effectively, understand their efficiency, and feel confident that things are as efficient as possible. There also has to be greater confidence in the answers – not just that you run the application, but that it’s producing a correct outcome. People would regularly utilize HPC, and it’d be simple enough to use for a wide range of purposes, not just for engineers and scientists, but also for individuals in other areas of scholarship and even in non-academic settings. “HPC at home” to help create solutions to things like electronics energy vampires, and to organize 20 years’ worth of photographs and memories. Access to HPC would be paramount.

A dystopian world of HPC is the opposite. Only the elite or those with substantial wealth and resources would have access to it. It’s a fear that many of us share regarding the direction of AI, specifically the concern that only the commercial sector will have the resources to meet the enormous scale that AI requires. Another aspect of the dystopia would be that people are unaware of whether the code is efficient, and they don’t care. This is problematic because of its impact on the environment. Inefficient applications use far more power (and water for cooling) than is necessary. They also lack knowledge of whether it works or not, and consequently, they don’t care. We are already seeing this in some areas.

Pairing Collaboration and Competition
I’m a strong advocate for open research, and that is the best path toward our HPC utopia. Open basic research is the most effective way of advancing our ability to do anything. Most of the projects I’ve been involved in are open source, even though they’ve been used to produce commercial and proprietary outcomes. MPI was used to train ChatGPT, as noted in the OpenAI blog, which discusses its implementation. It’s important to maintain open science, but, at the same time, you don’t have to sacrifice the commercial facet.

I’m also a big proponent of the roles of government and society to fund early-stage research. While the commercial sector could help with some of that, it’s not something that they’re doing as much of these days. There’s a tendency to be proprietary early, and I think that’s one of the reasons why the role of government is so important. You need both in research: collaboration and competition. There was an awful lot of government-funded early research that has enabled the current competitive landscape, which is doing a great job of advancing the state of the art.

One of the goals of MPI was to collectively create a standard that would enable people to write applications that utilize parallel computers. As I’ve said, no one really wants to write a parallel program; they do it because they have to. But because there was a standard, it meant that the vendors could now compete on the basis of the hardware and low-level software implementations they had, which they could not do effectively when the software was written for a particular system, because the cost of transporting the software from somebody else’s system to a new vendor’s system was too high. The collaboration actually enabled competition. MPI meant that the risk was mitigated, and that led to increased competition. Advanced computing research can do both.

That is what the HPC community should be thinking about. Competition makes you look hard at what you’re doing. It encourages you to innovate and take chances that you might not otherwise. One of the questions people have often heard me ask is, “What is the comparison to the cloud? We’re doing a data storage system. How does that compare to what I can get on the cloud?” We come back with good answers, but it’s a question you ask and one you have to keep asking. The answer may change over time, and you must be prepared for that.

Under new leadership, I’m confident NCSA will continue to ask the right questions, push the boundaries of innovation and nurture the growth of HPC in a creative and responsible manner. My suggestions will now truly be just that – suggestions. But I’m excited to see what the coming years hold for a place and a community that will always have a special place in my heart.