Workshop Abstract
Modern software engineering is getting increasingly complicated. Especially in the HPC field, we are dealing with cutting edge infrastructure and a novel problem with unprecedented scale. The ability to monitor and analyze the performance of such applications and infrastructure is imperative for the future of improvement, design, and maintenance. In the current era, the writing and maintenance of these applications have ceased to be the job solely of computer scientists and have grown to encompass a wide variety of experts in mathematics, science, and other engineering disciplines. The fact that many developers from these disciplines have not received a formal education in computer science and rely increasingly on the tools created by computer scientists to analyze and optimize their code shows that there's a need for a forum to work together.
Workshop Overview
The Workshop on Performance EngineeRing, Modelling, Analysis, and VisualizatiOn STrategy (PERMAVOST) goal is to bridge tools developers and end users of performance analysis tools. It is a half day workshop with a keynote in conjunction with HPDC 2026. We are hoping that the stakeholders, which are application developers, domain scientists, analyst, and tools developers can collaborate and build a bridge to fill in the gaps in various topics such as:
- Key metrics, patterns, and performance pitfalls: Identifying strategies to recognize and leverage performance insights to enhance application efficiency.
- Emerging challenges: Addressing issues arising from new computing architectures, programming paradigms, novel scientific problems, and the processing of data at varying scales.
- Usability in performance tools: Exploring how modern usability design principles can be integrated into performance analysis tools to better support their users.
- Broad-spectrum accessibility: Developing analysis methods and methodologies that cater to users with varying levels of HPC expertise.
Topics of Interest
Our workshop encompasses the following topics of interest, but are not limited to:
- Performance analysis and modeling of real-world applications
- Data visualization for performance analysis
- Usability studies and user-centric design of HPC tools
- Inefficiencies in programming patterns and computing architectures
- Patterns, anomaly detection, and performance characterization in HPC applications
- Performance engineering strategies, methodologies, and use cases
- Human-computer interfaces for performance data exploration
- Energy efficiency and management in performance analysis and engineering
- Performance analysis in emerging HPC domains, including Artificial Intelligence, Machine Learning, Quantum Computing, Containers, and Cloud computing
Call for Paper
All submitted papers should be formatted using the ACM Master Template with sigconf format (please be sure to use the current version). The necessary document can be found here.
General Instructions
- Full paper is 5-8 pages (including all text, figures, and references)
- Submissions must be in English and PDF format
- Only web-based submissions are allowed. Paper submission link is https://permavost26.hotcrp.com/
- We use a single-blind reviewing process so you can keep authors' names, publications, etc.
- Each paper will get at least three reviews from the committee members
- The submitted papers must be original work that has not previously been published or under consideration for publication in any other conference or journal
- Accepted papers will be published Open Access in the ACM Digital Library with no additional fee, as part of the workshop proceedings.
Program
Keynote Speech:
 |
Glenn Lockwood VAST Data |
"Workflow Performance Is Not the Sum of Its Parts"
Performance and progress have long been synonymous in the world of high-performance computing: a faster system was a better system, so performance tools and benchmarks were an effective way to predict how productive a supercomputer could be. This held because models were fixed, and computers only had to execute them quickly. Data-driven discovery (and AI) breaks this equivalency, because models must be trained and refined while they produce output. A machine can run at full speed while making no progress at all, because today's performance tools measure speed, not progress.
This forces a change in how performance must be measured. Because their models evolve over time, AI workflows have a far more complex relationship with infrastructure than traditional simulation, one that must be understood end-to-end, not layer by layer. Using production examples, I will show how individual components can each perform optimally in isolation while the composed system still runs data-driven workflows inefficiently. Performance is not the sum of its parts, and I will offer a few directions in which performance analysis tooling might need to evolve to capture performance as it actually emerges from these interactions.
Speaker Bio:
Glenn K. Lockwood is the Principal Technical Strategist at VAST Data, where he focuses on the relationship between AI workload mechanics and infrastructure architecture at scale. He has designed and operated some of the world's largest computing systems across multiple contexts: at Microsoft Azure, he contributed to GPU cluster architecture and reliability engineering for systems used to train some of the world's leading frontier language models, and at the National Energy Research Scientific Computing Center (NERSC), he led the design of hundreds of petabytes of data infrastructure ranging from early production all-NVMe Lustre file systems to deep tape archives. His work has spanned both on-premises supercomputers and cloud-scale infrastructure, giving him perspective on the evolution of extreme-scale computing across public and private sectors. He holds a Ph.D. in Materials Science from Rutgers University.
Invited Speech:
 |
Bogdan Nicolae Argonne National Laboratory, USA |
"Performance Analysis for LLM Inference: Key Metrics, Patterns, and Opportunities"
As AI becomes a primary instrument of scientific discovery and powers agentic, reasoning-driven workflows under initiatives such as the DOE Genesis Mission, inference has overtaken training as the dominant and most cost-sensitive AI workload. At its core sits the KV cache: the data structure that makes inference fast, and the bottleneck that makes it complex. Batching, paged attention, eviction, and parallelism interact in non-obvious, workload-dependent ways, so better systems cannot be designed by intuition, they must be measured. This talk discusses the role of performance analysis in this context and hints at a practical methodology: understand the system, collect the right metrics at the right granularity, and guard against the performance pitfalls, notably low-overhead monitoring. Drawing on recent studies of KV caching and inference scaling, it shows how detailed analysis exposes key patterns and performance pitfalls, turning them into concrete solutions for faster, more scalable LLM inference.
Speaker Bio:
Bogdan Nicolae is a Computer Scientist at Argonne National Laboratory, USA. In the past, he held appointments at Huawei Research Germany and IBM Research Ireland. He specializes in scalable storage, data management and fault tolerance for large scale distributed systems, with a focus on high performance architectures cloud computing. He holds a PhD from University of Rennes 1, France and a Dipl. Eng. degree from Politehnica University Bucharest, Romania. He is interested by and authored numerous papers in the areas of scalable I/O, checkpointing techniques, data and metadata decentralization and availability, multi-versioning, data-intensive and big data analytics, storage elasticity and virtualization, live migration.
Paper Presentation:
- Performance Analysis of Conveyors: Memory Dominates? - S. Singhal, A. Welch, O. Hernandez, S. Poole, A. Hayashi, V. Sarkar
- A Lock-Free Work-Stealing Algorithm for Bulk Operations - R. Kataru, D. Davarnia, A. Jannesari
- Accelerating high-accuracy runtime estimation in adaptive sample selection for parallel scalability analysis with reinforcement learning - P. Rici, E. Lucena, S. Xavier-de-Souza
*The paper presentation will be followed by Q&A session
Program Schedule
- 13:30 – 13:35 Welcome & Opening Remarks
- 13:35 – 14:25 Keynote by Glenn Lockwood
- 14:25 – 15:00 Invited Talk by Bogdan Nicolae
- 15:00 - 15:30 Coffee Break
- 15:30 – 16:55 Paper Presentation with Q&A
- Performance Analysis of Conveyors: Memory Dominates?
- A Lock-Free Work-Stealing Algorithm for Bulk Operations
- Accelerating high-accuracy runtime estimation in adaptive sample selection for parallel scalability analysis with reinforcement learning
- 16:55 – 17:00 Closing Remarks
Program Co-Chairs
- Radita Liem - Johannes Gutenberg University Mainz
- Zoya Masih - University of Göttingen
- Ray Sinurat - University of Chicago
Program Committee
- Chen Wang - Nanyang Technological University
- Connor Scully-Allison - The University of Chicago
- Francois Tessier - INRIA Rennes
- Hariharan Devarajan - Lawrence Livermore National Lab
- Huihuo Zheng - Argonne National Lab
- Jay Lofstead - Sandia National Lab
- Lenny Guo - PNNL
- Matthieu Dorier - Argonne National Lab
- Orcun Yildiz - Argonne National Lab
- Phil Carns - NVIDIA
- Sandra Mendez - Barcelona Supercomputing Center
- Sarah Neuwirth - Johannes Gutenberg University Mainz
- Shadi Ibrahim - INRIA Rennes