EXECUTIVE SUMMARY
Intersect360 Research surveyed the High Performance Computing (HPC) user community on their software usage for HPC. The complete study analyzes users’ current operating systems, middleware, developer tools, and primary applications across 11 different application domains, including:
- the overall mix of in-house, open-source, and commercial software
- the mix of public cloud versus on-premises computing, by application
- scalability and performance on various processing elements, by application domain
- the overall adoption of machine learning.
Our goal in this analysis is to provide an in-depth analysis of how HPC software trends. Intersect360 Research surveyed members of the HPC user community worldwide across industry, government, and academia. The survey was conducted in the fourth quarter of 2021 and received responses from 238 qualified sites, yielding tens of thousands of data points for analysis with respect to HPC software.
Intersect360 Research defines HPC as the use of servers, clusters, and supercomputers—plus associated software, tools, components, storage, and services—for scientific, engineering, or analytical tasks that are particularly intensive in computation, memory usage, or data management. Intersect360 Research reports available in this HPC Technology Survey report series include the following segmentations:
- Operating Systems and Middleware: including operating systems, HPC job management tools, workflow and workload management tools, and software for cluster management, monitoring, and provisioning
- Compilers and Developer Tools: including languages, compilers, and programming environments for HPC software development
- AI Adoption: including machine learning frameworks and current levels of organizational AI engagement
- Primary Application Software: 11 distinct reports covering applications used, typical and maximum job scalability, affinity to cloud computing, and impressions of performance by processor type, for 11 application domains, as follows:
- Primary Application Software—Biosciences: including pharmaceuticals, biochemistry, genomics, agricultural engineering, cryo-EM, etc.
- Primary Application Software—Business Analytics
- Primary Application Software—Chemistry
- Primary Application Software—Computational Fluid Dynamics (CFD)
- Primary Application Software—Engineering: including computer-aided engineering (CAE), finite element analysis (FEA), noise-vibration-harshness testing (NVH), materials science, etc.
- Primary Application Software—Finance: including banking, trading, pricing, risk management and value-at-risk (VaR), econometric modeling, insurance, etc.
- Primary Application Software—Geophysics: including seismic modeling, reservoir simulation, geology, etc.
- Primary Application Software—Physics: including pharmaceuticals, biochemistry, genomics, agricultural engineering, cryo-EM, etc.
- Primary Application Software—Scientific Research: including general fields of scientific inquiry not covered elsewhere
- Primary Application Software—Visualization
- Primary Application Software—Weather and Climate Modeling: including earth science and ocean modeling
This report provides a detailed examination of operating systems and middleware in HPC environments. Operating systems for HPC are found almost exclusively to be variations of Linux, with significant differences in preferences between commercial and public-sector HPC segments. Our middleware analysis is divided into job management and cluster management, monitoring, and provisioning. For each, respondents selected from common packages, listed based on their presence in previous surveys, or wrote in additional packages not listed. Intersect360 Research clients are encouraged to review the data and insights in this report in planning their future developments for all HPC technologies and services.
TABLE OF CONTENTS 4
INTRODUCTION 6
Methodology 6
What Is HPC? 8
OPERATING SYSTEMS 9
Overview: Operating Systems 9
Figure 1: Percent of Respondents Reporting Primary and Secondary Operating Systems 10
Table 1: Proportion of Primary and Secondary Operating Systems 10
Table 2: Operating Systems by Sector: Commercial/Industrial 11
Table 3: Operating Systems by Sector: Academic/NFP Research 11
Table 4: Operating Systems by Sector: Government 12
MIDDLEWARE 13
Overview: Middleware 13
Job Management Tools 13
Figure 2: Percent of Respondents Reporting Primary and Secondary Job Management Tools 13
Table 5: Percent of Respondents Reporting Primary and Secondary Job Management Tools: All Respondents 14
Table 6: Job Management Tools by Sector: Commercial/Industrial 14
Table 7: Job Management Tools by Sector: Academic/NFP Research 15
Table 8: Job Management Tools by Sector: Government 15
Cluster Management, Monitoring, and Provisioning 15
Figure 3: Percent of Respondents Reporting Primary and Secondary Cluster Management, Monitoring, and Provisioning Tools 16
Table 9: Cluster Management, Monitoring, and Provisioning Tools by Sector: All Respondents 17
Table 10: Cluster Management, Monitoring, and Provisioning Tools by Sector: Commercial/Industrial 17
Table 11: Cluster Management, Monitoring, and Provisioning Tools by Sector: Academic/NFP Research 18
Table 12: Cluster Management, Monitoring, and Provisioning Tools by Sector: Government 18
CONCLUSIONS 19
APPENDIX A: SURVEY DEMOGRAPHICS 20
Figure A1: Economic Sector of Respondent Organization 20
Figure A2: Number of Employees for Respondent Site and Organization 20
Figure A3: Annual HPC Budget for Respondent Site and Organization 21
Figure A4: Role of Survey Respondent in Organization 21
Figure A5: HPC Software Purchasing Responsibility of Respondent 22