E4S Software Stack
Script error: No such module "Draft topics". Script error: No such module "AfC topic". Script error: No such module "AfC submission catcheck". The Extreme-scale Scientific Software Stack, or E4S, is a community initiative that provides open-source software packages for scientific applications on high-performance computing (HPC) platforms. It includes from-source builds and containers of various HPC software packages.[1] E4S allows contributions from teams around the world and enables collaborative integration of open-source products. It also targets scalable next-generation computing platforms.[2] The software is based on the Spack package manager[3] and includes Exascale Computing Project (ECP) software technologies (ST) and related software tested for compatibility and portability across multiple architectures. It aims to distinguish between usability, quality, and community goals for software development kit (SDK), deployment, and testing purposes.[4] The ECP software stack, E4S Release 0.2, is a subset of the full ECP ST software stack and demonstrates the target approach for future delivery. It includes some ECP ST software products that are supported by Spack packages but are not yet interoperable. This release is mainly focused on demonstrating the ST software stack approach, and not all ECP ST software products were included.[2]
Interoperability and Software Delivery[edit]
To achieve software interoperability, E4S uses a dual-pronged approach consisting of Spack and SDKs.[5] It also allows the use of containers and being able to deploy packages on bare metal systems. Spack serves as a package manager for deploying systems using E4S recipes. Alternatively, it is possible to download a container and image from e4s.io and deploy a container based on the chosen image. Spack is also used to manage different versions of products, interoperability, and access to runtimes and compilers. It is tested for both interoperability and portability.[4] Spack uses Python to assist with the process of installing libraries and simulations and optimize their performance. It also allows users to build code configurations, ensures that all installed software runs correctly, and streamlines file management. Spack can additionally install numerous variants of the same build using different message-passing interface (MPI) implementations, different options, and different compilers.[6] E4S uses the Spack package manager for software delivery since it provides the ability to distinguish between versions of software packages that are interoperable and those that are not.[7] Writing package recipes for Spack consists of a single file containing a templated recipe for other builds of the same package, meaning recipe authors can differentiate between different versions by using a specification language developed by Todd Gamblin. It simplifies the installation of software packages by handling the complexity of connecting them with a consistent set of dependencies. Spack does this by creating different configurations for each software package, installing them in unique directories, and using RPATH[8] linking so that each package knows where to find its dependencies. Spack helps ensure compatibility between program modules by guaranteeing one configuration of each library per dependency graph. It enables users to specify dependencies without needing to understand the dependency graph structure, as well as offer an optional concretization feature that fills in missing configuration details based on user/site preference.[3] E4S facilitates the testing and use of an abundance of reusable HPC software packages through the use of Spack as a meta-build tool, along with supplying containers of pre-built binaries for Singularity,[9] CharlieCloud,[10] Docker,[11] and Shifter.[12][13] An ECP ST Software Development Kit is a group of related software tools (called packages) that can be used together to enhance user experience, provide consistent practices, and promote collaboration among teams that create similar or related capabilities. The Extreme-Scale Scientific Software Development Kit (xSDK) is featured in the initial version (0.2) of E4S.[5] SDKs help to reduce the complexity of delivery by utilizing hierarchical build targets and distributing software integration responsibilities.[4]
Objectives and Challenges[edit]
The objectives of composing a modular, interoperable, and deployable software stack include lowering the barrier to using ST products and enabling facilities to install all or parts of it, allowing for interoperability between the ST products, along with uniform application programming interfaces, and establishing an open, hierarchical software architecture that allows for collaboration between agencies and software institutions. These objectives also come with challenges, such as the large, diverse group of ST products, the difference in project management styles, the lack of initial drivers, and the complicated software ecosystem.[14] The E4S Software Stack provides a complete HPC software stack that does not significantly increase the number of products in the HPC ecosystem. It provides a way of building, installing, and testing the software, and also improves the stability of the ST stack. Users can get E4S on the web page, e4s.io, by downloading the containers and accessing the docker hub and the other build cache.[4][12]
Features[edit]
The E4S software stack offers a range of open-source libraries and tools for high-performance computing applications, including:
- High-performance numerical libraries
- Productivity tools and frameworks
- Performance analysis and profiling tools
- Machine learning libraries and frameworks
- Data management and visualization tools
- Support for multiple programming languages
- Distributed-memory parallel computation support through MPI, and shared-memory parallel computation support
Programming Languages[edit]
The E4S software stack includes support for a variety of programming languages.
Most of the packages in the E4S software stack are written in C++,[15] including packages such as Spack and Kokkos.[16][17]
Other programming languages are also supported in the E4S software stack. Python[18] is a language for scientific computing and data analysis, and many E4S packages have Python bindings, including packages like PyTorch[19] and TensorFlow.[20][21]
In addition to C++ and Python, Fortran[22] is also supported in some packages in the E4S software stack.[23] For example, PETSc[24] is a package for solving partial differential equations, and it has optional implementations of some computational kernels in Fortran.[25][17]
Other programming languages with bindings to E4S packages include C[26] and Matlab.[27]
Software Licenses[edit]
The E4S software stack consists of various packages, each with their own specific software license. Most of the packages in the E4S stack are open-source and licensed under the Apache License, Version 2.0.[28] However, there are some packages that use other open-source licenses, such as the BSD-3-Clause license[29] and the GNU Lesser General Public License (LGPL).[30] Additionally, some packages use proprietary licenses, such as the Intel Math Kernel Library (MKL).[31]
The E4S software stack also includes various dependencies, which have their own licensing terms. For example, the BLAS[32] and LAPACK[33] libraries, which are required dependencies for some packages in the E4S stack, are typically licensed under the BSD-3-Clause license.
External links[edit]
References[edit]
- ↑ Dröge, Bob; Holanda Rusu, Victor; Hoste, Kenneth; van Leeuwen, Caspar; O'Cais, Alan; Röblitz, Thomas (January 2023). "EESSI: A cross-platform ready-to-use optimised scientific software stack". Software: Practice and Experience. 53 (1): 176–210. doi:10.1002/spe.3075. ISSN 0038-0644. Unknown parameter
|s2cid=ignored (help) - ↑ 2.0 2.1 "Extreme-Scale Scientific Software Stack (E4S)". oaciss.uoregon.edu. Retrieved 2023-10-25.
- ↑ 3.0 3.1 "Spack". Spack. Retrieved 2023-10-25.
- ↑ 4.0 4.1 4.2 4.3 Gibson, Scott (2021-12-14). "Helping Users Deploy High-Performance Computing Tools and Libraries in a Consistent Interoperable Software Stack". Exascale Computing Project. Retrieved 2023-10-26.
- ↑ 5.0 5.1 "E4S - Home". e4s-project.github.io. Retrieved 2023-10-25.
- ↑ "Spack | Computing". computing.llnl.gov. Retrieved 2023-10-26.
- ↑ "Expediting Research with Spack". str.llnl.gov. Retrieved 2023-11-08.
- ↑ "RPATH support - EasyBuild - building software with ease". docs.easybuild.io. Retrieved 2023-10-26.
- ↑ "Singularity | Department of Computer Science Computing Guide". csguide.cs.princeton.edu. Retrieved 2023-10-26.
- ↑ "Overview — Charliecloud 0.34 documentation". hpc.github.io. Retrieved 2023-10-26.
- ↑ "Docker overview". Docker Documentation. 2023-09-08. Retrieved 2023-10-26.
- ↑ 12.0 12.1 "US DOE Exascale Computing Project Announced Availability of the Extreme-Scale Scientific Software Stack (E4S) v1.2". HPCwire. Retrieved 2023-10-26.
- ↑ "Shifter Containers — Nextflow 21.10.6 documentation". www.nextflow.io. Retrieved 2023-10-26.
- ↑ "The Extreme-Scale Scientific Software Stack (e4s) and Its Promise for. ." (PDF). November 19, 2019. Retrieved March 30, 2023.
- ↑ "C++ Introduction". www.w3schools.com. Retrieved 2023-10-26.
- ↑ "Kokkos Ecosystem – Part of the Exascale Project". Retrieved 2023-10-26.
- ↑ 17.0 17.1 "E4S: Extreme-scale Scientific Software Stack | Resources | Research IT". researchit.illinois.edu. Retrieved 2023-10-26.
- ↑ "Welcome to Python.org". Python.org. 2023-10-20. Retrieved 2023-10-26.
- ↑ "PyTorch". www.pytorch.org. Retrieved 2023-10-26.
- ↑ "TensorFlow". TensorFlow. Retrieved 2023-10-26.
- ↑ Robinson, Dan. "Extreme-scale Scientific Software Stack hits v22.02". www.theregister.com. Retrieved 2023-10-26.
- ↑ "The Fortran Programming Language — Fortran Programming Language". fortran-lang.org. Retrieved 2023-10-26.
- ↑ Menon, Harshitha; Parasyris, Konstantinos; Scogland, Tom; Gamblin, Todd (2022). "Searching for high-fidelity builds using active learning". Proceedings of the 19th International Conference on Mining Software Repositories. pp. 179–190. doi:10.1145/3524842.3528464. ISBN 9781450393034. Retrieved 2023-11-08. Unknown parameter
|s2cid=ignored (help) Search this book on
- ↑ "PETSc 3.20 — PETSc 3.20.0 documentation". petsc.org. Retrieved 2023-10-26.
- ↑ Menon, H. (February 10, 2022). "ReliaBuild: Searching for High-Fidelity Builds using Active Learning". arXiv:2202.05223 [cs.SE].
- ↑ "Conan 2.0: C and C++ Open Source Package Manager". conan.io. Retrieved 2023-10-26.
- ↑ "MATLAB". www.mathworks.com. Retrieved 2023-10-26.
- ↑ "Apache License, Version 2.0". Retrieved April 24, 2023.
- ↑ "The 3-Clause BSD License". Open Source Initiative. 2011-05-22. Retrieved 2023-10-26.
- ↑ "GNU Lesser General Public License v3.0 - GNU Project - Free Software Foundation". www.gnu.org. Retrieved 2023-10-26.
- ↑ "Accelerate Fast Math with Intel® oneAPI Math Kernel Library". Intel. Retrieved 2023-10-26.
- ↑ "BLAS (Basic Linear Algebra Subprograms)". netlib.org. Retrieved 2023-10-26.
- ↑ "LAPACK — Linear Algebra PACKage". netlib.org. Retrieved 2023-10-26.
- ↑ "Open MPI: Open Source High Performance Computing". www.open-mpi.org. Retrieved 2023-10-26.
- ↑ "What is HPC? Introduction to high-performance computing | IBM". www.ibm.com. Retrieved 2023-10-26.
- ↑ "DOE Explains...Exascale Computing". Energy.gov. Retrieved 2023-10-26.
- ↑ "Sandia National Laboratories: Exceptional Service in the National Interest". Sandia National Laboratories. Retrieved 2023-10-26.
- ↑ Nicolas Dao. "What you need to know to choose an open source license". Gist. Retrieved 2023-10-26.
- Siddiqui, Shahzeb; Shende, Sameer (1 March 2022). "Software Deployment Process at NERSC: Deploying the Extreme-scale Scientific Software Stack (E4S) Using Spack at the National Energy Research Scientific Computing Center (NERSC)" (PDF). National Energy Research Scientific Computing Center. Lawrence Berkeley National Laboratory. doi:10.2172/1868332. OSTI 1868332. Retrieved 9 November 2023. Unknown parameter
|s2cid=ignored (help)
- Siddiqui, Shahzeb; Shende, Sameer (1 March 2022). "Software Deployment Process at NERSC: Deploying the Extreme-scale Scientific Software Stack (E4S) Using Spack at the National Energy Research Scientific Computing Center (NERSC)" (PDF). National Energy Research Scientific Computing Center. Lawrence Berkeley National Laboratory. doi:10.2172/1868332. OSTI 1868332. Retrieved 9 November 2023. Unknown parameter
This article "E4S Software Stack" is from Wikipedia. The list of its authors can be seen in its historical and/or the page Edithistory:E4S Software Stack. Articles copied from Draft Namespace on Wikipedia could be seen on the Draft Namespace of Wikipedia and not main one.
