PETSc Users Manual

Argonne National Laboratory

Mathematics and Computer Science Division

---

Prepared by

S. Balay ¹, S. Abhyankar ², M. Adams ³, J. Brown ¹, P. Brune ¹, K. Buschelman ¹, L. Dalcin ⁴, A. Dener ¹, V. Eijkhout ⁶, W. Gropp ¹, D. Karpeyev ¹, D. Kaushik ¹, M. Knepley ¹, D. May ⁷, L. Curfman McInnes ¹, R. Mills ¹, T. Munson ¹, K. Rupp ¹, P. Sanan ⁸, B. Smith ¹, S. Zampini ⁴, H. Zhang ⁵, and H. Zhang ¹

¹Mathematics and Computer Science Division, Argonne National Laboratory

²Electricity Infrastructure and Buildings Division, Pacific Northwest National Laboratory

³Computational Research, Lawrence Berkeley National Laboratory

⁴Extreme Computing Research Center, King Abdullah University of Science and Technology

⁵Computer Science Department, Illinois Institute of Technology

⁶Texas Advanced Computing Center, University of Texas at Austin

⁷Department of Earth Sciences, University of Oxford

⁸Institute of Geophysics, ETH Zurich

---

This work was supported by the Office of Advanced Scientific Computing Research, Office of Science, U.S. Department of Energy, under Contract DE-AC02-06CH11357.

---

• Introduction to PETSc
  • About This Manual
  • Getting Started
    • Suggested Reading
    • Running PETSc Programs
    • Writing PETSc Programs
    • Simple PETSc Examples
    • Profiling Programs
    • Writing Application Codes with PETSc
    • Citing PETSc
    • Directory Structure
• Programming with PETSc
  • Vectors and Parallel Data
    • Creating and Assembling Vectors
    • Basic Vector Operations
    • Indexing and Ordering
    • Structured Grids Using Distributed Arrays
    • Vectors Related to Unstructured Grids
  • Matrices
    • Creating and Assembling Matrices
    • Basic Matrix Operations
    • Matrix-Free Matrices
    • Other Matrix Operations
    • Partitioning
  • KSP: Linear System Solvers
    • Using KSP
    • Solving Successive Linear Systems
    • Krylov Methods
    • Preconditioners
    • Solving Block Matrices
    • Solving Singular Systems
    • Using External Linear Solvers
  • SNES: Nonlinear Solvers
    • Basic SNES Usage
    • The Nonlinear Solvers
    • General Options
    • Inexact Newton-like Methods
    • Matrix-Free Methods
    • Finite Difference Jacobian Approximations
    • Variational Inequalities
    • Nonlinear Preconditioning
  • TS: Scalable ODE and DAE Solvers
    • Basic TS Options
    • DAE Formulations
    • Using Implicit-Explicit (IMEX) Methods
    • GLEE methods
    • Using fully implicit methods
    • Using the Explicit Runge-Kutta timestepper with variable timesteps
    • Special Cases
    • Monitoring and visualizing solutions
    • Error control via variable time-stepping
    • Handling of discontinuities
    • Using TChem from PETSc
    • Using Sundials from PETSc
  • Performing sensitivity analysis
    • Using the discrete adjoint methods
    • Checkpointing
  • Solving Steady-State Problems with Pseudo-Timestepping
  • High Level Support for Multigrid with KSPSetDM() and SNESSetDM()
    • Adaptive Interpolation
  • DMPlex: Unstructured Grids in PETSc
    • Representing Unstructured Grids
    • Data on Unstructured Grids
    • Evaluating Residuals
    • Networks
• Additional Information
  • PETSc for Fortran Users
    • C vs. Fortran Interfaces
    • Sample Fortran Programs
  • Using MATLAB with PETSc
    • Dumping Data for MATLAB
    • Sending Data to an Interactive MATLAB Session
    • Using the MATLAB Compute Engine
  • Profiling
    • Basic Profiling Information
    • Profiling Application Codes
    • Profiling Multiple Sections of Code
    • Restricting Event Logging
    • Interpreting -log_info Output: Informative Messages
    • Time
    • Saving Output to a File
    • Accurate Profiling and Paging Overheads
  • Hints for Performance Tuning
    • Maximizing Memory Bandwidth
    • Performance Pitfalls and Advice
  • Other PETSc Features
    • PETSc on a process subset
    • Runtime Options
    • Viewers: Looking at PETSc Objects
    • Using SAWs with PETSc
    • Debugging
    • Error Handling
    • Numbers
    • Parallel Communication
    • Graphics
    • Emacs Users
    • VS Code Users
    • Vi and Vim Users
    • Eclipse Users
    • Qt Creator Users
    • Visual Studio Users
    • XCode Users (The Apple GUI Development System)
  • Unimportant and Advanced Features of Matrices and Solvers
    • Extracting Submatrices
    • Matrix Factorization
    • Unimportant Details of KSP
    • Unimportant Details of PC
  • Running PETSc Tests
    • Quick start with the tests
    • Understanding test output and more information
  • Acknowledgments