Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GCR solver rewrite #1239

Merged
merged 33 commits into from
Mar 25, 2023
Merged

GCR solver rewrite #1239

merged 33 commits into from
Mar 25, 2023

Conversation

yanjen
Copy link
Collaborator

@yanjen yanjen commented Dec 12, 2022

This pull request finishes #1180 GCR solver.

@yanjen yanjen added type:solver This is related to the solvers is:duplicate This issue or pull request already exists. mod:reference This is related to the reference module. 1:ST:WIP This PR is a work in progress. Not ready for review. 1:ST:low-importance This issue/PR is not that important and can be ignored for now. labels Dec 13, 2022
@yanjen yanjen self-assigned this Dec 13, 2022
@MarcelKoch
Copy link
Member

I just gave it a quick look, and I think it should be easy to extend it to distributed types. Do you want to work on that? Alternatively, I could do that in a separate PR later on.

@yanjen
Copy link
Collaborator Author

yanjen commented Dec 13, 2022

Sounds like a good idea. I also want to get more familiar with the distributed stuff in Ginkgo. Since GCR is not an urgent implementation, I would like to try implement it myself. I would also prefer to do a separate PR for that.

@yanjen
Copy link
Collaborator Author

yanjen commented Dec 16, 2022

format!

@yanjen yanjen added 1:ST:ready-for-review This PR is ready for review and removed 1:ST:WIP This PR is a work in progress. Not ready for review. labels Dec 19, 2022
@yhmtsai
Copy link
Member

yhmtsai commented Dec 21, 2022

Could you also provide the algorithm reference?

@yanjen
Copy link
Collaborator Author

yanjen commented Dec 22, 2022

Here is a reference for the GCR algorithm. Iterative Methods for Sparse Linear Systems Second Edition by Saad, Yousef
The GCR algorithm is at page 205.

Copy link
Member

@yhmtsai yhmtsai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also need to add the corresponding type into test/solver/solver.cpp and test_install

common/unified/solver/gcr_kernels.cpp Show resolved Hide resolved
core/solver/gcr.cpp Show resolved Hide resolved
core/solver/gcr.cpp Show resolved Hide resolved
core/solver/gcr_kernels.hpp Show resolved Hide resolved
core/solver/gcr_kernels.hpp Outdated Show resolved Hide resolved
core/test/solver/gcr.cpp Outdated Show resolved Hide resolved
include/ginkgo/core/solver/gcr.hpp Outdated Show resolved Hide resolved
include/ginkgo/core/solver/gmres.hpp Outdated Show resolved Hide resolved
reference/solver/gcr_kernels.cpp Outdated Show resolved Hide resolved
reference/solver/gcr_kernels.cpp Outdated Show resolved Hide resolved
core/solver/gcr.cpp Outdated Show resolved Hide resolved
@yanjen yanjen requested a review from yhmtsai January 6, 2023 05:31
@yanjen yanjen force-pushed the gcr_restart branch 2 times, most recently from a57b15f to 1face54 Compare January 10, 2023 16:16
@yanjen
Copy link
Collaborator Author

yanjen commented Jan 10, 2023

format!

@upsj upsj requested a review from vasilisge0 February 6, 2023 13:26
Copy link
Collaborator

@greole greole left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just some minor comments

benchmark/solver/solver.cpp Outdated Show resolved Hide resolved
benchmark/solver/solver.cpp Outdated Show resolved Hide resolved
Copy link
Member

@yhmtsai yhmtsai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM in general. the computation of the memory might need to revisit again.

core/matrix/dense.cpp Outdated Show resolved Hide resolved
include/ginkgo/core/matrix/dense.hpp Show resolved Hide resolved
include/ginkgo/core/solver/gcr.hpp Show resolved Hide resolved
test/solver/gmres_kernels.cpp Show resolved Hide resolved
core/solver/gcr.cpp Outdated Show resolved Hide resolved
@yanjen
Copy link
Collaborator Author

yanjen commented Mar 7, 2023

format!

Copy link
Collaborator

@greole greole left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some more quick comments. Besides that LGTM

common/unified/solver/gcr_kernels.cpp Outdated Show resolved Hide resolved
core/solver/gcr.cpp Outdated Show resolved Hide resolved
core/solver/gcr.cpp Outdated Show resolved Hide resolved
common/unified/solver/gcr_kernels.cpp Outdated Show resolved Hide resolved
core/solver/gcr.cpp Outdated Show resolved Hide resolved
core/solver/gcr.cpp Show resolved Hide resolved
@ginkgo-bot
Copy link
Member

Note: This PR changes the Ginkgo ABI:

Functions changes summary: 0 Removed, 34 Changed (25109 filtered out), 857 Added functions
Variables changes summary: 0 Removed, 0 Changed, 56 Added variables

For details check the full ABI diff under Artifacts here

@sonarcloud
Copy link

sonarcloud bot commented Mar 24, 2023

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 41 Code Smells

85.9% 85.9% Coverage
4.2% 4.2% Duplication

@yanjen yanjen merged commit 30dcadf into develop Mar 25, 2023
@yanjen yanjen deleted the gcr_restart branch March 25, 2023 08:50
tcojean added a commit that referenced this pull request Jun 16, 2023
Release 1.6.0 of Ginkgo.

The Ginkgo team is proud to announce the new Ginkgo minor release 1.6.0. This release brings new features such as:
- Several building blocks for GPU-resident sparse direct solvers like symbolic
  and numerical LU and Cholesky factorization, ...,
- A distributed Schwarz preconditioner,
- New FGMRES and GCR solvers,
- Distributed benchmarks for the SpMV operation, solvers, ...
- Support for non-default streams in the CUDA and HIP backends,
- Mixed precision support for the CSR SpMV,
- A new profiling logger which integrates with NVTX, ROCTX, TAU and VTune to
  provide internal Ginkgo knowledge to most HPC profilers!

and much more.

If you face an issue, please first check our [known issues page](https://github.com/ginkgo-project/ginkgo/wiki/Known-Issues) and the [open issues list](https://github.com/ginkgo-project/ginkgo/issues) and if you do not find a solution, feel free to [open a new issue](https://github.com/ginkgo-project/ginkgo/issues/new/choose) or ask a question using the [github discussions](https://github.com/ginkgo-project/ginkgo/discussions).

Supported systems and requirements:
+ For all platforms, CMake 3.13+
+ C++14 compliant compiler
+ Linux and macOS
  + GCC: 5.5+
  + clang: 3.9+
  + Intel compiler: 2018+
  + Apple Clang: 14.0 is tested. Earlier versions might also work.
  + NVHPC: 22.7+
  + Cray Compiler: 14.0.1+
  + CUDA module: CUDA 9.2+ or NVHPC 22.7+
  + HIP module: ROCm 4.5+
  + DPC++ module: Intel OneAPI 2021.3+ with oneMKL and oneDPL. Set the CXX compiler to `dpcpp`.
+ Windows
  + MinGW: GCC 5.5+
  + Microsoft Visual Studio: VS 2019+
  + CUDA module: CUDA 9.2+, Microsoft Visual Studio
  + OpenMP module: MinGW.

### Version Support Changes
+ ROCm 4.0+ -> 4.5+ after [#1303](#1303)
+ Removed Cygwin pipeline and support [#1283](#1283)

### Interface Changes
+ Due to internal changes, `ConcreteExecutor::run` will now always throw if the corresponding module for the `ConcreteExecutor` is not build [#1234](#1234)
+ The constructor of `experimental::distributed::Vector` was changed to only accept local vectors as `std::unique_ptr` [#1284](#1284)
+ The default parameters for the `solver::MultiGrid` were improved. In particular, the smoother defaults to one iteration of `Ir` with `Jacobi` preconditioner, and the coarse grid solver uses the new direct solver with LU factorization. [#1291](#1291) [#1327](#1327)
+ The `iteration_complete` event gained a more expressive overload with additional parameters, the old overloads were deprecated. [#1288](#1288) [#1327](#1327)

### Deprecations
+ Deprecated less expressive `iteration_complete` event. Users are advised to now implement the function `void iteration_complete(const LinOp* solver, const LinOp* b, const LinOp* x, const size_type& it, const LinOp* r, const LinOp* tau, const LinOp* implicit_tau_sq, const array<stopping_status>* status, bool stopped)` [#1288](#1288)

### Added Features
+ A distributed Schwarz preconditioner. [#1248](#1248)
+ A GCR solver [#1239](#1239)
+ Flexible Gmres solver [#1244](#1244)
+ Enable Gmres solver for distributed matrices and vectors [#1201](#1201)
+ An example that uses Kokkos to assemble the system matrix [#1216](#1216)
+ A symbolic LU factorization allowing the `gko::experimental::factorization::Lu` and `gko::experimental::solver::Direct` classes to be used for matrices with non-symmetric sparsity pattern [#1210](#1210)
+ A numerical Cholesky factorization [#1215](#1215)
+ Symbolic factorizations in host-side operations are now wrapped in a host-side `Operation` to make their execution visible to loggers. This means that profiling loggers and benchmarks are no longer missing a separate entry for their runtime [#1232](#1232)
+ Symbolic factorization benchmark [#1302](#1302)
+ The `ProfilerHook` logger allows annotating the Ginkgo execution (apply, operations, ...) for profiling frameworks like NVTX, ROCTX and TAU. [#1055](#1055)
+ `ProfilerHook::created_(nested_)summary` allows the generation of a lightweight runtime profile over all Ginkgo functions written to a user-defined stream [#1270](#1270) for both host and device timing functionality [#1313](#1313)
+ It is now possible to enable host buffers for MPI communications at runtime even if the compile option `GINKGO_FORCE_GPU_AWARE_MPI` is set. [#1228](#1228)
+ A stencil matrices generator (5-pt, 7-pt, 9-pt, and 27-pt) for benchmarks [#1204](#1204)
+ Distributed benchmarks (multi-vector blas, SpMV, solver) [#1204](#1204)
+ Benchmarks for CSR sorting and lookup [#1219](#1219)
+ A timer for MPI benchmarks that reports the longest time [#1217](#1217)
+ A `timer_method=min|max|average|median` flag for benchmark timing summary [#1294](#1294)
+ Support for non-default streams in CUDA and HIP executors [#1236](#1236)
+ METIS integration for nested dissection reordering [#1296](#1296)
+ SuiteSparse AMD integration for fillin-reducing reordering [#1328](#1328)
+ Csr mixed-precision SpMV support [#1319](#1319)
+ A `with_loggers` function for all `Factory` parameters [#1337](#1337)

### Improvements
+ Improve naming of kernel operations for loggers [#1277](#1277)
+ Annotate solver iterations in `ProfilerHook` [#1290](#1290)
+ Allow using the profiler hooks and inline input strings in benchmarks [#1342](#1342)
+ Allow passing smart pointers in place of raw pointers to most matrix functions. This means that things like `vec->compute_norm2(x.get())` or `vec->compute_norm2(lend(x))` can be simplified to `vec->compute_norm2(x)` [#1279](#1279) [#1261](#1261)
+ Catch overflows in prefix sum operations, which makes Ginkgo's operations much less likely to crash. This also improves the performance of the prefix sum kernel [#1303](#1303)
+ Make the installed GinkgoConfig.cmake file relocatable and follow more best practices [#1325](#1325)

### Fixes
+ Fix OpenMPI version check [#1200](#1200)
+ Fix the mpi cxx type binding by c binding [#1306](#1306)
+ Fix runtime failures for one-sided MPI wrapper functions observed on some OpenMPI versions [#1249](#1249)
+ Disable thread pinning with GPU executors due to poor performance [#1230](#1230)
+ Fix hwloc version detection [#1266](#1266)
+ Fix PAPI detection in non-implicit include directories [#1268](#1268)
+ Fix PAPI support for newer PAPI versions: [#1321](#1321)
+ Fix pkg-config file generation for library paths outside prefix [#1271](#1271)
+ Fix various build failures with ROCm 5.4, CUDA 12, and OneAPI 6 [#1214](#1214), [#1235](#1235), [#1251](#1251)
+ Fix incorrect read for skew-symmetric MatrixMarket files with explicit diagonal entries [#1272](#1272)
+ Fix handling of missing diagonal entries in symbolic factorizations [#1263](#1263)
+ Fix segmentation fault in benchmark matrix construction [#1299](#1299)
+ Fix the stencil matrix creation for benchmarking [#1305](#1305)
+ Fix the additional residual check in IR [#1307](#1307)
+ Fix the cuSPARSE CSR SpMM issue on single strided vector when cuda >= 11.6 [#1322](#1322) [#1331](#1331)
+ Fix Isai generation for large sparsity powers [#1327](#1327)
+ Fix Ginkgo compilation and test with NVHPC >= 22.7 [#1331](#1331)
+ Fix Ginkgo compilation of 32 bit binaries with MSVC [#1349](#1349)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
1:ST:low-importance This issue/PR is not that important and can be ignored for now. 1:ST:ready-to-merge This PR is ready to merge. is:duplicate This issue or pull request already exists. mod:reference This is related to the reference module. type:solver This is related to the solvers
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants