Pixelpipe safety improvements and maintenance #17718
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
1. Avoid double reports for pipe starting 2. refactor dt_print_mem_usage(), it now gets a message string passed avoiding am extra dt_print() call 3. log memory consumption also after pixelpipe has ended 4. log pixelpipe cache checks also with DT_DEBUG_MEMORY 5. dt_tiling_piece_fits_host_memory() gets the piece as an additional parameter. This allows it to check for pipe type and more.
size_t dt_get_available_pipe_mem(const dt_dev_pixelpipe_t *pipe); returns dt_get_available_mem() reduced in case the pixelpipe is DT_DEV_PIXELPIPE_THUMBNAIL to allow safer background thumbnail generation. In case of low-memory systems (8GB or so) this will prevent crashes due to OOM killer or too high memory stress. All checks within tiling code or tests for CPU tiling-required now use this variant instead of full dt_get_available_mem() ASC
Resulting file when writing CPU vs GPU difference could be wrong due to wrong kernel parameters for dt_opencl_read_host_from_device().
jenshannoschwalm
added
bugfix
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Inspired by #17684 (provided logs explaining the issue) and various reports like #17016 #15939 #13464
As we don't keep track of allocated memory we can at least make sure the background thumbnail pipes don't take too much memory. For sure this is not a "complete" fix but it will certainly help to restrict memory consumption for those.
Docs from commit messages
Pixelpipe memory log maintenance
am extra dt_print() call
This allows it to check for pipe type and more.
Introduce dt_get_available_pipe_mem()
size_t dt_get_available_pipe_mem(const dt_dev_pixelpipe_t *pipe);
returns dt_get_available_mem() reduced in case the pixelpipe is DT_DEV_PIXELPIPE_THUMBNAIL
to allow safer background thumbnail generation.
In case of low-memory systems (8GB or so) this will prevent crashes due to OOM killer or
too high memory stress.
All checks within tiling code or tests for CPU tiling-required now use this variant instead of
full dt_get_available_mem()
Fix OpenCL parameters when writing PFM diff files
Resulting file when writing CPU vs GPU difference could be wrong due to wrong kernel parameters
for dt_opencl_read_host_from_device().