Skip to content

Commit 206a71c

Browse files
Implement a segmentation and rastermask UI frontend module
**Note 1:** we want to generate rastermasks based on **full** image data or from some external source that can be used in all other modules as usual. Examples for this would be AI segmentation algorithms, external mask files or other tools based on whole image data. As those algorithms can be very performance costly we would like to do the algorithm just once based on full image data for all pipes. To make this compliant with darktable's roi strategy without a significant performance drop this module must be very early in the pipe and requires some care as we want to support all image types. The segmentation is either done while developing in darkroom fullpipe or while exporting and keep results in per instance module->data for later usage. The results are validated via the `dt_dev_pixelpipe_piece_hash()` including model versioning, if we find a differing hash we do the segmentation algorithm again and refresh the preview pipes. Please note, all read & write to segmentation data must be protected via a mutex as all pipes will share this. ____________________________________________________________________________________________________________________ **Note 2:** the resulting rastermask is calculated from a selection of segment maps, the maximum number of possible segments is SEGMAP_MAXSEGMENTS. For all image locations we have data in segment maps, the number of generated segment maps depends on the algorithm. AI algorithms might do a segmentation - here the maps for each segment can overlap - providing multiple segment maps. Other algorithms might provide just one map or 3, maybe for each RGB channel. To keep memory consumption within limits we - keep segment map information in uint8_t maps - possibly save & restore maps in lower resolution and do a bilinear interpolation before they get transformed via the module->distort_mask() functions to the final rastermask. - When keeping maps in lower resolutions a model might provide a post_process function called when providing the rastermask, the defaults is a slight gaussian blur. ____________________________________________________________________________________________________________________ **Note 3:** the generated rastermask is combined from a list of selected segment maps. The module provides the user interface to select/deselect maps for the combined list. Whenever the module has focus we are in UI visualizing mode showing a false color representation on a dark grayscale image background. A pixel is - *brightened* if it is in any segment map. - *red* if it belongs to the segment under the mouse - *green* if it is included in the combined raster map list. - *yellow* if belongs to the segment under the mouse and that segment is included in the combined list. We can select/deselect segments from the combined list via the mouse, - a left click *adds* the segment under the mouse to the combined segments - a right click *removes* it - if combined with shift a click adds/removes **all** segments to/from the combined segments A left-mouse double-click de-focus the module for convenience. ____________________________________________________________________________________________________________________ **Note 4:** the segment maps are kept in a dt_segmentation_t struct so we can save/read all data via files (after agreeing how/where) to keep edits after possibly changing a segmentation model.
1 parent 2890431 commit 206a71c

File tree

4 files changed

+943
-1
lines changed

4 files changed

+943
-1
lines changed

src/common/iop_order.c

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
/*
22
This file is part of darktable,
3-
Copyright (C) 2018-2024 darktable developers.
3+
Copyright (C) 2018-2025 darktable developers.
44
55
darktable is free software: you can redistribute it and/or modify
66
it under the terms of the GNU General Public License as published by
@@ -82,6 +82,7 @@ const dt_iop_order_entry_t legacy_order[] = {
8282
{ { 1.0f }, "rawprepare", 0},
8383
{ { 2.0f }, "invert", 0},
8484
{ { 3.0f }, "temperature", 0},
85+
{ { 3.5f }, "rastermaps", 0},
8586
{ { 4.0f }, "highlights", 0},
8687
{ { 5.0f }, "cacorrect", 0},
8788
{ { 6.0f }, "hotpixels", 0},
@@ -177,6 +178,7 @@ const dt_iop_order_entry_t v30_order[] = {
177178
{ { 1.0 }, "rawprepare", 0},
178179
{ { 2.0 }, "invert", 0},
179180
{ { 3.0f }, "temperature", 0},
181+
{ { 3.5f }, "rastermaps", 0},
180182
{ { 4.0f }, "highlights", 0},
181183
{ { 5.0f }, "cacorrect", 0},
182184
{ { 6.0f }, "hotpixels", 0},
@@ -293,6 +295,7 @@ const dt_iop_order_entry_t v50_order[] = {
293295
{ { 1.0 }, "rawprepare", 0},
294296
{ { 2.0 }, "invert", 0},
295297
{ { 3.0f }, "temperature", 0},
298+
{ { 3.5f }, "rastermaps", 0},
296299
{ { 4.0f }, "highlights", 0},
297300
{ { 5.0f }, "cacorrect", 0},
298301
{ { 6.0f }, "hotpixels", 0},
@@ -411,6 +414,7 @@ const dt_iop_order_entry_t v30_jpg_order[] = {
411414
{ { 1.0 }, "rawprepare", 0 },
412415
{ { 2.0 }, "invert", 0 },
413416
{ { 3.0f }, "temperature", 0 },
417+
{ { 3.5f }, "rastermaps", 0},
414418
{ { 4.0f }, "highlights", 0 },
415419
{ { 5.0f }, "cacorrect", 0 },
416420
{ { 6.0f }, "hotpixels", 0 },
@@ -530,6 +534,7 @@ const dt_iop_order_entry_t v50_jpg_order[] = {
530534
{ { 1.0 }, "rawprepare", 0 },
531535
{ { 2.0 }, "invert", 0 },
532536
{ { 3.0f }, "temperature", 0 },
537+
{ { 3.5f }, "rastermaps", 0},
533538
{ { 4.0f }, "highlights", 0 },
534539
{ { 5.0f }, "cacorrect", 0 },
535540
{ { 6.0f }, "hotpixels", 0 },
@@ -1173,6 +1178,7 @@ GList *dt_ioppr_get_iop_order_list(const dt_imgid_t imgid,
11731178
_insert_before(iop_order_list, "nlmeans", "blurs");
11741179
_insert_before(iop_order_list, "filmicrgb", "sigmoid");
11751180
_insert_before(iop_order_list, "colorbalancergb", "colorequal");
1181+
_insert_before(iop_order_list, "rastermaps", "highlights");
11761182
}
11771183
}
11781184
else if(version >= DT_IOP_ORDER_LEGACY

src/iop/CMakeLists.txt

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -154,6 +154,7 @@ add_iop(blurs "blurs.c")
154154
add_iop(sigmoid "sigmoid.c")
155155
add_iop(primaries "primaries.c")
156156
add_iop(colorequal "colorequal.c")
157+
add_iop(rastermaps "rastermaps.c")
157158

158159
if(Rsvg2_FOUND)
159160
add_iop(watermark "watermark.c")

0 commit comments

Comments
 (0)