feat(segmentation export): add new cornerstone3D segmentation export adapter and optimize SEG read #692

sedghi · 2023-07-14T21:06:04Z

Context

This PR adds DICOM SEG export for cornerstone3D. It also uses the old importer and optimize and improve upon it for parsing DICOM SEG resulting in 3x faster read time for DICOM SEG.

Changes & Results

modifies the DICOM SEG export in adapters

Testing

Export

Should be able to run the yarn run example segmentationExport and export a SEG

Parsing

Test should be done later in OHIF when I create the PR

Checklist

PR

[] My Pull Request title is descriptive, accurate and follows the
semantic-release format and guidelines.

Code

[] My code has been well-documented (function documentation, inline comments,
etc.)
[] I have run the yarn build:update-api to update the API documentation, and have
committed the changes to this PR. (Read more here https://www.cornerstonejs.org/docs/contribute/update-api)

Public Documentation Updates

[] The documentation page has been updated as necessary for any public API
additions or removals.

Tested Environment

[] "OS:
[] "Node version:
[] "Browser:

netlify · 2023-07-14T21:06:10Z

✅ Deploy Preview for cornerstone-3d-docs ready!

Name	Link
🔨 Latest commit	`5ae85c7`
🔍 Latest deploy log	https://app.netlify.com/sites/cornerstone-3d-docs/deploys/64c3e0466a43590008c6d42b
😎 Deploy Preview	https://deploy-preview-692--cornerstone-3d-docs.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

…at/dicom-seg-adapter

rogerbramon · 2023-07-21T14:20:25Z

packages/adapters/examples/segmentationExport/index.ts

+        const volumeImages = volume.convertToCornerstoneImages();
+        const imagePromises = volumeImages.map(image => image.promise);
+
+        await Promise.all(imagePromises).then(images => {


Is PixelData needed to generate the segmentation, or metadata would be enough?

I believe dcmjs only need metadata to create a derived multiframe SEG from the reference

I checked it seems like it needs the referenceDataSet.PixelData

rogerbramon · 2023-07-21T17:18:53Z

packages/adapters/src/adapters/Cornerstone3D/Segmentation/generateLabelMaps2DFrom3D.ts

+            segmentsOnLabelmap3D.add(segmentIndex);
+        });
+
+        labelmaps2D[dimensions[2] - z] = labelmap2D;


Should that be dimensions[2] - 1 - z?

Is this generating label maps that are frame indexed starting with 1? If so, that fact should be in a comment here and should be noted up top.

packages/adapters/src/adapters/Cornerstone/Segmentation_4X.js

packages/adapters/src/adapters/helpers/codeMeaningEquals.ts

packages/adapters/src/adapters/helpers/graphicTypeEquals.ts

packages/adapters/src/adapters/helpers/toArray.ts

packages/adapters/src/index.ts

packages/dicomImageLoader/src/imageLoader/getDatasetModule.ts

wayfarer3130 · 2023-07-25T11:52:31Z

packages/dicomImageLoader/src/imageLoader/wadors/metaData/metaDataProvider.ts

+
+  // This is used for gathering all the metadata for export
+  if (type === 'dataset') {
+    const types = [


Can you externalize this list of types, and add a comment on it? It definitely is NOT all the types, but there doesn't seem to be a safe way to get all the actual types available.

packages/dicomImageLoader/src/imageLoader/wadors/metaData/metaDataProvider.ts

wayfarer3130 · 2023-07-25T12:02:29Z

packages/adapters/src/adapters/Cornerstone3D/Segmentation/generateSegmentation.ts

+/**
+ * _createMultiframeSegmentationFromReferencedImages - description
+ *
+ * @param images - An array of the cornerstone image objects.


Could you add a bit more information here on what the images are - I think given the context that the images are the segmentation images, and are thus single bit image objects containing the actual data in the segmentation, but it isn't at all obvious to me externally to this function that the images are just the segmentation single bit images.

Looking at the dcmjs side, these are the original images, and in fact the generation code for the derivation throws away all the pixel data itself, using only the length. I suggest not passing in the actual pixel data, but generating a dummy buffer containing just the byte lengths as required. That way, even the temporary buffer will have the right lengths, as the generated buffer is simply a copy of the derived dataset buffers, and that is usually twice the required length.

we already have the pixelData from the volume. So the cost is negligible compared to creating another temporary buffer etc.

packages/dicomImageLoader/src/imageLoader/wadouri/metaData/metaDataProvider.ts

packages/adapters/src/adapters/Cornerstone3D/Segmentation/generateLabelMaps2DFrom3D.ts

packages/adapters/src/adapters/Cornerstone/Segmentation_4X.js

wayfarer3130 · 2023-07-25T12:36:51Z

packages/adapters/src/adapters/Cornerstone/Segmentation_4X.js

@@ -1541,3 +1660,109 @@ function getSegmentMetadata(multiframe, seriesInstanceUid) {
        data
    };
 }
+
+function readFromUnpackedChunks(chunks, offset, length) {


Consider using the code from StaticDICOMweb that has all this functionality, exposing it as though it were a single Uint8Array object - that is asyncIterableToBuffer. That allows reading very large buffers and automatically splitting them as required. I think we will need that type of functionality in the future for better handling of streaming results so it would be nice to include. I could add it into dcmjs if that is where we think it belongs.

yeah I think it belongs to dcmjs

wayfarer3130 · 2023-07-25T12:38:33Z

packages/adapters/src/adapters/Cornerstone/Segmentation_4X.js

+    };
+}
+
+function calculateCentroid(imageIdIndexBufferIndex, multiframe) {


It feels to me like some of these should be added to a helper/utils export rather than being directly included here, as they may well be useful in other segmentation generations in the future, and also that allows explaining what they are doing cleanly.

it is really not a general utility as it is relying on imageIdIndexBufferIndex

wayfarer3130

Some clean up, but definitely looks good.

…nerstone3D into feat/dicom-seg-adapter

…at/dicom-seg-adapter

rogerbramon · 2023-07-26T06:00:03Z

packages/dicomImageLoader/src/imageLoader/wadouri/metaData/metaDataProvider.ts

@@ -245,6 +249,10 @@ function metaDataProvider(type, imageId) {
      actualFrameDuration: dicomParser.intString(dataSet.string('x00181242')),
    };
  }
+
+  if (type === 'instance') {
+    return getInstanceModule(imageId, metaDataProvider, instanceModuleNames);


Not sure if it would make sense to use the naturalizeDataset or namifyDataset methods of dcmjs DicomMetaDictionary to get the full metadata of the instance that already match the dcmjs naming.

the problem is that we don't have the instance stored anywhere to naturalize or namify it.

…at/dicom-seg-adapter

…adapter (#692) * chore(adapters): ability to cache and build ESM modules * wip * refactoring * remove bound ijk which is unnecessary since we moved to world * add data set module to the metadata providers * add a new method to segmentation state * add the new parser * work without rle encode * fix demo for export color * add centroid calculations * add trigger event for segmentation load progress * update dcmjs * update api * fix example * remove the promise await * apply review comments * add adapters section to examples * add adapters dep to docs * add eslint rule * example deploy

…#702) * fix(loader): Load colour images correctly when specified Float32Array * feat(streamingVolumeLoader): added IMAGE_VOLUME_LOADING_COMPLETED event (#699) * added IMAGE_VOLUME_LOADING_COMPLETED event * api doc * chore(release): publish [skip ci] * chore(script): add a dev script which should be used for linking (#706) * chore(release): publish [skip ci] * fix(SVGCursorDescriptor): improve CursorSVG typing (#705) * fix: improve CursorSVG typing * chore: update api --------- Co-authored-by: Carsten Walther <carsten.walther@mevis.fraunhofer.de> * chore(release): publish [skip ci] * feat(segmentation export): add new cornerstone3D segmentation export adapter (#692) * chore(adapters): ability to cache and build ESM modules * wip * refactoring * remove bound ijk which is unnecessary since we moved to world * add data set module to the metadata providers * add a new method to segmentation state * add the new parser * work without rle encode * fix demo for export color * add centroid calculations * add trigger event for segmentation load progress * update dcmjs * update api * fix example * remove the promise await * apply review comments * add adapters section to examples * add adapters dep to docs * add eslint rule * example deploy * chore(release): publish [skip ci] * fix(voi): fix the voi setting when the stack is composed of different orientations (#703) * fix(voi): fix the voi setting when the stack is composed of different orientations * fix the volume conversion to stack voi bug * fix(volumes): a volume should be able to grab data from another cached volume when needed * revert example change * chore(release): publish [skip ci] * feat(voiSync): add optoins to turn of invert sync for voisync (#708) * chore(release): publish [skip ci] * PR requested changes

sedghi added 8 commits May 3, 2023 21:03

chore(adapters): ability to cache and build ESM modules

eaeefe7

wip

101e778

update yarn lock

5060e3b

refactoring

7dc9c67

remove bound ijk which is unnecessary since we moved to world

b25c911

add data set module to the metadata providers

f310dec

add a new method to segmentation state

2750e8d

add the new parser

454da6e

sedghi changed the title ~~chore(adapters): ability to cache and build ESM modules~~ feat(segmentation export): add new cornerstone3D segmentation export adapter Jul 14, 2023

sedghi added 7 commits July 17, 2023 08:32

work without rle encode

9d4ad01

fix demo for export color

27b0c19

add centroid calculations

8917dd0

add trigger event for segmentation load progress

2f6655f

update dcmjs

936d9ac

update api

c153d0a

Merge branch 'main' of github.com:cornerstonejs/cornerstone3D into fe…

87c1e6d

…at/dicom-seg-adapter

sedghi requested review from wayfarer3130 and jbocce July 19, 2023 21:36

fix example

b6531db

rogerbramon reviewed Jul 21, 2023

View reviewed changes

wayfarer3130 reviewed Jul 24, 2023

View reviewed changes

packages/adapters/src/adapters/Cornerstone/Segmentation_4X.js Outdated Show resolved Hide resolved