Probable overscanning of sony ILCE-7M4 APS-C cropped RAWs

[koolfy]

Hello, I have noticed 100% of my APS-C cropped images getting out of darktable show lines of corrupted pixels on the right of the images (before rotation corrections)

I have found this interesting thread: https://www.dpreview.com/forums/thread/4637906
This seems to suggest that at least some generations of proprietary tools have struggled in a very similar fashion with this camera body, specifically with APS-C crops.

Visually it looks like we are simply overscaning the image beyond where there is actually information.

What's interesting is in any case we read more information than Sony presents in its in-body JPGs

Here is the in-body JPG for an uncropped picture (I disabled in-body lens distortion correction):

https://ibb.co/qxz5SvP

Same picture, out of darktable 4.4.2 built against rawspeed commit 2f1e314 (17 sept develop HEAD) (I went to the original step in the history stack to remove as much processing as possible)

https://ibb.co/128p7M5

Now the cropped ones:

in-body JPG for APS-C cropped image:

https://ibb.co/LQ9jtxL

And the exact same picture straight out of darktable (same build):

https://ibb.co/BgMpcYt

In all cases, we read much more data around the edges than Sony does, and in the APS-C crop format, we appear to be actively overscanning the right bound on the image (but are fine vertically)

One interesting note, using digikam quickly (libraw), I noticed they suffer from a similar overscan on both the cropped and uncropped file (rawspeed reads correctly uncropped files but libraw overscans them).

I was going to compare pixel counts on the in-body camera and the raw file exports, but I don't trust how different settings in the body might affect pixel counts on the in-body JPGs.

Here is the link to a google drive containing all versions including actual raw .ARW files:

https://drive.google.com/drive/folders/1sSbFRuQuxJi14ZYcVAzrMeM2At-hOtFV?usp=sharing

One even more interesting note is that the google drive preview for APS-C crops seems to suffer from the exact same symptom.
Maybe my camera is causing this at a very low level? But it would be extremely strange that the dpreview thread would outline the exact same issue.

If it's not an incorrect RAW decoding, might it be a commonplace firmware bug that nobody noticed before?

UPDATE: in case it ends up being relevant, I'm using firmware 2.00 from Sony

[LebedevRI]

Yup, not really surprised that this happens.
Given that the crop information is hardcoded
(generally, as offsets to the image size) in an XML,
that is kind-of expected.

A "fix" would be to adjust the specified crop.
A proper fix would be to read it from EXIF.

[kmilos]

Please upload an APS-C crop sample (in all compression modes) to https://raw.pixls.us/

[LebedevRI]

Please upload an APS-C crop sample (in all compression modes) to https://raw.pixls.us/

And please ideally rename the files before upload
to something like APSC_1[24]bit_(lossless_)?_compressed
so it's obvious what is what...

[kmilos]

Looks like the same occurs w/ the 7CM2, which is not surprising, as they share the sensor and the processor...

We really need those APS-C samples uploaded to RPU, both lossless and uncompressed (or lossy) @koolfy

Right now it seems the crop from right for APS-C for 7M4/7CM2 should be 40 instead of 8 for the FF. (You can tweak this by hand in the raw black/white point module by enabling plugins/darkroom/rawprepare/allow_editing_crop=true in your ~/.config/darktablerc)

This is really unlucky, because for 7RM5 (and I guess TBC for 7CR) it was the same value for FF and APS-C. Currently we don't have a way of detecting the APS-C mode for Sony.

[kmilos]

Currently we don't have a way of detecting the APS-C mode for Sony.

OK, we could possibly use the Quality or APS-CSizeCapture Sony MakerNotes, this is from 7RM5 samples:

With that, maybe we can add a hint for a different crop, or a new mode altogether?

Edit: Unfortunately, both of these are in parts of Sony MakerNotes rawspeed doesn't decode/decrypt yet... Quality is the easiest to get to as part of the simple/regular/unencrypted 0x927C MakerNotes IFD rawspeed already parses, just checked that mRootIFD->getEntryRecursive(TiffTag(0x202E)) works.

[koolfy]

I already made suitable pictures in all modes that I think are relevant, I'll upload them tomorrow during the day, sorry for the wait, busy life :)

I'll name them explicitly as well.

I don't think I found in the menus how to change color depth, I think the default is 12 or 14bits, do you think it's relevant to find how to change that and include different color depth raws as well? I don't think so but RPU seemed to suggest so.

edit: I'll have to take them again, because I didn't properly label them and it's been too long to remember the order in which I took which one and what setting I changed at what time -_-

[LebedevRI]

Thank you!
I think 12bit ones were previously being written either in Bulb mode, or in the highest shutter speed mode.

[koolfy]

Sorry for the huge delay, I messed up my methodology several times in a row :)

I just uploaded the following set of raw files to raw.pixls.us, although they don't appear on the repo list yet:

• ILCE-7M4_DSC06673_FullFrame-Raw-Uncompressed.ARW
• ILCE-7M4_DSC06674_FullFrame-LossLess-Compressed-Large.ARW
• ILCE-7M4_DSC06675_FullFrame-LossLess-Compressed-Medium.ARW
• ILCE-7M4_DSC06676_FullFrame-LossLess-Compressed-Small.ARW
• ILCE-7M4_DSC06677_FullFrame-Raw-Compressed.ARW
• ILCE-7M4_DSC06678_APS-C-Raw-Uncompressed.ARW
• ILCE-7M4_DSC06679_APS-C-LossLess-Medium.ARW
• ILCE-7M4_DSC06680_APS-C-LossLess-Small.ARW
• ILCE-7M4_DSC06681_APS-C-Raw-Compressed.ARW

The names should be self-explanatory.

These are all the combinations possible.
Please note that the APS-C-LossLess-Large does not exist, because as I understand it, LossLess-Large/Medium/Small correspond to 33mpx/14mpx/??mpx respectively, and obviously as the APS-C sensor crop produces a 14mpx image, it can only start at 14mpx (medium) and can't come up with a 33mpx version :)

Here is a table of what I notice with each picture:

format	can be read	crop corruption on the right
FF raw uncompressed	yes	no
FF LossLess-Large	yes	no
FF LossLess-Medium	no	n/a
FF LossLess-Small	no	n/a
FF raw compressed (lossy)	yes	no
APS-C raw uncompressed	yes	yes
APS-C LossLess-Medium	yes	no
APS-C LossLess-Small	no	n/a
APS-C raw compressed (lossy)	yes	yes

What's interesting to me is that we "correctly" crop APS-C LossLess-Medium, but incorrectly crop APS-C raw-uncompressed and raw-compressed (lossy). I know that LossLess decoding was introduced recently, so it makes sense that it would crop differently. It at least seems to be closer to the "correct" crop than the other two.

Again, "correct" may not mean "100% pixels shown", as it seems even the camera-jpg throw away some pixels around the borders.

The ideal solution would be to find a metadata field to follow and crop accordingly, the least-effort solution would probably be to find out whatever crop value we use for the LossLess-medium APS-C crop and use it for APS-C raw-uncompressed and APS-C-compressed(lossy)

It is also interesting that we support LossLess-Medium with the APS-C crop, but can't open the LossLess-Medium in FullFrame crop.

[LebedevRI]

Sorry for the huge delay, I messed up my methodology several times in a row :)

I just uploaded the following set of raw files to raw.pixls.us, although they don't appear on the repo list yet:

• ILCE-7M4_DSC06673_FullFrame-Raw-Uncompressed.ARW
• ILCE-7M4_DSC06674_FullFrame-LossLess-Compressed-Large.ARW
• ILCE-7M4_DSC06675_FullFrame-LossLess-Compressed-Medium.ARW
• ILCE-7M4_DSC06676_FullFrame-LossLess-Compressed-Small.ARW
• ILCE-7M4_DSC06677_FullFrame-Raw-Compressed.ARW
• ILCE-7M4_DSC06678_APS-C-Raw-Uncompressed.ARW
• ILCE-7M4_DSC06679_APS-C-LossLess-Medium.ARW
• ILCE-7M4_DSC06680_APS-C-LossLess-Small.ARW
• ILCE-7M4_DSC06681_APS-C-Raw-Compressed.ARW

@koolfy thank you very much for the contribution! Verified uploads (replacing the existing set.)

[kmilos]

Thank you @koolfy

So it's confirmed the 7M4 APS-C is a bit of an exception unfortunately, and needs -40 in the lossy/uncompressed mode vs -8 for FF. This is done by comparing TIFF ImageWidth/ImageHeight for lossy/uncompressed vs "true" SonyRawImageSize active area (which is also confirmed empirically):

For the 7RM5 everything works out as the difference in FF and APS-C is identical.

Lossless is ok in any mode as we ignore XML crop and use the "true" SonyRawImageSize active area directly. Unfortunately this tag is not available in the older lossy/uncompressed file formats.

So, it looks like we either need a new hint/mode to accommodate for this, or break backwards compatibility and forget the "supplies biggest possible images" premise and just go w/ the smaller, manufacturer embedded Exif crop which seems to be always available in Sony ARWs.

Thanks Sony. 😕

[koolfy]

In the meantime, is there a XML value somewhere I can edit to temporarily get a "proper" crop on those specific files?

I see https://github.com/darktable-org/rawspeed/blob/develop/data/cameras.xml#L13830 but it doesn't seem to target a specific format or compression… maybe that's what you're referring to as a new hint/mode?

I'm not in a position to have an opinion on the proper mid/longterm fix, and am simply looking for a quick hack for the pictures I'm processing in the meantime.

I'm okay with turning it into an acceptable PR too if that's helpful and within my skills.

[LebedevRI]

Regardless of the particular solution chosen for this problem,
i'm pretty sure we should start reading crop from the metadata,
always, even if we will then override it by XML metadata.
(I.e. what #490 did for Panasonic)

[kmilos]

@koolfy You basically have two workaround options for the moment, and you can choose one depending on whether you have more FF or APS-C raws (note again that these apply only to uncompressed/lossy, and for lossless you don't have to do anything):

1. (preferred IMHO) Enable crop settings (set plugins/darkroom/rawprepare/allow_editing_crop=true in your ~/.config/darktable/darktablerc), and then in "raw black/white point" module bump the crop right slider from 8 to 40; note that you'll have to do this for every single APS-C image (you can create a preset though), and/or

2. Edit your cameras.xml and change the -8 to -40 in the 7M4/7CM2 section; note that you'll then have to restore to 8 by method 1 for individual FF images (and also remember to edit cameras.xml again if you update dt).