Mismatch lens detection with Canon RF 24-105 versions (STM / USM) in darktable

[GitHubbero]

I apologize, if the issue is already on GitHub, but i couldn't find anything.

While working with darktable, i saw it detects my lens wrong.
I used for the photo the Canon RF 24-105 F4-7.1 IS STM
but darktable detects it as Canon RF 24-105 F4L IS USM

I am sorry, if the problem is minor, but i can't work properly with exiv2 to check it myself. So i thought the problem of the lens detection have to be from exiv2 and its tags, because the lens in the exif-data is correctly (checked with exiftool and DPP4).
Hope you could help me. And if there is something unclear, text me.
Thank you all.

[postscript-dev]

Looking at the source code, there is an entry for:

61182, "Canon RF 24-105mm F4L IS USM"

If Exiv2 incorrectly identifies a lens then there are two solutions:

1. You can create an Exiv2 lens configuration file on your computer, which will override any existing value for that lens ID (i.e., 61182).
2. Add the lens data into Exiv2 - which has a global effect. To do this, you need to submit an image that we can add to the project for testing (under a GPL 2.0 license). If you are concerned about privacy, then leave the lens cap when taking a photo. After this is submitted, I will make the change to Exiv2.

[GitHubbero]

Thank you @postscript-dev ,
just to clarify, the "RF ... F4L" is detected but wrong. It should be the "RF...F4-7.1"
So the Lens code should be "61182 //25" but is wrongly "61182 //2"

As I am using Windows and can't figure out, how to use exiv2, maybe you can check the lens detection of exiv2?
Just to make sure that the problem is not from exiv2 but at darktable itself.

Thank you so much at this point

"

[postscript-dev]

@GitHubbero: Usually in posts like yours, the lens value is missing and it is a simple case of adding an entry and a test file. However, this time it is different, as your lens is already defined in Exiv2.

The Exiv2 sample program mentioned in https://www.exiv2.org/manpage.html, is a small command line tool that enables viewing and editing of image metadata. I can use this to output the lens tag from your test file:

$ exiv2 --Print kt --key Exif.CanonCs.LensType Canon_RF_24-105_F4-7.1_IS_STM.jpg
Exif.CanonCs.LensType                         Unknown Lens (61182)

Although the lens is defined in Exiv2, it is currently not being found and this is something that I will fix. However, even if I submitted this today, due to release dates, it could still take more than 6 months before arriving in darktable. There is another solution though, that will only take a couple of minutes for you to complete.

If you add a lens configuration file to your computer then this will fix the problem on your machine. To do this in Windows:

1. Open a new document in any text editor (e.g., Notepad, Wordpad, Libre Office Writer, MS Word, etc)
2. Copy the following into the program:

[canon]
61182=Canon RF 24-105mm F4-7.1 IS STM

3. Save the file in your home directory (i.e., C:\Users\<Your Windows username>, e.g., C:\Users\postscript-dev) as a text file (.txt) called: exiv2.ini . Some text editors automatically add .txt on the end of any filename and such a file will need to be renamed to exiv2.ini
4. The new definition will now be displayed for that user in any software using the Exiv2 library (e.g. darktable, GIMP)

If I rerun the test from above, you can see that the output has changed.

$ exiv2 --Print kt --key Exif.CanonCs.LensType Canon_RF_24-105_F4-7.1_IS_STM.jpg
Exif.CanonCs.LensType                         Canon RF 24-105mm F4-7.1 IS STM

To make the correct value appear in darktable:

1. Start/restart darktable
2. Delete any existing files that have the incorrect lens value
3. Re-add the files and now the lens entry of the image information will use the correct value

I don't use darktable much, so there may be a simpler way to refresh the files than what I have shown.

[GitHubbero]

Thank you for your great work.
I will run the test tomorrow, and will inform you about the result.
Thanks a lot

[EC1000]

I should be corrected by #1692 👍 (at least, the Canon RF 50mm F1.8 version is now correctly identified while it was identified as the F1.2 version with exiv2 0.27.5)

Note that all Canon RF lenses have the same LensType=61182.

[GitHubbero]

[canon]
61182=Canon RF 24-105mm F4-7.1 IS STM

Thank you @EC1000 . I knew that the RF lenses have all the same type, but they are different with the addition e.g. //2 aren't they?
Is there a way to implement these differentiations into the manual exiv2.ini (from @postscript-dev ) for the time until the update? (I just need the defferentiations of 2 or 3 lenses)
Or can i somehow update the main data by myself? I don't use exiv2 on my own, just darktable.

I found exiv2.mo files in my share folder of darktable. I assume i have to exchange them with the new one, am I right?
But where can i find the newest file in the exiv2/main branch?

[postscript-dev]

@EC1000:

Note that all Canon RF lenses have the same LensType=61182.

I have no experience with Canon cameras or lenses, so this is helpful.

@GitHubbero:

I knew that the RF lenses have all the same type, but they are different with the addition e.g. //2 aren't they?

The //2 is only a comment and is ignored by the software. If not using a lens configuration file, Exiv2 will produce a list of lenses that match the definitions that it has.

Is there a way to implement these differentiations into the manual exiv2.ini (from @postscript-dev ) for the time until the update? (I just need the defferentiations of 2 or 3 lenses)

Out of interest, what are your other lenses?

[GitHubbero]

Out of interest, what are your other lenses?

There is the RF24-240 4-6.3 and RF 100-400 5.6-8

[EC1000]

What exiv2 version are you using?

If you have three RF lenses that are not identified, you can find a possible solution here (set 61182=000 in .exiv2 file, so exiv2/dt will pick the LensModel name instead of the LensType; note the small difference: without "Canon" at the beginning).

All should be ok with future exiv2 V1.00 version :)

[GitHubbero]

If you have three RF lenses that are not identified, you can find a possible solution here

On the funny side, my 100-400 lens is identified correctly (couldn't figure out how it's with the 24-240 because it's just on my wishlist by now)
So I thought the problem has to be with maybe the naming of the lens or the detection. Following the correctly identification of the one lens, there should no problem with the single Lens-ID for all RF lenses

I couldn't find out which version I am using, because I just have darktable installed. But it should be 0.27.5, at least 0.27.4.

[clanmills]

It's very likely that the darktable "About" box (or preference panel) will list the version of every library.

[GitHubbero]

It's really weird, because there is no "about" box in darktable. I searched for it before, and also now but i can't find something like that. Nor anything like this in the options/preferences
But instead I compared the date of the used exiv2.mo file of darktable and it was just 2 days "newer" than the newest version of exiv2 (23.10.2021 / 21.10.2021) So I assume it's the current version v0.27.5

[clanmills]

Some of the darktable folks are on the Exiv2 chat server and they'll advise. When I retired from Exiv2 last year, I left the chat room. Time for me to leave Exiv2 to other great young guys like @postscript-dev.

[postscript-dev]

@GitHubbero: Sorry for the delay replying, I needed to finish some other jobs first. I don't have much free time this week but your issue is the next task that I intend to work on.

[clanmills]

@postscript-dev Thanks for getting involved with this, Peter. I know you're doing lots of good stuff for Exiv2 AND you have a life to live! I'm sure @GitHubbero appreciates your efforts.

@GitHubbero I've thought of other ways to check the version of exiv2 being used by darktable.

1. use $ lsof | grep darktable | grep exiv2
   On my Linux VM (polluted by building exiv2) it shows libexiv2.so.0.27.5.1

2. If you are using an anonymous exiv2 shared object, grep it for the string '0.27.5' (which is baked into the shared object) by the compiler/linker: For example:

539 rmills@rmillsm1-ubuntu:~/gnu/github/exiv2/0.27-maintenance/build/lib $ ls -l
total 4616
lrwxrwxrwx 1 rmills rmills      14 Feb  6 11:18 libexiv2.so -> libexiv2.so.27
-rwxrwxr-x 1 rmills rmills 3675944 Feb  6 11:18 libexiv2.so.0.27.5
lrwxrwxrwx 1 rmills rmills      18 Feb  6 11:18 libexiv2.so.27 -> libexiv2.so.0.27.5
-rw-rw-r-- 1 rmills rmills 1046168 Feb  6 11:17 libexiv2-xmp.a
540 rmills@rmillsm1-ubuntu:~/gnu/github/exiv2/0.27-maintenance/build/lib $ grep 0.27.5 libexiv2.so.0.27.5 
Binary file libexiv2.so.0.27.5 matches
541 rmills@rmillsm1-ubuntu:~/gnu/github/exiv2/0.27-maintenance/build/lib $ grep 0.27.4 libexiv2.so.0.27.5
542 rmills@rmillsm1-ubuntu:~/gnu/github/exiv2/0.27-maintenance/build/lib $

[GitHubbero]

@postscript-dev I appreciate definitely you effort. And there is no pressure by my side.

And thank you @clanmills for your thoughts, but I use Windows as platform and so it's always a bit harder around the darktable environment. Sorry for that.
But I can't imagine, that the newest darktable version (which I installed only about 4 weeks ago) uses an old exiv2 version. Therefore I assume I have the exiv2 version 0.27.5, but haven't got a proof.

[clanmills]

@GitHubbero I forgot you're using Windows. BTW I like Windows (but don't tell anybody). Use procexp.exe to determine which exiv2.dll is being used. https://docs.microsoft.com/en-us/sysinternals/downloads/process-explorer We'll have to update the exiv2 build to add a public "version" to the DLL properties which would be visible in procexp.exe

If your version of darktable supports BMFF (Canon .CR3, Apple .HEIC), then it's using exiv2 v0.27.4 or later.

Using grep, or dumpbin will probably reveal the version string baked into exiv2.dll.

[GitHubbero]

If your version of darktable supports BMFF (Canon .CR3, Apple .HEIC), then it's using exiv2 v0.27.4 or later.

Yes I am working with .CR3 and darktable shows me correctly the image data. Except the lens data of RF 24-105

[postscript-dev]

@GitHubbero:
I am sorry for not working on your issue sooner. Thanks for being patient. I have not had much free time recently.

@hassec:
While working on this issue, I have been looking at the src/canonmn_int.cpp particularly the printCsLensTypeByMetadata() function. I see that you updated this recently and I have a question. In

exiv2/src/canonmn_int.cpp

Lines 2781 to 2784 in b2b6d83

	if (flMin != exifFlMin || flMax != exifFlMax || exifAperMax < (aperMaxShort - .1 * tc) ||
	exifAperMax > (aperMaxTele + .1 * tc)) {
	continue;
	}

f

where does the 0.1 value come from?

[GitHubbero]

@GitHubbero: I am sorry for not working on your issue sooner. Thanks for being patient. I have not had much free time recently.

@postscript-dev you don't have to apologize. I am glad that you take the trouble to solve the problem.
Thank you all so much.

[postscript-dev]

@GitHubbero:
For the past 3 months, I haven't been able to contribute to Exiv2 as I wanted to. Now I have more time and can finish jobs such as yours that are still outstanding. Thanks for being very patient.

I have submitted a PR to fix your problem (#2235) which will now output:

$ exiv2 --Print kyyvt --key Exif.CanonCs.LensType Canon_RF_24-105_F4-7.1_IS_STM.jpg
Exif.CanonCs.LensType                        Short      61182  Canon RF 24-105mm F4L IS USM *OR* Canon RF 24-105mm F4-7.1 IS STM

Providing a list of these two lenses is the best we can do at the moment.

When the PR is merged, the issue will be fixed on the main branch.

Thanks again.

[kmilos]

FWIW, I have dropped using Exif.CanonCs.LensType in darktable when 61182 is detected, it uses vendor Exif.Canon.LensModel for RF lenses now on master (and the upcoming 4.0).