Entry preview: Umlaut and ring diacritic not rendered correctly when APA and IEEE preview layout is selected

[KarlMagnusLarsson]

JabRef version commit be5fa3a on
Mac OS X 10.13.6 x86_64
Java 13.0.1

If I build from source, then entry preview works, but umlauts and ring diacritics are not rendered correctly when APA and IEEE preview layout is selected. Preview layout "preview" is fine.

This is not only JabRef version commit be5fa3a. I think it has been like this for a while.

Steps to reproduce the behavior:

1. git clone https://github.com/JabRef/jabref.git
2. cd jabref
3. ./gradlew run
4. add this bibtex entry to the bibtex library, where {"a} is ä:
   @book{Wickman2009non,
   author={Wickman, Per-Olof and Persson, Hans},
   title={Naturvetenskap och naturorienterande {\"a}mnen i grundskolan: En {\"a}mnesdidaktisk v{\"a}gledning},
   publisher={Liber},
   address={Stockholm},
   year={2009},
   ISBN={978-91-47-05333-9},
   }
5. This is rendered as:
   

It should render as:
Wickman, P.-O. & Persson, H. (2009). Naturvetenskap och naturorienterande
ämnen i grundskolan: En ämnesdidaktisk vägledning. Stockholm,
Liber.
{"a} becomes [& a uml;] instead of ä. (I added [] and spaces for clairity),

6. add this bibtex entry to the bibtex library {"o}=ö.
   @Article{lundstrom2015att,
   author = {Lundstr{\"o}m, U and R{\"o}nnberg, L},
   title = {Att styra skolan med marknaden som f{\"o}rebild. I: {S. Lindblad.}, & {L. Lundahl.(red.)}},
   journal = {Utbildning. Makt och politik},
   year = {2015},
   }

7. This is rendered as:
   

It should render as:
Lundström, U. & Rönnberg, L. (2015). Att styra skolan med marknaden som
förebild. I: S. Lindblad., & L. Lundahl.(red.) Utbildning. Makt och
politik.

8. add this bibtex entry to the bibtex library {\aa}=å
   @book{elstgeest1996vaga,
   author = {Elstgeest, J. and Harlen, W. and Moback, M. and Persson, H.},
   title = {V{\aa}ga spr{\aa}nget!: om att undervisa barn i naturvetenskapliga {\"a}mnen},
   year = {1996},
   series = {A \& W pedagogen},
   publisher = {Almqvist \& Wiksell},
   isbn = {9789121147788},
   }

Then {\aa} becomes [V & a ring;] (I added [] and spaces for clairity), instead of å.

[Siedlerchr]

Thanks for the detailed report. Does it work, if you directly add the Unicode char/symbol in the bibtex entry?

[KarlMagnusLarsson]

Your suggestion works. I think you are suggesting to test what I call case 2) below.

I know of two types of bibtex entries for diacritics:

1. either the {\"a}, {\"o} and {\aa} way, or,
2. with \usepackage[utf8]{inputenc} and \usepackage{biblatex} in the latex souce, I can have ä, ä and å directly in the bibtex entry.

If I download a journal reference, the type 1) is the one that is the most common, perhaps because it is more portable.

Type 2) with \usepackage[utf8]{inputenc}, \usepackage{biblatex} and å,ä, ö etc in entry, work fine in Jabref entry preview. I have tested that. (This form is not so common if you download bibtex entries from journals).

[Siedlerchr]

Thanks for the follow up. We do have a latex2unicode formatter. As a solution it should be possible to call the latex2unicode formatter before generating the preview for the citation styles.
Alternatively, biblatex and biber allow Unicode directly.

[KarlMagnusLarsson]

Is it not a bug, instead of an enhancement?

The reason for my position is that almost all bibtex entries from Swedish research journals, proceedings and books use {\"a}, {\"o} and {\aa} etc. for diacritics and it seems in my test that {\"a}, {\"o} and {\aa} etc. do not render correctly. I am talking about the case in Sweden, but I think this extends to Norway, Finland, Denmark, Germany and Austria where diacritics are common. German has other cases like Eszett, that I have not tested.

It is true that biblatex and biber allow umlaut and ring diacritics directly. This is my setup and it works fine, but this is more on the latex/pdf side of things.

Why should not Jabref render APA and IEEE preview layout correctly for {\"a}, {\"o} and {\aa} and so on, when Jabref handles the preview layout "preview" with {\"a}, {\"o} and {\aa} perfectly?

However my position on bug/enhancement is not strong. I am not even sure it is a bug? Is it only I who see this, or can this be reproduced?

[Siedlerchr]

The preview layout is JabRef internal, it's based on the same functionality as the export formatters and already part of JabRef since several versions.

The other like IEEE or APA come from an external library, CSL, which is relatively new.
https://citationstyles.org

It doesn't matter if it's a bug or enhancement, the fix will be simple.

[jjsfernandez]

Hi, is this already addressed somewhere else? I can work on this.

[Siedlerchr]

Welcome to JabRef, thanks for your interest! Feel free to work on this. Please read the contribution guide.
Code wise you should look into the citationstyle package under logic.
The formatter should be called before the entry is passed to csl engine

[Siedlerchr]

Thanks to @JosejeSinohui this issue should be fixed now..

We would like to ask you to use a development build from https://builds.jabref.org/master and report back if it works for you. Please remember to make a backup of your library before trying-out this version..

[KarlMagnusLarsson]

Thank you @Siedlerchr for looking into this. Thank you @JosejeSinohui the rendering of diacritics works fine after your patch.

During testing i noticed something else that is not related to diacritics. The character \& for ampersand does not render correctly and becomes "& amp;", for instance in this publisher record:
@book{MacLane2013,
author = {Mac Lane, Saunders},
title = {Categories for the working mathematician},
year = {2013},
volume = {5},
publisher = {Springer Science \& Business Media},
}

It is the same story, APA and IEEE renders "& amp;" and "preview" works and produce &.

Maybe this merits a separate issue.

Merry X-mas,

[tobiasdiez]

@KarlMagnusLarsson Thanks for the feedback. I guess the issue you mention is the same as #3840, right?

[KarlMagnusLarsson]

@tobiasdiez, yes you are right. It is the same thing.

I should have checked, but I did not, in this post closed comment. However I see the same thing as you point out in #3840: the internal "preview" works and other formats like APA and IEEE do not.