Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unabbreviate on import #1436

Closed
jlaehne opened this issue Feb 18, 2020 · 112 comments
Closed

Unabbreviate on import #1436

jlaehne opened this issue Feb 18, 2020 · 112 comments

Comments

@jlaehne
Copy link

jlaehne commented Feb 18, 2020

Report ID:
MUF2FD2N-euc

Exporter used:
Better BibTeX

Expected behavior:
No matter whether the journal field contains a full our abbreviated journal name, the journal titles should be written to the .bib file in the same way.

Actual behavior:
If the full journal name is given in the publication field in Zotero (e.g. Applied Physics Letters), the title is correctly abbreviated in bibtex: journal = {Appl. Phys. Lett.}
If the publication field contains an already abbreviated journal name (e.g. Appl. Phys. Lett.), the exported bibtex entry omits the dots: journal = {Appl Phys Lett}
Apparently, the abbreviator throws out the dots in case the journal name is already abbreviated, while it should just keep the entry as is.

I have a mixture of publisher entries in my library, as some are directly imported in Zotero and others were imported from a .bib file with abbreviated journal names. However, in the resulting bibliography, I want them all to appear the same (with dots).

I tried to change the hidden preference autoAbbrevStyle to the American Physics Society instead of Nature style, but that does not change anything.

@jlaehne jlaehne added the bug label Feb 18, 2020
@retorquere retorquere added question and removed bug labels Feb 18, 2020
@retorquere
Copy link
Owner

retorquere commented Feb 18, 2020

The Zotero item in MUF2FD2N-euc has a filled-out Journal Abbreviation field without periods. If a Journal Abbreviation is filled out, that always takes precedence, exactly as-is.

BibTeX doesn't have a separate field for abbreviations, so for bibtex, "use journal abbreviation" means "replace the journal name with the abbreviation". I don't know why people would want this, but some people do.

If you want both the journal title and the abbrev, you will have to export using Better BibLaTeX rather than Better BibTeX.

@retorquere
Copy link
Owner

The auto abbrev style only works in Juris-M, but in the case of the item you submitted in the log it wouldn't make a difference there either, since the item has a journal abbrev field filled out.

@jlaehne
Copy link
Author

jlaehne commented Feb 19, 2020

Well, in Zotero the journal abbreviation field is actually empty and only the journal field is filled out, see attached screenshot:
Zotero-Screenshot

If the debug report says the journal abbreviation field is filled out, is that maybe a bug of Zotero?

This is the resulting BibTex file where the dots are gone:

Exported Items.txt

@retorquere
Copy link
Owner

You're right -- the debug log code had a bug that filled the auto-abbrev into the abbrev field during export, and that's what I was looking at. That's fixed now, but that leaves your problem.

I'm afraid that still comes down to pretty much the previous explanation: you have "Automatic Abbreviation on, which will ask Zotero to provide a journal abbreviation if you did not fill one out yourself. If Zotero provides one, that will be used as if you filled out that field yourself.

Since BibTeX does not have any field for journal abbreviations, "use journal abbreviations" means "replace the journal title with the journal abbreviation, if available". Maybe this should be "replace the journal title with the abbreviation of the user filled one out". That should solve your problem.

I can't do anything about the dots however. I don't generate the abbrevs myself, Zotero contains a citation processing library that it uses to generate its own bibliographies, and I just ask that library to give me an abbreviation. If you generate a bibliography with Zotero, it would show the same abbreviation.

Repository owner deleted a comment from blip-bloop Feb 19, 2020
@retorquere
Copy link
Owner

wait -- something else is going on. When I export that item, I get

@article{ZaremDirectdeterminationambipolar1989,
  title = {Direct Determination of the Ambipolar Diffusion Length in {{GaAs}}/{{AlGaAs}} Heterostructures by Cathodoluminescence},
  author = {Zarem, HA and Sercel, PC and Lebens, JA and Eng, LE and Yariv, A and Vahala, KJ},
  year = {1989},
  volume = {55},
  pages = {1647--1649},
  publisher = {{AIP}},
  doi = {10.1063/1.102226},
  journal = {Appl. Phys. Lett.},
  number = {16}
}

@retorquere
Copy link
Owner

Can you go into the advanced prefs and clear the cache?

@jlaehne
Copy link
Author

jlaehne commented Feb 19, 2020

Sorry, I was not able to log-in to GH for a while.

I can't find that option?

Though, I did clear translators and reinstalled BBT. With no avail.

@jlaehne
Copy link
Author

jlaehne commented Feb 19, 2020

I also did a few more checks:

  • The LibreOffice plugin correctly exports the dot.
  • If I use the BibTeX exporter of Zotero instead of BBT, it also exports the dot

So it seems to be a BBT issue and not one of Zotero. But why do you get the correct export and me (as well as another colleague) are missing the dot?

@retorquere
Copy link
Owner

It's definitely a BBT issue. Go into Preferences - BBT - Advanced - export - reset cache.

I can see the dotless version in your cache -- I don't know how it got there, but that's why you're seeing this.

@jlaehne
Copy link
Author

jlaehne commented Feb 19, 2020

I reset the cache, but no change in the behaviour.

This behaviour is not unique to this one item, but affects a large number of similar items.

@retorquere
Copy link
Owner

Can you shut down zotero and remove all json files in the zotero/better-bibtex folder and try again?

@jlaehne
Copy link
Author

jlaehne commented Feb 19, 2020

Did so -- no change :-(

@jlaehne
Copy link
Author

jlaehne commented Feb 19, 2020

As mentioned, a colleague has the same problem.

If I deactivate journal abbreviation, it exports the dot - but then unabbreviated journals are of course exported with the full name as it simply leaves the field untouched.

@jlaehne
Copy link
Author

jlaehne commented Feb 19, 2020

By the way, why is the "Use Journal Abbreviation" setting duplicated? Both in the BBT preferences and in the export dialog it has to be checked, if one of the two is deactivated the names are not abbreviated.

@retorquere
Copy link
Owner

Can you:

  • insert a citation in a Word or LibreOffice document using the Zotero toolbar
  • Go into "document preferences" in the Word/LO toolbar and make sure "Use MEDLINE" is checked
  • Add a bibliography

Does the bibliography show the abbreviation with or without periods?

By the way, why is the "Use Journal Abbreviation" setting duplicated? Both in the BBT preferences and in the export dialog it has to be checked, if one of the two is deactivated the names are not abbreviated.

The do different things, but there is interplay. The one in the preferences means:

If the item does not have a journal abbreviation, generate one and make the export belief that was filed in by the user.

The "use journal" in the export popup (for bibtex) means

If the user had filled out a journal abbrev (or one has been provided by the auto-abbrev), use that instead of the journal title.

@retorquere
Copy link
Owner

This behaviour is not unique to this one item, but affects a large number of similar items.

That's OK (for diagnostic purposes at least). I need to solve it for one, the rest will follow, and it's easier to discuss one item.

@jlaehne
Copy link
Author

jlaehne commented Feb 19, 2020

Well it's getting more and more puzzling.

When I tried at work (I think without MedLine activated, but not sure), the dot was there.

Now I tried at home with the same citation. And the dot is not there. No matter if I activate MedLine or not and both for Nature or APS styles.

@blip-bloop
Copy link
Collaborator

🤖 this is your friendly neighborhood build bot announcing test build 5.2.14.6043 ("Merge branch 'master' into gh-1436")

Install in Zotero by downloading test build 5.2.14.6043, opening the Zotero "Tools" menu, selecting "Add-ons", open the gear menu in the top right, and select "Install Add-on From File...".

@blip-bloop
Copy link
Collaborator

🤖 this is your friendly neighborhood build bot announcing test build 5.2.14.6044 ("Merge branch 'master' into gh-1436")

Install in Zotero by downloading test build 5.2.14.6044, opening the Zotero "Tools" menu, selecting "Add-ons", open the gear menu in the top right, and select "Install Add-on From File...".

retorquere added a commit that referenced this issue Mar 3, 2020
@jlaehne
Copy link
Author

jlaehne commented Mar 3, 2020

Most of the titles in not-unabbreviated.txt are journals where the abbreviation is the same as the full name, and a few conferences or books. The overall performance of the routine is quite impressive now, definitively much better than what I could have done with JabRef. Just two points that I still noticed.

Abbreviations containing special characters (Umlauts and accents) do not seem to be touched, even though they are in the list:

Z. Für Krist.
Z. Für Phys.
Comptes Rendus Académie Sci. - Ser. - Math.

Also, he has the lengthy full name American Association for the Advancement of Science. Science for Science, which I never see used anywhere. I explicitly added Science;Science to unabbr-amendments.csv in one of the past iterations, but it did not help!?

@retorquere
Copy link
Owner

New build incoming in ~45mins.

Special characters should be handled now too.

The second problem was because I was simply filtering out abbr=full instances; I now treat them (and abbr=) as "delete this abbr from the mapping".

not-unabbreviated.txt

@blip-bloop
Copy link
Collaborator

🤖 this is your friendly neighborhood build bot announcing test build 5.2.14.6057 ("timeout for import test")

Install in Zotero by downloading test build 5.2.14.6057, opening the Zotero "Tools" menu, selecting "Add-ons", open the gear menu in the top right, and select "Install Add-on From File...".

retorquere added a commit that referenced this issue Mar 3, 2020
retorquere added a commit that referenced this issue Mar 3, 2020
retorquere added a commit that referenced this issue Mar 3, 2020
@retorquere
Copy link
Owner

Are we good with the current state of affairs? I need to put out a new release.

@retorquere
Copy link
Owner

I have found a problem. Will probably have a new version tonight.

retorquere added a commit that referenced this issue Mar 4, 2020
retorquere added a commit that referenced this issue Mar 4, 2020
retorquere added a commit that referenced this issue Mar 4, 2020
retorquere added a commit that referenced this issue Mar 4, 2020
retorquere added a commit that referenced this issue Mar 4, 2020
@blip-bloop
Copy link
Collaborator

🤖 this is your friendly neighborhood build bot announcing test build 5.2.16.6079 ("optional unabbrev")

Install in Zotero by downloading test build 5.2.16.6079, opening the Zotero "Tools" menu, selecting "Add-ons", open the gear menu in the top right, and select "Install Add-on From File...".

@retorquere
Copy link
Owner

OK, think I have it now. Would appreciate feedback, I'm looking to cut a new release.

@jlaehne
Copy link
Author

jlaehne commented Mar 5, 2020

Wow, I'm impressed. I'll run a few more test libraries and then I'll get back to you.

@jlaehne
Copy link
Author

jlaehne commented Mar 5, 2020

LGTM!

@retorquere
Copy link
Owner

Superb. This needs two layers of tests so it will take a bit to clear, and I want to give the translator volunteers a chance to do their work, but I'll cut a new release somewhere tonight or tomorrow.

@blip-bloop
Copy link
Collaborator

🤖 this is your friendly neighborhood build bot announcing test build 5.2.16.6094 ("new abbrevs")

Install in Zotero by downloading test build 5.2.16.6094, opening the Zotero "Tools" menu, selecting "Add-ons", open the gear menu in the top right, and select "Install Add-on From File...".

@github-actions
Copy link

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Feb 17, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

4 participants
@retorquere @jlaehne @blip-bloop and others