JabRef groups format 4

[retorquere]

JabRef version after 3.8.1 have a new groups format; I can't say I'm a fan, but it's there going forward. I've attached a sample file. In the new format:

1. there's no @comment{jabref-meta: groupsversion:3;}; instead, there's either @Comment{jabref-meta: databaseType:bibtex;} (if the DB was in bibtex format) or @Comment{jabref-meta: databaseType:biblatex;} (if the DB was in biblatex format)
2. the groups are structured the same but always have zero references. The references themselves now have a groups field which contains a comma-separated list of group names. References belong to the first group that has that name, regardless of how the groups are nested. More details are here and here. I don't know whether JabRef's assignment is depth-first or breadth-first but the bare fact that this could matter makes the format broken from my POV. Anyhow.

I don't really need support for this in the parser as I have a simple workaround I can apply post-parse, but I thought I'd give you a heads-up.
jr381.bib.txt

[johanneswilm]

We have something like 80-100 lines dealing with the previous format so far. We should probably have an overall strategy -- if you want to move all the jabref parsing into your code, that's fine with me. Or we can keep it all here and then add for other formats. For how long do we need to support different versions of the format? Is the format we currently support version 3?

[retorquere]

Oh please keep the jabref parsing! There's tons of people still using the "old" format; it's available in Jabref as an option, and it really is the superior option. Right now I just let this parser do it's thing, and then I do roughly this to whenever I find a groups field:

// when a groups field is found
var groups = value.split(',').map(function(group) { return group.trim() }).filter(function(group) { return group })
var group, i, len, name;

for (i = 0, len = groups.length; i < len; i++) {
  name = groups[i];
  if (group = findGroup(name)) {
    group.references.push(ref.entry_key);
  }
}

This could in principle be added after all the refs have been parsed and then you'd have seamless jabref groups 3 and 4 parsing.

[retorquere]

and yes, the format we currently support is 3; the new format doesn't have a version number.

[johanneswilm]

Ok, why don't we add it here then? Just have one, outdated version seems a bit odd.

[retorquere]

(I call it 4 to distinguish it from 3, but that's a misnomer the JabRef authors wouldn't recognize)

[retorquere]

Version 3 is odd, but version 4 is almost as odd and fundamentally broken on top of that. See the ignored complaints in those two jabref issues.

[retorquere]

I can have a stab at adding the 4 code to the parser, yeah?

[johanneswilm]

yeah, let me remove it from the main file first
then you can do that
it shouldn't take long

[retorquere]

You don't want my code for this in any case 🙄

It would look roughly like this:

diff --git a/src/import/biblatex.js b/src/import/biblatex.js
index 5d44b47..67fdd59 100644
--- a/src/import/biblatex.js
+++ b/src/import/biblatex.js
@@ -797,11 +797,34 @@ export class BibLatexParser {
         )
     }
 
+    findGroup (name, groups) {
+      for (let i = 0, i < groups.length; i++) {
+        if (groups[i].name === name) return group;
+        if (let group = this.findGroup(name, group.groups)) return group;
+      }
+      return null;
+    }
+    jabrefGroups4() {
+      for (let id in this.bibDB) {
+        let ref = this.bibDB[id];
+        if (ref.unknown_fields.groups) {
+          let groups = ref.unknown_fields.groups[0].text.split(',').map(function(name) { return name.trim() }).filter(function(name) { return name })
+          for (i = 0, len = groups.length; i < len; i++) {
+            if (group = findGroup(groups[i])) {
+              group.references.push(ref.entry_key)
+            }
+          }
+          delete ref.unknown_fields.groups
+        }
+      }
+    }
+
     get output() {
         this.replaceTeXChars()
         this.stepThroughBibtex()
         this.createBibDB()
         this.cleanDB()
+        this.jabrefGroups4(this.groups)
         return this.bibDB
     }

[johanneswilm]

Ok, I'm done. It's split into its own file. Precaution in case we end up supporting 15 different group formats. Could you add your code on top of that?

[retorquere]

I've added it on a separate branch, but I'm not happy with it for several reasons:

1. The groups parser needs access to the references
2. The groups parser modifies the references
3. It doesn't work

Can you have a look what I did wrong? I lifted the code to resolve the JR4 groups from BBT and there it does work.

[johanneswilm]

I have fixed it somewhat. Also, I have split the jabref3 and jabref4 filters as much as possible, just so that it will be as easy as possible to remove one of them in the future, should that be necessary. I am not sure whether the output is what you need. Please feel free to continue working n this until you feel satisfied and then send me a PR.

[retorquere]

It's good as-is, I've submitted a PR. All my tests pass, would love to see an npm update for this.

[johanneswilm]

Sorry, the version was released a few days ago.