Make lookahead, lookbehind and/or conditional if else statements work for 'linked file regular expression search' and 'automatically set file links' (F7)

[ThiloteE]

Relevant UI components:

Problem:

• It is impossible to create proper search expressions that capture a fine grained date. Whenever [YEAR] is used, it leads to multiple attachments for linked entries that happened in the same year, regardless if the correct file was already linked via [DATE] prior to the [YEAR] search. See graphic:

Describe the solution you'd like

Solutions i am aware of:

• Make field markers (such as [YEAR] or [DATE] work within the placeholder modifier :(x) for linked file regular expression search
  The following expression would be great to have, but is not possible currently:
  **/.*[auth.etal:regex("\\."," & "):regex("\\& etal","et al.")].*[Organization].*[DATE]:([YEAR][MONTH]:([YEAR]).*\\.[extension]
  (Translation: if and only if YYYY-MM-DD cannot be found, search for YYYY-MM. If and only if YYYY-MM cannot be found, search for YYYY.)
• Add a more specific modifier or field marker that makes [DATE] work for YYYY or YYYY-MM. (Take into account that the entry in Jabref may consist of YYYY-MM or YYYY-MM-DD, whereas the file in folder may just be named YYYY or YYYY-MM. It still should work in these edge cases.
• Allow a modifier to be marked as empty.
  If :(x) could be empty (and with that i mean literally no value as output), it would be possible to move on to the next search expression. I think this could create If, then statements (boolean expressions).
  Example: .*[DATE]:().*[YEAR].*\\-[MONTH]:().*[YEAR]
  Translation: If Date is empty, then search for YYYY-MM. If MM is empty, search for YYYY.

I don't know if all this or one one of these solutions are feasible within javaFX. I hope so.

Additional context

https://discourse.jabref.org/t/help-with-search-expression-for-automatically-set-file-links-f7-attach-files-to-entry/2991/3

JabRef 5.4--2021-10-15--93b8025
Windows 10 10.0 amd64
Java 16.0.2
JavaFX 17.0.0.1+1

[ThiloteE]

Reminder for myself:

• Play around with https://stackoverflow.com/questions/3687921/conditional-regular-expression-in-java
• Try to use lookahead or lookbehind constructs
  • https://www.rexegg.com/regex-conditionals.html
  • Java equivalents: https://docs.oracle.com/javase/8/docs/api/java/util/regex/Pattern.html

[ThiloteE]

I was not able to make lookahead, lookbehind and conditional if else statements work.

Even if such expressions were possible, the way the date and year fields are currently captured is too flexible for my use case. I mean, it is NICE that it is the way it is. There is a certain logic behind it and it is a good logic! It's just ... my use case requires a different logic 😅

Example:

Regular expression in linked file search: **/.*[DATE].*\\.[extension]

When having 4 pdf files in my folder with the following names:

1. test (2021).pdf
2. test (2021-07).pdf
3. test (2021-07-07).pdf
4. test.pdf

THEN

@Test{202107y,
  month = {07},
  title = {Year+Month},
  year  = {2021},
}

finds 2. test (2021-07).pdf and 3. test (2021-07-07).pdf.

I was surprised [DATE] did not find a simple YYYY here, but later i understood the logic. The logic is that it always gets at least the minimum date from the entry (in this example the year and month fields), but also does include any search results from the pdf file that go beyond. The minimum as fetched from the entry here is YYYY-MM, whereas search results at the pdf file not only result in YYYY-MM, but also go beyond this: YYYY-MM-DD.

@Test{2021y,
  title = {year},
  year  = {2021},
}

finds 1. test (2021).pdf and 2. test (2021-07).pdf

@Test{20210707d20,
  date  = {2021-07-07},
  title = {Date 2021-07-07},
}

finds 3. test (2021-07-07).pdf. This is the most strict one.

@Test{202107d20,
  date  = {2021-07},
  title = {Date 2021-07},
}

finds 2. test (2021-07).pdf and 3. test (2021-07-07).pdf

@Test{2021d2,
  date  = {2021},
  title = {Date 2021},
}

finds 1. test (2021).pdf and 2. test (2021-07).pdf and also 3. test (2021-07-07).pdf.

As explained above, the minimum search result fetched from the entry [DATE] field can also lead to very suboptimal results. When 3. test (2021-07-07).pdf is deleted from the folder (or renamed to something different), then this happens:

@Test{20210707d20,
  date  = {2021-07-07},
  title = {Date 2021-07-07},
}

finds: nothing (!)

This one is particularly annoying, if the file i have in my folder is not named with fully YYYY-MM-DD, but rather YYYY-MM. This one here alone basically makes it impossible to use [DATE]. It is nice that it finds the full YYYY-MM-DD, and if the pdf file is named that way, indeed it is nice that all the other files that are formated with YYYY-MM or YYYY do NOT show up, but you can see, how a conditional statement here could make this a lot nicer. E.g.: Search for YYYY-MM or YYYY only, if YYYY-MM-DD was not found. Alas, i was not able to make these conditional statements work.

Replacing [DATE] with [YEAR] is not a proper solution. [YEAR] alone finds too much.

• [YEAR] finds YYYY, YYYY-MM and YYYY-MM-DD.

[ThiloteE]

IF, conditional statements were working, having a [MONTH-MINIMUM], as well as [YEAR-STRICT], [MONTH-STRICT] and [DATE-STRICT] field markers, would be the the solution to this thread i think.

• Example for [YEAR-STRICT]: Find YYYY from entry in file-directory. Do not show files that have dates that go beyond. Do not show YYYY-MM-DD or YYYY-MM
• Example for [MONTH-MINIMUM]: Show YYYY-MM and YYYY-MM-DD. Basically a field marker that does the same as what [DATE] or [YEAR] currently do, just that the minimum starts at YYYY-MM. Do not show YYYY.

Maybe relevant:

• https://stackoverflow.com/questions/2137929/how-can-i-use-a-regular-expression-to-validate-month-input
• https://www.regular-expressions.info/dates.html

[ThiloteE]

Conditionals (lookahead and lookbehind regular expressions) seem to work also with Jabref, but so far only within a string: JabRef/user-documentation#525

[ThiloteE]

To do:

Try

Allow a modifier to be marked as empty.
If :(x) could be empty (and with that i mean literally no value as output), it would be possible to move on to the next search expression. I think this could create If, then statements (boolean expressions).
Example: .[DATE]:().[YEAR].\-[MONTH]:().[YEAR]
Translation: If Date is empty, then search for YYYY-MM. If MM is empty, search for YYYY.

with

https://stackoverflow.com/questions/3012788/how-to-check-if-a-line-is-blank-using-regex