feat: date time formats from locales #4029

YUCLing · 2024-09-27T15:40:10Z

Fixes flarum-lang/chinese-simplified#23 and flarum-lang/chinese-simplified#27 (in another way)

Changes proposed in this pull request:
This PR adds some time formats to dayjs, which makes the customized time formats used by Flarum localizable in locale object of dayjs locales.

The PR also made some improvements to humanTime and liveHumanTimes's comments.

Reviewers should focus on:
The new formats' naming and whether is it a good implementation.

All language packs should contain the following formats after this is merged:

f and F: For humanTime
ff and FF: For scrubber

It's not necessary though, a fallback will be used if they are not provided.

Although I've tested on my local Flarum installation, it would be nice for you to test it again.

Screenshot

QA
All places where humanTime is used and post stream's scrubber.

Necessity

Has the problem that is being solved here been clearly explained?
If applicable, have various options for solving this problem been considered?
For core PRs, does this need to be in core, or could it be in an extension?
Are we willing to maintain this for years / potentially forever?

Confirmed

Frontend changes: tested on a local Flarum installation.
Backend changes: tests are green (run composer test).
Core developer confirmed locally this works as intended.
Tests have been added, or are not appropriate here.

Required changes:

Related documentation PR: (Remove if irrelevant)
Related core extension PRs: (Remove if irrelevant)

ExerciseBook · 2024-09-28T23:44:50Z

framework/core/js/src/common/index.ts

@@ -11,9 +11,11 @@ import 'jquery.hotkeys/jquery.hotkeys';

 import relativeTime from 'dayjs/plugin/relativeTime';
 import localizedFormat from 'dayjs/plugin/localizedFormat';
+import dayjsPlugins from './utils/dayjsPlugins';


avoid to use XXXXXPlugins, considering another more specific name like humanTimeFormatter.

It's not only for humanTime.

so i said like, better not to use common name

ExerciseBook · 2024-09-28T23:49:30Z

framework/core/js/src/common/utils/dayjsPlugins.ts

@@ -0,0 +1,24 @@
+const customFormats: import('dayjs').PluginFunc = function (_option, c, _factory) {


And there need some head comments above the function signature.

Need to explain how f and F work, what they mean, and how to replace.

Because this function contains regex which is hard to understand, so if there are no notes about how this function works, it may be hard to maintain it in the future.

And there need some head comments above the function signature.

It's a DayJS plugin. You can get some info from DayJS docs, I can add @see.

Need to explain how f and F work, what they mean, and how to replace.

Their meaning is already in the PR's description. And to replace them, you only need to do it like how you did with DayJS's Localized Formats.

this function contains regex which is hard to understand

It works exactly like DayJS's Localized Format plugin.

add it to the code comments. Although know that, the later programmer may not.

ExerciseBook · 2024-09-29T01:55:44Z

I have another question, because we are adding features in Flarum core, can we make this feature more configurable?

such as

> new Intl.DateTimeFormat('zh-TW-u-ca-roc', {  year: 'numeric',  month: 'long',  day: 'numeric', }).format(new Date(2025, 4, 14));
'民國114年5月14日'

so I prefer that we make a flarum extender thus language-pack or something else plugin can define a new format.

YUCLing · 2024-09-29T01:58:33Z

so I prefer that we make a flarum extender thus language-pack or something else plugin can define a new format.

This is possible with the version in my working tree. But I don't see the possible use cases.

ExerciseBook · 2024-09-29T02:02:32Z

https://x.com/taltalasuka/status/1839500307094450552

fyi

YUCLing · 2024-09-29T02:11:36Z

I have another question, because we are adding features in Flarum core, can we make this feature more configurable?

such as
> new Intl.DateTimeFormat('zh-TW-u-ca-roc', {  year: 'numeric',  month: 'long',  day: 'numeric', }).format(new Date(2025, 4, 14));
'民國114年5月14日'
so I prefer that we make a flarum extender thus language-pack or something else plugin can define a new format.

I see what you mean, the problem is that this PR only solves the hardcoded format problem with Flarum and DayJS's Localized Format plugin. The feature you requested needs a big change to how Flarum and DayJS work.

ExerciseBook · 2024-09-29T02:15:34Z

make sense

rob006 · 2024-09-29T10:16:30Z

Note that there was some discussion about it in: https://discuss.flarum.org/d/22913-what-date-format-do-you-use

As for the problem, I would much more prefer format defined as part of forum translation, like:

ago = d.format(app.translator.trans('core.forum.date-format.humanTimeShort'));

It will be easier to understand and maintain for language pack maintainers, and you can quite easily adjust it also as a forum owner. And overall it seems to be simpler and less hacky solution than introducing some cryptic FF formats.

YUCLing · 2024-09-29T11:31:23Z

As for the problem, I would much more prefer format defined as part of forum translation

I think this is kinda messy since you have one set of formats in DayJS that cannot be easily edited by forum owners and another set of formats in Flarum locale system.

This is the reason I didn't implement it in this way at the beginning.

rob006 · 2024-09-29T11:59:32Z

I think this is kinda messy since you have one set of formats in DayJS that cannot be easily edited by forum owners and another set of formats in Flarum locale system.

I'm not sure what the problem is. Most language packs copied dayjs translations from the source without any changes (I actually had plans to automate it and fetch updates automatically from dayjs repo). This format can be quite cryptic and tricky to change, but there is no really other way to translate dayjs.

If your solution relies on editing dayjs translations, then it inherits all its problems: it is more problematic for language pack maintainers and much harder to override by forum owners. On the other hand, if we use app.translator.trans() to pick format, then adjustring in Weblate or using Linguist should be quite easy. And in the future Flarum could add option to override these formats in admin panel (it could just override specific translations from language pack).

YUCLing · 2024-09-29T14:06:14Z

On the other hand, if we use app.translator.trans() to pick format

Actually, my latest code on my working tree already removed the Localized Format plugin from DayJS, so it's possible to make it also use the formats available in the language pack instead of the one from DayJS locale.

In this way, the priority of the format can become like this:
Language pack (Flarum translator) > DayJS locale > Hardcoded default English format

So we can make all formats available in DayJS Localized Format plugin and the ones that Flarum uses editable in Flarum's translator.

Does this sound good to you?

YUCLing · 2024-09-29T15:16:01Z

Some details of the changes made still need some discussion, the PR is now a draft.

chore: fetch format from translator

YUCLing · 2024-09-29T15:38:33Z

@rob006 I've pushed new changes, would you like to check if it looks good to you? I haven't tested it yet.

rob006 · 2024-09-29T16:57:39Z

Is there any reason why you implemented this in this way instead of passing app.translator.trans() result to format()?

YUCLing · 2024-09-30T00:35:42Z

Is there any reason why you implemented this in this way instead of passing app.translator.trans() result to format()?

I just did what DayJS did to implement the localized format and make it work with Flarum. Also made it extendable.

SychO9

Thanks for the PR! much appreciated!

I think the FF ff formats with the dayjs plugin will only add to the confusion with date time formats and future maintenance.

I'm going to propose something a little more simple but more clear.

Let's add a format(format: string, datetime: Dayjs) to the Translator class in common.

this method would:

check if the provided format exists in a custom item list (customFormats()) and use that.
Fallback to app.translator.trans('core.lib.datetime_formats.'+format)
Fallback to the actual format.

There are a lot of different formats used throughout the app, (+extensions) so this would give language packs more power over how those formats are changed.

The locale would be literally the format as a key, so:

core:
  lib:
    datetime_formats:
      "h A": "h A"
      "MMMM YYYY": "MMMM YYYY"
      "YYYY-MM-DD": "YYYY-MM-DD"
      "YYYY-MM-DD h:mm A": "YYYY-MM-DD h:mm A"
      "YYYY-MM-DD h:mm:ss A": "YYYY-MM-DD h:mm:ss A"
      "LL": "LL"
      "LLL": "LLL"
      "LLLL": "LLLL"

core would have one single example commented out. While language packs can fill more entries as needed.

The new method would be called like so:

app.translator.format('MMMM YYYY', datetime)

@YUCLing @rob006 what do you think? do we see any potential issues with that approach?

rob006 · 2024-10-01T19:15:32Z

@SychO9 I think it would be much more practical and intuitive to override/translate format used in specific situation/place, than just overwrite dayjs formats. For example in Polish language pack I would prefer to use LL instead of ll in humanTime(), but it does not mean that I want to overwrite ll everywhere.
Also, from translator perspective, it is much easier to understand key like core.lib.datetime_formats.humanTimeFull than core.lib.datetime_formats.LL. And spaces and special characters in translations keys could be a can of worms...

@YUCLing Sorry, it looks overcomplicated to me and I don't see a practical reason to implement this in that way.

YUCLing · 2024-10-02T16:56:35Z

@SychO9 @ExerciseBook Check if the latest changes still contain issues 😃

rob006 · 2024-10-02T19:15:20Z

What is the use case for dateTimeFormats and fallback?: string parameter?

SychO9 · 2024-10-02T19:25:20Z

@YUCLing awesome work!

What is the use case for dateTimeFormats and fallback?: string parameter?

dateTimeFormats is for extensibility, it allows extensions to add custom behavior.
the fallback is meant to be used in case for whatever reason the locale value is not available so it won't produce an erroneous value. Language packs might not have it right away. Though I wonder if we really need it.

rob006 · 2024-10-02T19:49:27Z

dateTimeFormats is for extensibility, it allows extensions to add custom behavior.

I'm asking for some practical example. I'm not sure how and when extension should use it over overriding specific translation key.

the fallback is meant to be used in case for whatever reason the locale value is not available so it won't produce an erroneous value. Language packs might not have it right away. Though I wonder if we really need it.

If language pack do not have specific key, then English translation will be used. AFAIK such fallback will be used only when extension will call formatDateTime() with key that is intentionally not defined in YAML file. But this is problematic for translators - Weblate won't detect this as phrase to translate and most translators will not be aware that they can adjust it. I don't think we should promote such usage of formatDateTime(), and I'm not sure why would you want to use fallback over defining translation key in YAML file.

SychO9 · 2024-10-02T19:56:21Z

If language pack do not have specific key, then English translation will be used.

Ah I see, I wasn't actually aware of that. Then yeah we can do without the fallback parameter indeed.

I'm asking for some practical example. I'm not sure how and when extension should use it over overriding specific translation key.

For example, this would allow building an extension that uses admin level defined formats or even user preference formats to use as a first priority. Then fallback to the default behavior which is locales.

framework/core/js/src/common/Translator.tsx

rob006 · 2024-10-02T20:21:35Z

For example, this would allow building an extension that uses admin level defined formats or even user preference formats to use as a first priority. Then fallback to the default behavior which is locales.

But these formats should be localized. I assume that such extension would have predefined formats defined as translations (so these could be translated by language packs). So this would result formatDateTime() call inside of formatCallback?

SychO9 · 2024-10-02T20:26:06Z

But these formats should be localized. I assume that such extension would have predefined formats defined as translations

No the use case would involve raw formats user-defined. Either through text or selection fields. The specifics would be up to the extension/developer's use case.

rob006 · 2024-10-02T20:32:56Z

@SychO9 So if I have forum that supports 10 languages, then I should have separate settings for these 10 languages in admin panel?

SychO9 · 2024-10-02T20:37:23Z

@rob006 if you want an extension like that, that's up to you. Not every forum needs more than 1 language, so for many it's a valid use case.

Point is, this is for custom behavior. Up to the developer/webmaster to determine what they want, up to the dev to make it happen. We just enable proper ways of extending behavior/components. Rather than leave things to hacking up the code. This enables us to make changes to the core code without worrying constantly about breaking backwards compatibility.

Co-authored-by: Sami Mazouz <sychocouldy@gmail.com>

rob006 · 2024-10-03T07:46:21Z

@SychO9 My point is: if we don't know exactly how it is supposed to be used, then maybe adding an API to cover use case we don't fully understand may be a dead end and will require some adjustments in the future (this may result a BC break). It is already possible to adjust date formats by overriding translations, so you don't even need dedicated extension, you can use Linguist for that. So dateTimeFormats is required to cover some edge case we don't know right now?
Personally I don't like that it do not involve currently used locale - it is easy to forget the fact that formats may differ for different languages. Passing locale to formatCallback() would at least suggest to this is something you should care about.

@YUCLing Can we have this PR against 1.x branch?

rob006 · 2024-10-03T07:54:15Z

OK, I checked new implementation and now I'm even more confused. I assumed that formatCallback() will get translation key (formatCallback('core.lib.datetime_formats.human_time_short')), so it could be used to overwrite translation for specific context. But it looks like it gets format taken from translation (formatCallback('ll')) - I have no idea what is practical use case for this.

SychO9 · 2024-10-03T07:58:50Z

@rob006 but I gave you a simple valid use case. For the user to be able to fill the formats themselves as a preference. I understand it is not something you personally see the point in. This is something I can do in other forum software as well and isn't all that odd.

Also, even without this a developer could still make it happen. But by modifying parts of the code that we consider private API. What we add here is a public API where even if we make changes to the code in the future, allows us to prevent BC breaks by handling the item list as necessary.

rob006 · 2024-10-03T08:13:37Z

For the user to be able to fill the formats themselves as a preference. I understand it is not something you personally see the point in. This is something I can do in other forum software as well and isn't all that odd.

And IMO this is a wrong place to cover this use case. I does not allow you to remove "2 minutes ago" format and use precise date instead (for example I can do this in phpBB). I we want the ability to change date format used by forum it makes more sense to override humanTime() instead of adding some hooks in formatDateTime().

rob006 · 2024-10-03T08:55:27Z

BTW: #4053 just got merged to 1.x branch, so I guess there is no point to rebase this PR and we can leave 2.x as target.

SychO9 · 2024-10-03T16:01:21Z

And IMO this is a wrong place to cover this use case. I does not allow you to remove "2 minutes ago" format and use precise date instead (for example I can do this in phpBB). I we want the ability to change date format used by forum it makes more sense to override humanTime() instead of adding some hooks in formatDateTime().

Sorry but that's just a different use case. It does not invalidate the simple use case of just wanting wanting to directly replace a specific format for a raw format value, which is possible regardless and is just a matter of how best to allow it being done.

Shouldn't we do the same for other formats in the codebase? I see a lot .format('LLLL') and .format('LLL') for example.

framework/core/js/src/common/Translator.tsx

rob006 · 2024-10-03T17:06:39Z

Sorry but that's just a different use case.

How is this different use case? If the goal is to allow customization of the date format used in the forum by a specific user, then dateTimeFormats would allow for the implementation of a half-way solution at best. Do we really need 3 official ways of adjusting date formats (1. by translation for simple cases, 2. by overriding humanTime() for advanced scenarios, 3. dateTimeFormats for something in the middle)?

SychO9 · 2024-10-03T18:37:53Z

I assumed that formatCallback() will get translation key

It should by the way. There is no point if it doesn't. Probably just a mistake.

Do we really need 3 official ways of adjusting date formats (1. by translation for simple cases, 2. by overriding humanTime() for advanced scenarios, 3. dateTimeFormats for something in the middle)?

You can't override functions in flarum, so I'm not following how that's related.

I'm not sure I understand why you are against the use case of: as a user I want to be able to customize the scrubber date format.

YUCLing · 2024-10-04T05:35:32Z

Shouldn't we do the same for other formats in the codebase? I see a lot .format('LLLL') and .format('LLL') for example.

My bad, I didn't search in tsx files.

Also, the formats in admin panel is currently not considered.

YUCLing marked this pull request as ready for review September 27, 2024 18:45

YUCLing requested a review from a team as a code owner September 27, 2024 18:45

YUCLing added 8 commits September 28, 2024 23:27

chore: humanTime improvements

46f7c86

chore: correct liveHumanTimes comment

eefb7cb

add: flarum customized time formats

57c2a73

chore: quote styles

f5eba9f

add: fallback for uppercased formats

9712a64

chore: fix code style

5262b72

fix: dayjsPlugins cannot be added to reg

8517f48

chore: code style formatting

5532684

ExerciseBook approved these changes Sep 28, 2024

View reviewed changes

YUCLing marked this pull request as draft September 29, 2024 15:15

YUCLing changed the title ~~feat: flarum customized time formats~~ WIP: feat: flarum customized time formats Sep 29, 2024

refactor: make the plugin extendable

6782636

chore: fetch format from translator

YUCLing force-pushed the im/time-formats branch from 0092d89 to 6782636 Compare September 29, 2024 15:47

SychO9 requested changes Oct 1, 2024

View reviewed changes

YUCLing changed the title ~~WIP: feat: flarum customized time formats~~ feat: flarum customized time formats Oct 2, 2024

fix: code format

b6bf4d0

SychO9 reviewed Oct 2, 2024

View reviewed changes

framework/core/js/src/common/Translator.tsx Outdated Show resolved Hide resolved

YUCLing and others added 2 commits October 3, 2024 09:49

remove: fallback parameter

2921372

Co-authored-by: Sami Mazouz <sychocouldy@gmail.com>

add: pass format to date time callback

5325305

YUCLing requested review from SychO9 and ExerciseBook October 3, 2024 05:48

YUCLing changed the title ~~feat: flarum customized time formats~~ feat: date time formats from locales Oct 3, 2024

ExerciseBook reviewed Oct 3, 2024

View reviewed changes

framework/core/js/src/common/Translator.tsx Show resolved Hide resolved

framework/core/js/src/common/Translator.tsx Outdated Show resolved Hide resolved

add: this arg for date time format callback

f89ab96

add: full time format

cd880d7

fix: code style

486de88

		@@ -0,0 +1,24 @@
		const customFormats: import('dayjs').PluginFunc = function (_option, c, _factory) {

feat: date time formats from locales #4029

Are you sure you want to change the base?

feat: date time formats from locales #4029

Conversation

YUCLing commented Sep 27, 2024 • edited Loading

ExerciseBook Sep 28, 2024

Choose a reason for hiding this comment

YUCLing Sep 29, 2024

Choose a reason for hiding this comment

ExerciseBook Sep 29, 2024

Choose a reason for hiding this comment

ExerciseBook Sep 28, 2024

Choose a reason for hiding this comment

YUCLing Sep 29, 2024

Choose a reason for hiding this comment

ExerciseBook Sep 29, 2024

Choose a reason for hiding this comment

ExerciseBook commented Sep 29, 2024

YUCLing commented Sep 29, 2024

ExerciseBook commented Sep 29, 2024

YUCLing commented Sep 29, 2024 • edited Loading

ExerciseBook commented Sep 29, 2024

rob006 commented Sep 29, 2024

YUCLing commented Sep 29, 2024

rob006 commented Sep 29, 2024

YUCLing commented Sep 29, 2024 • edited Loading

YUCLing commented Sep 29, 2024

YUCLing commented Sep 29, 2024

rob006 commented Sep 29, 2024

YUCLing commented Sep 30, 2024

SychO9 left a comment • edited Loading

Choose a reason for hiding this comment

rob006 commented Oct 1, 2024

YUCLing commented Oct 2, 2024

rob006 commented Oct 2, 2024

SychO9 commented Oct 2, 2024

rob006 commented Oct 2, 2024

SychO9 commented Oct 2, 2024

rob006 commented Oct 2, 2024

SychO9 commented Oct 2, 2024

rob006 commented Oct 2, 2024

SychO9 commented Oct 2, 2024 • edited Loading

rob006 commented Oct 3, 2024

rob006 commented Oct 3, 2024

SychO9 commented Oct 3, 2024

rob006 commented Oct 3, 2024 • edited Loading

rob006 commented Oct 3, 2024

SychO9 commented Oct 3, 2024

rob006 commented Oct 3, 2024

SychO9 commented Oct 3, 2024

YUCLing commented Oct 4, 2024 • edited Loading

YUCLing commented Sep 27, 2024 •

edited

Loading

YUCLing commented Sep 29, 2024 •

edited

Loading

YUCLing commented Sep 29, 2024 •

edited

Loading

SychO9 left a comment •

edited

Loading

SychO9 commented Oct 2, 2024 •

edited

Loading

rob006 commented Oct 3, 2024 •

edited

Loading

YUCLing commented Oct 4, 2024 •

edited

Loading