[Proposal] Keeping translations up-to-date

[canpolat]

The English version of the documentation is changing over time and it is not easy to keep track of all changes that need to be reflected in various translations. Sometimes, the translation is so old that a whole file needs a walk-through, but often, only a couple of sentences change...

I thought it may be useful to have a way to keep track of incremental updates. Probably, there are tools out there to do this easily, but I don't know any of them. So, I wrote a hack-y Powershell script to output the changes in documentation.

The idea is to do some comparisons file by file. For each file, I find the latest commit hashes for the English and translated version (for each language). Then I check if the English version has been updated since that last commit on the translated file. If there is a diff, then I display it as a change, otherwise, I document that there are no changes.

Below is a table showing the diffs for French translation files. Unfortunately, GitHub does not seem to provide a way to compare two versions of a single file, so I included a Git command to see the diff locally. When you issue the Git command, the diff for the English version will be displayed in the configured external diff tool. This way one can see what needs an update.

File	Translated version	English version	?	Git command to get the diff
/core/export.md	48f3bc7	afc4f7f	±	git difftool -y 48f3bc7 afc4f7f -- docs/en/core/export.md
/concepts/roots.md	Not Found	8125fa4	☉	N/A - no existing translation
/faq.md	01d2e71	2d99339	±	git difftool -y 01d2e71 2d99339 -- docs/en/faq.md
/core/file-list.md	8dafd45	afc4f7f	✓	No change between versions
/concepts/wysiwym.md	Not Found	9a1228b	☉	N/A - no existing translation
/core/search.md	48f3bc7	afc4f7f	±	git difftool -y 48f3bc7 afc4f7f -- docs/en/core/search.md
/core/attachments.md	48f3bc7	9a1228b	±	git difftool -y 48f3bc7 9a1228b -- docs/en/core/attachments.md
/core/import.md	c0a6b7a	9a1228b	✓	No change between versions
/core/math.md	6f5d3e8	6f5d3e8	✓	No change between versions
/core/localisation.md	6f5d3e8	afc4f7f	✓	No change between versions
/guides/guide-zettelkasten.md	48f3bc7	ccf3cda	±	git difftool -y 48f3bc7 ccf3cda -- docs/en/guides/guide-zettelkasten.md
/academic/projects.md	48f3bc7	742a99a	±	git difftool -y 48f3bc7 742a99a -- docs/en/academic/projects.md
/reference/keyboard-shortcuts.md	Not Found	6a9b0c4	☉	N/A - no existing translation
/get-involved.md	1116a0d	023e038	±	git difftool -y 1116a0d 023e038 -- docs/en/get-involved.md
/core/custom-css.md	7765a61	afa8763	±	git difftool -y 7765a61 afa8763 -- docs/en/core/custom-css.md
/academic/custom-templates.md	48f3bc7	742a99a	±	git difftool -y 48f3bc7 742a99a -- docs/en/academic/custom-templates.md
/reference/markdown-basics.md	5e64283	6a9b0c4	±	git difftool -y 5e64283 6a9b0c4 -- docs/en/reference/markdown-basics.md
/core/editor.md	6f5d3e8	d3392c9	±	git difftool -y 6f5d3e8 d3392c9 -- docs/en/core/editor.md
/academic/pomodoro.md	48f3bc7	742a99a	±	git difftool -y 48f3bc7 742a99a -- docs/en/academic/pomodoro.md
/reference/spell-checking.md	48f3bc7	6a9b0c4	±	git difftool -y 48f3bc7 6a9b0c4 -- docs/en/reference/spell-checking.md
/installing-latex.md	Not Found	023e038	☉	N/A - no existing translation
/install.md	c3f66d3	023e038	±	git difftool -y c3f66d3 023e038 -- docs/en/install.md
/concepts/context.md	Not Found	9a1228b	☉	N/A - no existing translation
/core/tabs.md	1c3f001	afc4f7f	✓	No change between versions
/index.md	6f5d3e8	616b743	✓	No change between versions
/core/print-preview.md	d8faf21	220162a	✓	No change between versions
/academic/readability.md	48f3bc7	742a99a	±	git difftool -y 48f3bc7 742a99a -- docs/en/academic/readability.md
/installing-pandoc.md	Not Found	023e038	☉	N/A - no existing translation
/core/tables.md	48f3bc7	afc4f7f	±	git difftool -y 48f3bc7 afc4f7f -- docs/en/core/tables.md
/guides/guide-ide.md	7765a61	ccf3cda	±	git difftool -y 7765a61 ccf3cda -- docs/en/guides/guide-ide.md
/reference/settings.md	48f3bc7	6a9b0c4	±	git difftool -y 48f3bc7 6a9b0c4 -- docs/en/reference/settings.md
/academic/presentations.md	48f3bc7	742a99a	±	git difftool -y 48f3bc7 742a99a -- docs/en/academic/presentations.md
/assets/app.css	48f3bc7	33c431f	✓	No change between versions
/core/autocorrect.md	6be9f1f	bd1f1ee	±	git difftool -y 6be9f1f bd1f1ee -- docs/en/core/autocorrect.md
/academic/citations.md	48f3bc7	742a99a	±	git difftool -y 48f3bc7 742a99a -- docs/en/academic/citations.md
/get-started.md	Not Found	c16ce75	☉	N/A - no existing translation
/academic/zkn-method.md	48f3bc7	22d8ba7	±	git difftool -y 48f3bc7 22d8ba7 -- docs/en/academic/zkn-method.md
/core/yaml-frontmatter.md	f205013	19cf428	±	git difftool -y f205013 19cf428 -- docs/en/core/yaml-frontmatter.md
/guides/guide-notes.md	41d6ed7	ccf3cda	±	git difftool -y 41d6ed7 ccf3cda -- docs/en/guides/guide-notes.md

[nathanlesage]

An easier algorithm would be to simply compare file timestamps: If a file of any language has been changed after the English pendant, this is a strong indicator that the file is up to date, whereas if any file has been changed and the English pendant is more recent, that there are changes that still need to be incorporated?

I'm heavily in favor of automation that makes use of GitHub actions, but I'm pretty much against some shell-script magic (less because it doesn't work, but rather because it keeps everything maintainable…)

[canpolat]

An easier algorithm would be to simply compare file timestamps: If a file of any language has been changed after the English pendant, this is a strong indicator that the file is up to date

Yes, this is the simpler of the two scenarios and timestamps provide enough data to say "translation file is most likely up-to-date" (we cannot really know the state of the translation unless we look at the contents, anyway).

whereas if any file has been changed and the English pendant is more recent, that there are changes that still need to be incorporated?

Yeah, things get complicated here. Timestamp will only indicate that there is a change. But to be able to know if it was fixing of a typo, a complete rewrite of the file, or something in between, one needs to look at the diffs. I don't see any other way around it (unless we expect the translators to compare the whole translation file to the English version sentence by sentence). And Git is quite handy here. It provides all the information needed. One can see exactly what has changed in the English version and easily estimate the amount of work that needs to be done.

I'm heavily in favor of automation that makes use of GitHub actions, but I'm pretty much against some shell-script magic (less because it doesn't work, but rather because it keeps everything maintainable…)

There is nothing "magical" about this. The script is very simple and just calls Git to gather the data and present in a human-readable format. As I said, unless the translator is willing to compare whole files after the English version is changed, I don't see a way to keep the translations up-to-date where Git is not involved. And I think, it would have been easier if there was a place where there was an always up-to-date state of all translations (like in the case of UI translations).

But... I don't contribute to any of the translations. So, all that said, it's up to the people doing the translations if they want to use this or not. The script is there. You can close this issue if you don't see the value.

[nathanlesage]

And I think, it would have been easier if there was a place where there was an always up-to-date state of all translations (like in the case of UI translations).

Absolutely, I even thought about that, but translations are coherent pages of a lot of text, and thus it's easier to maintain them through git …

I mean, very basically, after each release the translators only need to go through the Changelog of the app — I document absolutely every new and changed behaviour in there, it's the most complete and most comprehensive information for what has changed, and what has stayed the same. Maybe we could do it this way, that I'll just paste the changelog in here for others to refer to and apply the changes …?

[canpolat]

Absolutely, I even thought about that, but translations are coherent pages of a lot of text, and thus it's easier to maintain them through git …

I don't really understand this argument because what the script does is use Git in an automated way to prevent manual mistakes. The whole point of the script is to construct the correct Git command to show the diffs. That's literally the only thing i does.

Anyway... As I stated in my previous comment, I don't contribute to any translations, so I don't think it makes sense for me to drag this discussion on. I'm bowing out.