Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Try to reduce image size #5748

Closed
wants to merge 46 commits into from
Closed

Conversation

koppor
Copy link
Member

@koppor koppor commented Dec 16, 2019

Use rsync to speed up uploads.

Enable the "old" upload-script. The advantage of the script is to create symlinks to the latest files.

Current issue:

Connected to build-upload.jabref.org.
sftp> cd www/
sftp> mkdir change-upload-back-to-script
Couldn't create directory: Failure
sftp> cd change-upload-back-to-script
Couldn't canonicalize: No such file or directory

Google does not say something specific about "Failure". I was in the server, access rights are OK. Since the last working execution of the script, I security-updated the server software. Maybe, there was a change.

Alternative to fix the upload issues. Https://github.com/appleboy/scp-action (does not create symlinks, too; but tries the uploads more harder)

@Siedlerchr
Copy link
Member

That sh script works also under windows?
if yes that's good

@tobiasdiez
Copy link
Member

If the script does not want to work, then maybe it would be also an option to manually create the symlinks (since the file names only change after a release which is not that often).

@@ -5,7 +5,7 @@
echo "Using branch $branch"

command="cd www/\n"
command="${command}mkdir $branch\ncd $branch\n"
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure whether -p works on sftp

Furthermore, the parent directory exists.

We have three (!) parallel builds: windows, linux, mac. Each of that uploads to the SAME directory. Thus, the second and the third one cannot create the directory. cd works fine (see output)

The \n is a newline character. It separates the command on SFTP (=secure FTP). So, with your change, SPACEcd is executed instead of plain cd. Think, this is unintended?

@koppor koppor force-pushed the change-upload-back-to-script branch 2 times, most recently from 2306188 to 3244cf2 Compare December 18, 2019 21:08
@koppor koppor force-pushed the change-upload-back-to-script branch from b56bca1 to 54fe80d Compare December 19, 2019 19:50
@koppor koppor changed the title Switch back to upload-to-builds.jabref.org.sh (to enable symlinks) Switch to rsync for upload (to keep old files during upload) Dec 19, 2019
@koppor
Copy link
Member Author

koppor commented Dec 20, 2019

Even though the sources are the same for the build, the binary files differ.

Total bytes sent: 1,562,723,021
Total bytes received: 1,018,290

sent 1,562,723,021 bytes  received 1,018,290 bytes  13,657,129.35 bytes/sec
total size is 2,647,503,235  speedup is 1.69

Some difference example:

  1064536  2019-12-19 23:32   JabRef/runtime/lib/jvm.lib      |   1064536  2019-12-20 01:14   JabRef/runtime/lib/jvm.lib
186273910  2019-12-19 23:32   JabRef/runtime/lib/modules      | 186273746  2019-12-20 01:14   JabRef/runtime/lib/modules

The timestamp differs (this is OK; first fie). However the modules also differ (second line). I wonder why the build is non-deterministic.

@koppor koppor marked this pull request as ready for review December 20, 2019 01:36
@koppor koppor added the status: ready-for-review Pull Requests that are ready to be reviewed by the maintainers label Dec 20, 2019
@koppor koppor removed the status: ready-for-review Pull Requests that are ready to be reviewed by the maintainers label Dec 20, 2019
@koppor koppor changed the title [WIP!] Switch to rsync for upload (to keep old files during upload) [WIP!] Try to reduce image size (and switch to rsync for upload (to keep old files during upload)) Dec 21, 2019
@koppor koppor mentioned this pull request Dec 21, 2019
@koppor koppor force-pushed the master branch 5 times, most recently from b8ef7b7 to 21c6e5e Compare March 4, 2020 17:02
@koppor koppor changed the title [WIP!] Try to reduce image size (and switch to rsync for upload (to keep old files during upload)) [WIP] Try to reduce image size Mar 21, 2020
@tobiasdiez
Copy link
Member

@koppor is this PR still required in view of #6711 ?

@koppor koppor closed this Jul 27, 2020
@koppor
Copy link
Member Author

koppor commented Jul 27, 2020

No, it is not. Thank you for linking

@koppor koppor deleted the change-upload-back-to-script branch July 27, 2020 20:19
koppor pushed a commit that referenced this pull request Dec 15, 2021
60bf7d5 Add encyclopedia type to wikipedia-templates.csl (#5778)
031afe1 Update and rename dependent/organization-studies.csl to organization-… (#5779)
7ed71e7 Update harvard-newcastle-university.csl (#5765)
46bab91 Update annals-of-oncology.csl (#5760)
6158ae6 Create research-in-plant-disease.csl (#5738)
04422a8 Create chemmedchem.csl (#5753)
7c11521 Create clinical-kidney-journal.csl (#5749)
e7ee983 Create kit-karlsruher-institut-fur-technologie-germanistik-ndl-neuere… (#5729)
96a1106 Update article format for STM journal (#5755)
a4ca057 Update historia-scribere.csl (#5748)

git-subtree-dir: buildres/csl/csl-styles
git-subtree-split: 60bf7d5
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Decrease size of distribution
3 participants