Add tool for building DEB/RPM packages

[weiss]

Add make-binaries and make-packages scripts to the tools directory:

• make-binaries creates self-contained OTP releases for Linux/x64 and Linux/arm64 systems. All optional ejabberd features are fully supported. Erlang/OTP, Elixir, and all dependencies are retrieved and built from source (using crosstool-NG). The generated binaries depend only on glibc 2.17+, the other dependencies are linked statically. The script should work on any non-ancient Linux/x64 system and requires just a few common C development tools (to bootstrap crosstool-NG).
• make-packages creates DEB and RPM packages from the archives generated by make-binaries. It requires fpm to be in the $PATH (can be installed below $HOME/.gem/ruby by calling gem install --no-document --user-install fpm).

Note: As dependencies are linked statically, it's crucial to track updates (e.g., by subscribing to OpenSSL's announcement list, to GitHub release notifications for Expat, and so on), so that the packages can be updated in case of security issues (just bump the iteration by running e.g. tools/make-packages -i 2 and tools/make-installers -i 2 if the ejabberd version didn't change). The make-binaries script checks most dependency versions to make sure they're up-to-date unless CHECK_DEPS=false is set.

Also note that running the scripts can take several hours, at least on first invocation. Subsequent invocations will be significantly faster if the BUILD_DIR is kept, as the crosstool-NG toolchains are reused. (However, all other dependencies are rebuilt from scratch if any of their versions was bumped, as handling dependency updates would be a bit cumbersome with static linking.)

[coveralls]

Coverage remained the same at 33.598% when pulling 0924415 on weiss:feature/binary-packages into 2739a52 on processone:master.

[weiss]

I've now pushed another commit to add tools/make-installers. That script creates simple command line installers from the archives generated by make-binaries. It requires makeself to be in the $PATH (available as Debian package).

The installers are used like this:

chmod +x ejabberd-21.12-1-linux-x64.run
sudo ./ejabberd-21.12-1-linux-x64.run

The installation is removed like this:

sudo systemctl --now disable ejabberd
sudo rm -rf /opt/ejabberd* /etc/systemd/system/ejabberd.service

If the installer is called from an interactive terminal, it will mention the installation paths and ask for confirmation (but there's no way to adjust the paths, to keep things simple). If it's running unattended, ejabberd is installed without asking questions. If the data directory (/opt/ejabberd) already exists, it won't be touched.

See https://ejabberd.messaging.one for examples of those binary packages.

[weiss]

I've now added a GitHub Action for creating packages from each commit. Allows for easy testing of development versions and makes sure problems with building binaries are noticed early on. (But this is in a separate commit, so it can easily be reverted if it's not desired.)

[sando38]

Hi Holger,
I suppose the packages, which would be created through this pull-request, are build in the same way as here:
https://ejabberd.messaging.one/

I installed them on a raspberry pi, it works also as intended. However, if I try to install a additional module like mod_cron or ejabberd_auth_http, it throws an error. I also tried it on an AMD64 architecture effectively giving the same output (s. below):

root@ejabberd:/opt/ejabberd# /opt/ejabberd-21.12/bin/ejabberdctl module_install mod_cron
/opt/ejabberd/.ejabberd-modules/sources/ejabberd-contrib/mod_cron/src/mod_cron.erl:21:10: can't find include file "ejabberd_commands.hrl"
%   21| -include("ejabberd_commands.hrl").
%     |          ^

/opt/ejabberd/.ejabberd-modules/sources/ejabberd-contrib/mod_cron/src/mod_cron.erl:22:10: can't find include file "ejabberd_http.hrl"
%   22| -include("ejabberd_http.hrl").
%     |          ^

/opt/ejabberd/.ejabberd-modules/sources/ejabberd-contrib/mod_cron/src/mod_cron.erl:23:10: can't find include file "ejabberd_web_admin.hrl"
%   23| -include("ejabberd_web_admin.hrl").
%     |          ^

/opt/ejabberd/.ejabberd-modules/sources/ejabberd-contrib/mod_cron/src/mod_cron.erl:24:10: can't find include file "logger.hrl"
%   24| -include("logger.hrl").
%     |          ^

/opt/ejabberd/.ejabberd-modules/sources/ejabberd-contrib/mod_cron/src/mod_cron.erl:25:10: can't find include file "translate.hrl"
%   25| -include("translate.hrl").
%     |          ^

/opt/ejabberd/.ejabberd-modules/sources/ejabberd-contrib/mod_cron/src/mod_cron.erl:114:6: undefined macro 'DEBUG/2'
%  114|     ?DEBUG("MS To Go Fixed: ~p ~p", [MSToGo1, MSToGo2]),
%     |      ^

/opt/ejabberd/.ejabberd-modules/sources/ejabberd-contrib/mod_cron/src/mod_cron.erl:217:7: undefined macro 'ERROR_MSG/2'
%  217|             ?ERROR_MSG("Error in scheduled task ~p:~p~p:~n~p", [Mod, Fun, Args, Reason]);
%     |              ^

/opt/ejabberd/.ejabberd-modules/sources/ejabberd-contrib/mod_cron/src/mod_cron.erl:278:46: undefined macro 'T/1'
%  278|     [{<<"cron">>, translate:translate(Lang, ?T("Cron Tasks"))} | Acc].
%     |                                              ^

/opt/ejabberd/.ejabberd-modules/sources/ejabberd-contrib/mod_cron/src/mod_cron.erl:285:13: undefined macro 'XC/2'
%  285|     Res = [?XC(<<"h1">>, <<"Cron Tasks">>)] ++ Tasks_table,
%     |             ^

/opt/ejabberd/.ejabberd-modules/sources/ejabberd-contrib/mod_cron/src/mod_cron.erl:295:10: undefined macro 'XE/2'
%  295|                       ?XE(<<"tr">>,
%     |                        ^

/opt/ejabberd/.ejabberd-modules/sources/ejabberd-contrib/mod_cron/src/mod_cron.erl:15:2: function run_task/3 undefined
%   15| -export([cron_list/1, cron_del/1,
%     |  ^

/opt/ejabberd/.ejabberd-modules/sources/ejabberd-contrib/mod_cron/src/mod_cron.erl:15:2: function web_menu_host/3 undefined
%   15| -export([cron_list/1, cron_del/1,
%     |  ^

/opt/ejabberd/.ejabberd-modules/sources/ejabberd-contrib/mod_cron/src/mod_cron.erl:15:2: function web_page_host/3 undefined
%   15| -export([cron_list/1, cron_del/1,
%     |  ^

/opt/ejabberd/.ejabberd-modules/sources/ejabberd-contrib/mod_cron/src/mod_cron.erl:125:5: function run_task/3 undefined
%  125|     run_task(M, F, A),
%     |     ^

/opt/ejabberd/.ejabberd-modules/sources/ejabberd-contrib/mod_cron/src/mod_cron.erl:166:24: function begin_fixed_timer/4 undefined
%  166|                        begin_fixed_timer(TaskId, TimeUnit, TimeNum, [Mod, Fun, Args])
%     |                        ^

/opt/ejabberd/.ejabberd-modules/sources/ejabberd-contrib/mod_cron/src/mod_cron.erl:252:6: record ejabberd_commands undefined
%  252|      #ejabberd_commands{name = cron_list, tags = [cron],
%     |      ^

/opt/ejabberd/.ejabberd-modules/sources/ejabberd-contrib/mod_cron/src/mod_cron.erl:257:6: record ejabberd_commands undefined
%  257|      #ejabberd_commands{name = cron_del, tags = [cron],
%     |      ^

Error: error
Error: {compilation_failed,"/opt/ejabberd/.ejabberd-modules/sources/ejabberd-contrib/mod_cron/src/mod_cron.erl"}

I tried to check if a package is missing, but could not discover anything.
Maybe you have an idea. I ran the ejabberdctl module_install mod_cron command on a plain install.

Thanks and wishing a great day!

[weiss]

Thank you for reporting this issue! I force-pushed a fix into this PR and updated the packages on https://ejabberd.messaging.one.

[sando38]

yes, works like a charm!