Non recursive automake and Debian packaging changes #700

saiarcot895 · 2022-10-18T19:37:49Z

This PR brings in three major changes:

Switch to using non-recursive automake. The primary advantage of this is that builds for source files across subdirectories can happen in parallel, which reduces the build time on systems with sufficient cores. On my dev VM, the build time went from about 2:55 to 2:28 for the Bullseye build.
Update the Debian packaging files to a more recent syntax/features supported in Debian Buster (and newer).
Disable building static libraries. The build time dropped to about 1:30 on my dev VM.

This can help increase build parallelization. On my dev VM, this decreased the compilation time from 2:55 to about 2:28 (Bullseye, Python 2 disabled, -j12). Time improvements may be more noticeable on systems with more CPU cores. Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>

Update to using debhelper compat 12, remove some overrides that are no longer needed, use automatically-generated dbgsym packages, and disable the static build. Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>

Make sure hardened build flags are always used. Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>

…iners have that installed instead of just libboost-dev Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>

qiluo-msft

LGTM. Please wait for other reviewers.

Update sonic-swss-common submodule pointer to include the following: * 3352881 Prevent sonic-db-cli generate core dump ([sonic-net#749](sonic-net/sonic-swss-common#749)) * 43cadec Add ProfileProvider class to support read profile config from PROFILE_DB. ([sonic-net#683](sonic-net/sonic-swss-common#683)) * 8b09f90 Update path to sairedis tests ([sonic-net#747](sonic-net/sonic-swss-common#747)) * 85f3776 Non recursive automake and Debian packaging changes ([sonic-net#700](sonic-net/sonic-swss-common#700)) Signed-off-by: AntonHryshchuk <antonh@nvidia.com>

Update sonic-swss-common submodule pointer to include the following: * 565ad4b Fix common path issue ([sonic-net#751](sonic-net/sonic-swss-common#751)) * 3352881 Prevent sonic-db-cli generate core dump ([sonic-net#749](sonic-net/sonic-swss-common#749)) * 43cadec Add ProfileProvider class to support read profile config from PROFILE_DB. ([sonic-net#683](sonic-net/sonic-swss-common#683)) * 8b09f90 Update path to sairedis tests ([sonic-net#747](sonic-net/sonic-swss-common#747)) * 85f3776 Non recursive automake and Debian packaging changes ([sonic-net#700](sonic-net/sonic-swss-common#700)) Signed-off-by: dprital <drorp@nvidia.com>

Update sonic-swss-common submodule pointer to include the following: * 565ad4b Fix common path issue ([#751](sonic-net/sonic-swss-common#751)) * 3352881 Prevent sonic-db-cli generate core dump ([#749](sonic-net/sonic-swss-common#749)) * 43cadec Add ProfileProvider class to support read profile config from PROFILE_DB. ([#683](sonic-net/sonic-swss-common#683)) * 8b09f90 Update path to sairedis tests ([#747](sonic-net/sonic-swss-common#747)) * 85f3776 Non recursive automake and Debian packaging changes ([#700](sonic-net/sonic-swss-common#700)) Signed-off-by: dprital <drorp@nvidia.com>

Update sonic-swss-common submodule pointer to include the following: * 565ad4b Fix common path issue ([sonic-net#751](sonic-net/sonic-swss-common#751)) * 3352881 Prevent sonic-db-cli generate core dump ([sonic-net#749](sonic-net/sonic-swss-common#749)) * 43cadec Add ProfileProvider class to support read profile config from PROFILE_DB. ([sonic-net#683](sonic-net/sonic-swss-common#683)) * 8b09f90 Update path to sairedis tests ([sonic-net#747](sonic-net/sonic-swss-common#747)) * 85f3776 Non recursive automake and Debian packaging changes ([sonic-net#700](sonic-net/sonic-swss-common#700)) This is a reland of sonic-net#13950, with the debug image build fix. Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>

Update sonic-swss-common submodule pointer to include the following: * 06ffb51 Define ACL_TABLE and ACL_RULE table in STATE_DB ([sonic-net#748](sonic-net/sonic-swss-common#748)) * 1b369ab [ci] Fix apt-get install unable locate package issue. ([sonic-net#753](sonic-net/sonic-swss-common#753)) * 619d4ec Improve unit test for go wrapper ([sonic-net#752](sonic-net/sonic-swss-common#752)) * 565ad4b Fix common path issue ([sonic-net#751](sonic-net/sonic-swss-common#751)) * 3352881 Prevent sonic-db-cli generate core dump ([sonic-net#749](sonic-net/sonic-swss-common#749)) * 43cadec Add ProfileProvider class to support read profile config from PROFILE_DB. ([sonic-net#683](sonic-net/sonic-swss-common#683)) * 8b09f90 Update path to sairedis tests ([sonic-net#747](sonic-net/sonic-swss-common#747)) * 85f3776 Non recursive automake and Debian packaging changes ([sonic-net#700](sonic-net/sonic-swss-common#700)) Signed-off-by: AntonHryshchuk <antonh@nvidia.com>

saiarcot895 · 2023-03-09T00:07:04Z

Requesting cherry-pick to 202205 and 202211 for space reduction purposes (removal of static libraries).

Update sonic-swss-common submodule pointer to include the following: * 565ad4b Fix common path issue ([sonic-net#751](sonic-net/sonic-swss-common#751)) * 3352881 Prevent sonic-db-cli generate core dump ([sonic-net#749](sonic-net/sonic-swss-common#749)) * 43cadec Add ProfileProvider class to support read profile config from PROFILE_DB. ([sonic-net#683](sonic-net/sonic-swss-common#683)) * 8b09f90 Update path to sairedis tests ([sonic-net#747](sonic-net/sonic-swss-common#747)) * 85f3776 Non recursive automake and Debian packaging changes ([sonic-net#700](sonic-net/sonic-swss-common#700)) Signed-off-by: dprital <drorp@nvidia.com>

yxieca · 2023-03-15T16:47:42Z

@saiarcot895 please create separate PR for 202205 branch.

* Switch to using non-recursive automake This can help increase build parallelization. On my dev VM, this decreased the compilation time from 2:55 to about 2:28 (Bullseye, Python 2 disabled, -j12). Time improvements may be more noticeable on systems with more CPU cores. Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com> * Update debian packaging Update to using debhelper compat 12, remove some overrides that are no longer needed, use automatically-generated dbgsym packages, and disable the static build. Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com> * Always enable all hardening Make sure hardened build flags are always used. Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com> * Re-add libboost1.71-dev as a dependency, since the Buster slave containers have that installed instead of just libboost-dev Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com> --------- Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>

…t#700)" This reverts commit 7f40cde.

Update sonic-swss-common submodule pointer to include the following: * 6e4daf1 Revamp module build script to make it work for 5.15 on Ubuntu 20.04 (sonic-net/sonic-swss-common#720) * 7f40cde Non recursive automake and Debian packaging changes (sonic-net/sonic-swss-common#700) Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>

…et#700) * Switch to using non-recursive automake This can help increase build parallelization. On my dev VM, this decreased the compilation time from 2:55 to about 2:28 (Bullseye, Python 2 disabled, -j12). Time improvements may be more noticeable on systems with more CPU cores. Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com> * Update debian packaging Update to using debhelper compat 12, remove some overrides that are no longer needed, use automatically-generated dbgsym packages, and disable the static build. Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com> * Always enable all hardening Make sure hardened build flags are always used. Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com> * Re-add libboost1.71-dev as a dependency, since the Buster slave containers have that installed instead of just libboost-dev Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com> --------- Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>

…772) Cherry-pick of #700 Switch to using non-recursive automake This can help increase build parallelization. On my dev VM, this decreased the compilation time from 2:55 to about 2:28 (Bullseye, Python 2 disabled, -j12). Time improvements may be more noticeable on systems with more CPU cores. Update debian packaging Update to using debhelper compat 12, remove some overrides that are no longer needed, use automatically-generated dbgsym packages, and disable the static build. Always enable all hardening Make sure hardened build flags are always used. Re-add libboost1.71-dev as a dependency, since the Buster slave containers have that installed instead of just libboost-dev

Update sonic-swss-common submodule pointer to include the following: * 6e4daf1 Revamp module build script to make it work for 5.15 on Ubuntu 20.04 (sonic-net/sonic-swss-common#720) * 7f40cde Non recursive automake and Debian packaging changes (sonic-net/sonic-swss-common#700) Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>

saiarcot895 added 4 commits October 18, 2022 17:18

Update debian packaging

e832de1

Update to using debhelper compat 12, remove some overrides that are no longer needed, use automatically-generated dbgsym packages, and disable the static build. Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>

Always enable all hardening

ff8155d

Make sure hardened build flags are always used. Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>

Re-add libboost1.71-dev as a dependency, since the Buster slave conta…

1404402

…iners have that installed instead of just libboost-dev Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>

saiarcot895 force-pushed the non-recursive-automake branch from 80c3cf2 to 1404402 Compare October 19, 2022 00:19

saiarcot895 requested a review from qiluo-msft October 20, 2022 15:40

qiluo-msft requested review from liuh-80 and zbud-msft October 22, 2022 00:17

qiluo-msft previously approved these changes Oct 22, 2022

View reviewed changes

Merge remote-tracking branch 'origin/master' into non-recursive-automake

635ec3a

saiarcot895 dismissed qiluo-msft’s stale review via 635ec3a January 7, 2023 00:23

qiluo-msft approved these changes Feb 10, 2023

View reviewed changes

saiarcot895 merged commit 85f3776 into sonic-net:master Feb 10, 2023

saiarcot895 deleted the non-recursive-automake branch February 10, 2023 23:19

saiarcot895 mentioned this pull request Feb 10, 2023

Exclude *.a files from python deb packages #553

Closed

dprital mentioned this pull request Feb 23, 2023

[submodule] Advance sonic-swss-common pointer sonic-net/sonic-buildimage#13950

Merged

8 tasks

saiarcot895 mentioned this pull request Mar 7, 2023

[submodule] Advance sonic-swss-common pointer sonic-net/sonic-buildimage#14142

Merged

8 tasks

saiarcot895 added Request for 202205 Branch Request for 202211 Branch labels Mar 9, 2023

yxieca added the Cherry Pick Conflict_202205 label Mar 15, 2023

StormLiangMS added the Included in 202211 Branch label Mar 19, 2023

keboliu added a commit to keboliu/sonic-swss-common that referenced this pull request Apr 1, 2023

Revert "Non recursive automake and Debian packaging changes (sonic-ne…

f3157e6

…t#700)" This reverts commit 7f40cde.

saiarcot895 mentioned this pull request Apr 3, 2023

[202211][submodule] Advance sonic-swss-common pointer sonic-net/sonic-buildimage#14504

Merged

8 tasks

saiarcot895 mentioned this pull request Apr 5, 2023

[202205] Non recursive automake and Debian packaging changes (#700) #772

Merged

yxieca added the Included in 202205 Branch label Apr 6, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Non recursive automake and Debian packaging changes #700

Non recursive automake and Debian packaging changes #700

saiarcot895 commented Oct 18, 2022 •

edited

Loading

qiluo-msft left a comment

saiarcot895 commented Mar 9, 2023

yxieca commented Mar 15, 2023

Non recursive automake and Debian packaging changes #700

Non recursive automake and Debian packaging changes #700

Conversation

saiarcot895 commented Oct 18, 2022 • edited Loading

qiluo-msft left a comment

Choose a reason for hiding this comment

saiarcot895 commented Mar 9, 2023

yxieca commented Mar 15, 2023

saiarcot895 commented Oct 18, 2022 •

edited

Loading