Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[baseimage] set default locale en_US.UTF-8 #1988

Merged
merged 4 commits into from
Sep 11, 2018
Merged

[baseimage] set default locale en_US.UTF-8 #1988

merged 4 commits into from
Sep 11, 2018

Conversation

richard28530
Copy link
Contributor

Signed-off-by: chenhu chenhu@didichuxing.com

- What I did
set default locale en_US.UTF-8
to avoid client (SSH) inject locale while login to switch

- How I did it

- How to verify it
login from a machine with non en_US.UTF-8
verify 'env' command output to check 'LANG' environment variable
use 'TAB' to do the auto-completion

- Description for the changelog

Signed-off-by: chenhu <chenhu@didichuxing.com>
@qiluo-msft
Copy link
Collaborator

We are conservative on disk space, so we originally remove the locale files. I found another solution to solve the issue #1681 without any new packages.

You just need to create a new configuration file like:

sudo vi /etc/profile.d/locale.sh
export LANG=C
export LANGUAGE=C

logout and login. You will have no LANG injection from ssh client.
However, I notice the Debian Stretch bash prompt's color is lost. I don't find the clue right now...

@lguohan
Copy link
Collaborator

lguohan commented Aug 27, 2018

how large is the locale file?

@richard28530
Copy link
Contributor Author

/usr/share/locale$ du -d 0 -h
50M .

@lguohan
Copy link
Collaborator

lguohan commented Aug 27, 2018

damn, it is pretty large, we are still dealing with devices with 2G disk which is quite challenging. any idea how can we get it shrink if we only care about en_US.UTF-8?

@sergeypopovich-ord
Copy link

sergeypopovich-ord commented Aug 29, 2018

How about localepurge Debian package? It's purpose to remove all unused locale message files and other stuff. While it is a hack it might be very useful to conserve space:

# localepurge
localepurge: Disk space freed in /usr/share/locale: 86292 KiB
localepurge: Disk space freed in /usr/share/man: 7088 KiB

Total disk space freed by localepurge: 93380 KiB

On test system after installation and manual run.

Also I think it is a good idea to run update-locale LANG=en_US.UTF-8 after we locale-gen en_US.UTF-8 to update /etc/default/locale.

@richard28530
Copy link
Contributor Author

richard28530 commented Aug 30, 2018

/usr/share/locale
/usr/share/man
already cleaned in the previous version of build_debian.sh

/usr/share/i18n/locales/
is cleaned after install & update locales, about 9M
other files in locales.deb is very small, less than 1M

build_debian.sh Outdated

#Adds a locale to a debian system in non-interactive mode
sudo sed -i '/^#.* en_US.* /s/^#//' $FILESYSTEM_ROOT/etc/locale.gen && \
sudo sed -i "\$aLANG=en_US.UTF-8" $FILESYSTEM_ROOT/etc/default/locale && \
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe it is better to use update-locale after locale-gen here?

https://www.thomas-krenn.com/en/wiki/Configure_Locales_in_Ubuntu

@lguohan
Copy link
Collaborator

lguohan commented Aug 30, 2018

looks like the disk is not an issue, what about the update-locale suggested by sergey?

@richard28530
Copy link
Contributor Author

Adjusting Locales
The default settings are stored in the /etc/default/locale file.

$ cat /etc/default/locale
LANG=en_US.UTF-8
This file can either be adjusted manually or updated using the tool, update-locale.

update-locale LANG=de_DE.UTF-8

already manually modified the file, no need to update again

@lguohan
Copy link
Collaborator

lguohan commented Aug 31, 2018

I understand both way can achieve it, but isn't it better to use the tool to update the file so that you do not need to worry about the exact format of the file?

@richard28530
Copy link
Contributor Author

add update-locale after manually modify it

I tried this below, doesn't work

#Adds a locale to a debian system in non-interactive mode
sudo sed -i '/^#.* en_US.* /s/^#//' $FILESYSTEM_ROOT/etc/locale.gen && \
    sudo LANG=C DEBIAN_FRONTEND=noninteractive chroot $FILESYSTEM_ROOT locale-gen "en_US.UTF-8"
sudo LANG=en_US.UTF-8 DEBIAN_FRONTEND=noninteractive chroot $FILESYSTEM_ROOT update-locale "en_US.UTF-8"
sudo LANG=C chroot $FILESYSTEM_ROOT bash -c "find /usr/share/i18n/locales/ ! -name 'en_US' -type f -exec rm -f {} +"

@lguohan
Copy link
Collaborator

lguohan commented Sep 4, 2018

what is the error message?

Signed-off-by: Guohan Lu <gulv@microsoft.com>
@lguohan
Copy link
Collaborator

lguohan commented Sep 11, 2018

update-locale "LANG=en_US.UTF-8" is the correct command.

@lguohan lguohan merged commit 2d43385 into sonic-net:master Sep 11, 2018
dgsudharsan added a commit to dgsudharsan/sonic-buildimage that referenced this pull request Oct 29, 2021
The submodule update includes the following commits
ac3103a Add missing neighbor resolution for MPLS route programming (sonic-net#1968)
bfba0ad [vlanmgr]Fix for STATE_DB port check logic (sonic-net#1980)
9ef2ba4 Update VLAN removal code to work with 5.10 kernel and new
41fb26c Handle setting unknown mux state (sonic-net#1984)
ac09bde Increase timeout for VS tests (sonic-net#1988)
da8a43e Check if appl DB exists before deleting (sonic-net#1983)
553d75a Change tunnel orch order (sonic-net#1977)
7444e96 Add rekey period in macsec mgr (sonic-net#1958)
d95823d [Buffermgr]Graceful handling of buffer model change (sonic-net#1956)
b0aa6a0 EVPN VxLAN enhancement to support P2MP tunnel based programming for L
85bdf54 Fix the option missing in kernel config issue (sonic-net#1973)
6b15584 Orchagent validates mirror session queue parameter against maximum va
fc9ffb9 [copp] Add ISIS, LDP and micro-BFD trap types to CoPP manager (sonic-net#1890)
452cbc1 [macsecorch]: Add IPG adjusting for MACsec gearbox model (sonic-net#1925)

Signed-off-by: Sudharsan Dhamal Gopalarathnam <sudharsand@nvidia.com>
@dgsudharsan dgsudharsan mentioned this pull request Oct 29, 2021
5 tasks
lguohan pushed a commit that referenced this pull request Nov 1, 2021
The submodule update includes the following commits
ac3103a Add missing neighbor resolution for MPLS route programming (#1968)
bfba0ad [vlanmgr]Fix for STATE_DB port check logic (#1980)
9ef2ba4 Update VLAN removal code to work with 5.10 kernel and new
41fb26c Handle setting unknown mux state (#1984)
ac09bde Increase timeout for VS tests (#1988)
da8a43e Check if appl DB exists before deleting (#1983)
553d75a Change tunnel orch order (#1977)
7444e96 Add rekey period in macsec mgr (#1958)
d95823d [Buffermgr]Graceful handling of buffer model change (#1956)
b0aa6a0 EVPN VxLAN enhancement to support P2MP tunnel based programming for L
85bdf54 Fix the option missing in kernel config issue (#1973)
6b15584 Orchagent validates mirror session queue parameter against maximum va
fc9ffb9 [copp] Add ISIS, LDP and micro-BFD trap types to CoPP manager (#1890)
452cbc1 [macsecorch]: Add IPG adjusting for MACsec gearbox model (#1925)

Signed-off-by: Sudharsan Dhamal Gopalarathnam <sudharsand@nvidia.com>
stepanblyschak added a commit to stepanblyschak/sonic-buildimage that referenced this pull request Nov 11, 2021
```
5f8ebfa (HEAD, origin/master, origin/HEAD, master) [AclOrch] move ACL counters to flex counter infrastructure (sonic-net#1943)
8119ec0 [bfdorch] Orchagent support hardware BFD (sonic-net#1883)
15074ac [sonic-swss]:enable unconfiguring PFC on last TC on a port (sonic-net#1962)
05c7c05 [Mux orch] set default as standby, change mux orch priority (sonic-net#2010)
fe5b2a9 [pytest]: Ignore errors deleting host ifs (sonic-net#2005)
70da9af [ci]: use native arm64 and armhf pool (sonic-net#2013)
e14a071 [qos] Add EXP to TC map support (sonic-net#1954)
c91a7f2 [switchorch] Implement VXLAN src port range feature  (sonic-net#1959)
b20f0f4 Gcov for swss daemon (sonic-net#1737)
01c243a [CRM][MPLS] Fix the mpls nexthop CRM attribute (sonic-net#2008)
8448a60 [vs tests]Migrating sonic-swss tests to use hwsku instead of fakeplatform (sonic-net#1978)
faa26db Fix random failure in PR/CI build. (sonic-net#2006)
e03edb6 Allow interface type value none (sonic-net#1991)
71b9650 [orchagent] Fix group name of port-buffer-drop in flexcounterorch.cpp (sonic-net#1967)
facdef5 [VS test] Skip flaky virtual chassis test (sonic-net#2004)
8261c1f [pytest]: Increase timeout when checking services (sonic-net#2000)
67278be [teammgrd]: Handle LAGs cleanup gracefully on Warm/Fast reboot. (sonic-net#1934)
e92c1df Enable FEC statistics collection for Ethernet ports (sonic-net#1994)
9f30ca1 VxLAN Tunnel Counters and Rates implementation (sonic-net#1859)
ac3103a Add missing neighbor resolution for MPLS route programming (sonic-net#1968)
bfba0ad [vlanmgr]Fix for STATE_DB port check logic (sonic-net#1980)
9ef2ba4 [vlanmgr]: Update VLAN removal code to work with 5.10 kernel and newer iproute2 versions (sonic-net#1970)
41fb26c [Mux orch] Handle setting unknown mux state (sonic-net#1984)
ac09bde [azp]: Increase timeout for VS tests (sonic-net#1988)
da8a43e [pytest]: Check if appl DB exists before deleting (sonic-net#1983)
553d75a [tunnel decap] Change tunnel orch order (sonic-net#1977)
7444e96 [macsecmgr]: Add rekey period in macsec mgr (sonic-net#1958)
d95823d [Buffermgr]Graceful handling of buffer model change (sonic-net#1956)
b0aa6a0 EVPN VxLAN enhancement to support P2MP tunnel based programming for Layer2 extension (sonic-net#1858)
85bdf54 Fix the option missing in kernel config issue (sonic-net#1973)
6b15584 Orchagent validates mirror session queue parameter against maximum value from SAI (sonic-net#1957)
fc9ffb9 [copp] Add ISIS, LDP and micro-BFD trap types to CoPP manager (sonic-net#1890)
452cbc1 [macsecorch]: Add IPG adjusting for MACsec gearbox model (sonic-net#1925)
```

Signed-off-by: Stepan Blyschak <stepanb@nvidia.com>
vivekrnv pushed a commit to vivekrnv/sonic-buildimage that referenced this pull request Jan 6, 2022
Signed-off-by: Vivek Reddy Karri <vkarri@nvidia.com>

30f5dd6 Update the example for pfcwd start command (sonic-net#1984)
9e30871 [Auto Techsupport] Event driven Techsupport Bug Fixes (sonic-net#1986)
fbd565d Fix wrong help message for cable length setting (sonic-net#1978)
b3a5052 [GCU] Using simulated config instead of target config when validating replace operation in NoDependencyMoveValidator (sonic-net#1987)
35cb524 [GCU] Copying config_db before callding sonic_yang.loadData (sonic-net#1983)
a98858d [GCU] Different apply-patch runs should produce same sorted steps (sonic-net#1988)
8c81ae3 [breakout] Fix the check  when port is not present in BREAKOUT_CFG table (sonic-net#1765)
bc8fe7c [doc][DPB] Update DPB related interface breakout command Info (sonic-net#1438)
1a2a9a3 [config] Fix 'config reload -l' command to get filename by default (sonic-net#1611)
ed2fa69 [debug dump util] FDB debug dump util changes (sonic-net#1968)
3b642c9 [GCU] Loading yang-models only once (sonic-net#1981)
bb56fc2 Update swss_ready check to check per namespace swss service (sonic-net#1974)
4f39f9f [GCU] Moving PatchSorter unit-test to json file to make it easier to read/maintain (sonic-net#1977)
1a75870 [CLI][Help string] Changed the show command help text to be more consistent with each other.
818dcbf [GCU] Implementing DryRun by printing patch-sorter steps/imitating config_db (sonic-net#1973)
judyjoseph added a commit that referenced this pull request Jan 9, 2022
4236bc4 [config reload] Fixing config reload when timer based delayed services are disabled (#1967)
d2514e4 [GCU] Different apply-patch runs should produce same sorted steps (#1988)
2878adb [GCU] Using simulated config instead of target config when validating replace operation in NoDependencyMoveValidator (#1987)
fb8ca98 [GCU] Loading yang-models only once (#1981)
f88ee92 [GCU] Copying config_db before callding sonic_yang.loadData (#1983)
9ed0e91 [GCU] Implementing DryRun by printing patch-sorter steps/imitating config_db (#1973)
b36b5e3 [GCU] Moving PatchSorter unit-test to json file to make it easier to read/maintain (#1977)
c0fa28b [generic-config-updater] Improving CreateOnly validator and marking /LOOPBACK_INTERFACE/LOOPBACK#/vrf_name as create-only (#1969)
0559d04 [generic-config-updater] Adding non-strict mode (#1929)
b07f477 [debug dump util] FDB debug dump util changes (#1968)
6d8757a [warm/fast-reboot] Fix kexec portion to support platforms based on Device Tree (#1966)
cc1409e [Auto Techsupport] Event driven Techsupport Bug Fixes (#1986)
6c48bd5 Fix wrong help message for cable length setting (#1978)
c0bbbe3 [breakout] Fix the check  when port is not present in BREAKOUT_CFG table (#1765)
5bb8cad [doc][DPB] Update DPB related interface breakout command Info (#1438)
e6fd990 [config] Fix 'config reload -l' command to get filename by default (#1611)
bd8f7bb Update swss_ready check to check per namespace swss service (#1974)
5439f94 [soft-reboot] Add support for platforms based on Device Tree (#1963)
7c5810a [config] Add portchannel support  for static route  (#1857)
7cb6a1b preserve old order for config reload (#1964)
20bddbd [Auto-Techsupport] Issues related to Multiple Cores crashing handled (#1948)
@xumia xumia mentioned this pull request Jan 17, 2022
5 tasks
theasianpianist pushed a commit to theasianpianist/sonic-buildimage that referenced this pull request Feb 5, 2022
Workaround for some tests timing out at 99% completion
taras-keryk pushed a commit to taras-keryk/sonic-buildimage that referenced this pull request Apr 28, 2022
…nic-net#1988)

#### What I did
Fixes sonic-net#1976

Different runs of `apply-patch` would produce different sorted steps. It is better to be consistent while producing the sorted steps to make it easier to debug issues in the future.

The issue is with using `set(list)` to remove duplicates from the list. Looping over the set elements does not guarantee same order every time we iterate. To reproduce this do the following:
1. Create a file named `test.py` with the content
```python
x = ["XYZ", "ABC", "DEF"]
y = set(x)
print(y)
```
2. Run `python3 test.py`
3. Again `python3 test.py`
4. Again `python3 test.py`
The output of these runs will be different each time.

#### How I did it
Instead of returning a `set`, return a list of `ref_paths` which does not have duplicated elements inserted to begin with.

#### How to verify it
unit-tests

#### Previous command output (if the output of a command-line utility has changed)

#### New command output (if the output of a command-line utility has changed)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants