Publishing release as single file does not include all libraries (libgssapi_krb5)

[kalaskarsanket]

---

Issue moved from microsoft/dotnet#1273

• Please respond to @lastcoolnameleft.

---

From @lastcoolnameleft on Monday, November 23, 2020 4:42:46 PM

I'm trying to create a single-file binary to run on Linux; however, it does not include all libraries. I'm unclear why the Kerberos library is needed.

To reproduce:

➜  dotnet --version
5.0.100
➜  dotnet new console -o single-file
The template "Console Application" was created successfully.

Processing post-creation actions...
Running 'dotnet restore' on single-file/single-file.csproj...
  Determining projects to restore...
  Restored /Users/thfalgou/tmp/dotnet/single-file/single-file.csproj (in 64 ms).
Restore succeeded.
➜  cd single-file 
➜  dotnet publish -r linux-x64 -p:PublishSingleFile=true -c Release
Microsoft (R) Build Engine version 16.8.0+126527ff1 for .NET
Copyright (C) Microsoft Corporation. All rights reserved.

  Determining projects to restore...
  Restored /Users/thfalgou/tmp/dotnet/single-file/single-file.csproj (in 89 ms).
  single-file -> /Users/thfalgou/tmp/dotnet/single-file/bin/Release/net5.0/linux-x64/single-file.dll
  single-file -> /Users/thfalgou/tmp/dotnet/single-file/bin/Release/net5.0/linux-x64/publish/                                                                           
➜  file bin/Release/net5.0/linux-x64/publish/single-file
bin/Release/net5.0/linux-x64/publish/single-file: ELF 64-bit LSB pie executable, x86-64, version 1 (GNU/Linux), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 2.6.32, BuildID[sha1]=6ca69f7bc7061d8eab20013d6820ef5a5c2f7ff4, stripped, too many notes (256)

Start an Ubuntu instance via docker

➜  docker run -it -v $PWD/bin/Release/net5.0/linux-x64/publish:/tmp/single-file ubuntu bash
root@e91793747e67:/# cd /tmp/single-file/
root@e91793747e67:/tmp/single-file# ls
single-file  single-file.pdb
root@e91793747e67:/tmp/single-file# ./single-file
./single-file: error while loading shared libraries: libgssapi_krb5.so.2: cannot open shared object file: No such file or directory
root@e91793747e67:/tmp/single-file# ldd ./single-file
	linux-vdso.so.1 (0x00007ffc371c4000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f1b744cd000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f1b744c7000)
	libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f1b744ab000)
	libgssapi_krb5.so.2 => not found
	librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f1b744a0000)
	libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f1b742bf000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f1b7416e000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f1b74153000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f1b73f61000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f1b75126000)

As you can see, libgssapi_krb5.so.2 was not included.

Due to -p:PublishSingleFile=true, I would expect the application to be a statically linked library instead of a dynamically linked one.

[kalaskarsanket]

---

Issue moved from microsoft/dotnet#1273

• Please respond to @lastcoolnameleft.

---

From @lastcoolnameleft on Monday, November 23, 2020 5:40:27 PM

FYI, adding --self-contained doesn't fix the issue.

[ghost]

Tagging subscribers to this area: @agocke, @vitek-karas
See info in area-owners.md if you want to be subscribed.

Issue Details

---

---

Issue moved from microsoft/dotnet#1273

• Please respond to @lastcoolnameleft.

---

From @lastcoolnameleft on Monday, November 23, 2020 4:42:46 PM

I'm trying to create a single-file binary to run on Linux; however, it does not include all libraries. I'm unclear why the Kerberos library is needed.

To reproduce:

➜  dotnet --version
5.0.100
➜  dotnet new console -o single-file
The template "Console Application" was created successfully.

Processing post-creation actions...
Running 'dotnet restore' on single-file/single-file.csproj...
  Determining projects to restore...
  Restored /Users/thfalgou/tmp/dotnet/single-file/single-file.csproj (in 64 ms).
Restore succeeded.
➜  cd single-file 
➜  dotnet publish -r linux-x64 -p:PublishSingleFile=true -c Release
Microsoft (R) Build Engine version 16.8.0+126527ff1 for .NET
Copyright (C) Microsoft Corporation. All rights reserved.

  Determining projects to restore...
  Restored /Users/thfalgou/tmp/dotnet/single-file/single-file.csproj (in 89 ms).
  single-file -> /Users/thfalgou/tmp/dotnet/single-file/bin/Release/net5.0/linux-x64/single-file.dll
  single-file -> /Users/thfalgou/tmp/dotnet/single-file/bin/Release/net5.0/linux-x64/publish/                                                                           
➜  file bin/Release/net5.0/linux-x64/publish/single-file
bin/Release/net5.0/linux-x64/publish/single-file: ELF 64-bit LSB pie executable, x86-64, version 1 (GNU/Linux), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 2.6.32, BuildID[sha1]=6ca69f7bc7061d8eab20013d6820ef5a5c2f7ff4, stripped, too many notes (256)

Start an Ubuntu instance via docker

➜  docker run -it -v $PWD/bin/Release/net5.0/linux-x64/publish:/tmp/single-file ubuntu bash
root@e91793747e67:/# cd /tmp/single-file/
root@e91793747e67:/tmp/single-file# ls
single-file  single-file.pdb
root@e91793747e67:/tmp/single-file# ./single-file
./single-file: error while loading shared libraries: libgssapi_krb5.so.2: cannot open shared object file: No such file or directory
root@e91793747e67:/tmp/single-file# ldd ./single-file
	linux-vdso.so.1 (0x00007ffc371c4000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f1b744cd000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f1b744c7000)
	libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f1b744ab000)
	libgssapi_krb5.so.2 => not found
	librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f1b744a0000)
	libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f1b742bf000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f1b7416e000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f1b74153000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f1b73f61000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f1b75126000)

As you can see, libgssapi_krb5.so.2 was not included.

Due to -p:PublishSingleFile=true, I would expect the application to be a statically linked library instead of a dynamically linked one.

Author:	kalaskarsanket
Assignees:	-
Labels:	area-Single-File, untriaged
Milestone:	-

[agocke]

Single-file doesn't change the machine dependencies required by dotnet, it only links the dotnet libraries themselves together. You'll still need to install everything in https://docs.microsoft.com/en-us/dotnet/core/install/linux-ubuntu#dependencies

[agocke]

@danmosemsft Do you happen to know if we have a doc on why certain dotnet dependencies are machine dependencies, vs distributed w/ dotnet?

[danmoseley]

I do not know of a document that describes all our dependencies. It may be these simply have to be uncovered by testing. Perhaps @wfurt has context on this one.

[vitek-karas]

The main difference between single-file and non-single-file is that the superhost currently requires all of its dependencies - so if they are not present, that app won't even start. Normal apphost doesn't have this, the dependencies are only loaded when the relevant managed code is executed - which may never happen for some apps. So while it's technically correct that the dependency set hasn't changed, the effective behavior does change for certain apps.

[wfurt]

do we know where this comes from? AFAIK, all the dependencies from NegotiateStream should be hidden behind Native shim we dlopen when needed. SO I would expect that libgssapi_krb5 is NOT needed to run.

[vitek-karas]

In single-file we can't bundle the native shims as .so files, because we don't want to write them to disk on execution and we can't 'dlopen' them from the single-file. So we statically link all the native shims into the executable. But that means they are not delay-loaded anymore... and that's where the problem comes from.

[wfurt]

libSystem.Net.Security.Native.so:
        linux-vdso.so.1 (0x00007ffc7ed01000)
        libgssapi_krb5.so.2 => /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2 (0x00007fa64efee000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fa64ebfd000)
        libkrb5.so.3 => /usr/lib/x86_64-linux-gnu/libkrb5.so.3 (0x00007fa64e927000)
        libk5crypto.so.3 => /usr/lib/x86_64-linux-gnu/libk5crypto.so.3 (0x00007fa64e6f5000)
        libcom_err.so.2 => /lib/x86_64-linux-gnu/libcom_err.so.2 (0x00007fa64e4f1000)
        libkrb5support.so.0 => /usr/lib/x86_64-linux-gnu/libkrb5support.so.0 (0x00007fa64e2e6000)
        /lib64/ld-linux-x86-64.so.2 (0x00007fa64f43d000)
        libkeyutils.so.1 => /lib/x86_64-linux-gnu/libkeyutils.so.1 (0x00007fa64e0e2000)
        libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007fa64dec7000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fa64dca8000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fa64daa4000)

note that if we link libSystem.Net.Security.Native.so directly, we will not only include it's dependencies but recursively also their dependencies.

cc: @janvorli for ideas how to do this programatically.

[vitek-karas]

@VSadov did most of this work, so I'll have to leave it up to him to comment on what exactly happened.

[mthalman]

This is the entire purpose of the runtime-deps Docker images we provide. They install the required package dependencies on top of the base OS. It is recommended for self-contained deployments to be based on a runtime-deps Docker image. An example of this is here.

[gsmachado]

It would be awesome to have this feature, publishing a single file with all runtime .so libs. Just saying. 😄 😅

[michaldobrodenka]

I had the same problem and solved it by abandoning single file idea.

If app is published as self contained but NOT as single file, everything works on the same linux distribution.

[carychai]

Is there any method to solve the problem？I got the same error，and the Linux do not allowed to install other librarys.

[agocke]

@VSadov Was this resolved in #46771 if you're not actually using the API?

[agocke]

Never mind, that's only for OpenSsl. Let's see if there's any way to shim these calls with dlopen.

[agocke]

Created a new issue specifically for lazy loading, since this issue is about including all the possible dependencies, which isn't necessarily something we want to do as it would bloat the size of the file, regardless of whether or not you're using, say, kerberos support.

#54347

[VSadov]

This should be fixed in #55037