[CT-2412] [Bug] The current Dockerfile.test is not built

[eshkinkot]

Is this a new bug in dbt-core?

• I believe this is a new bug in dbt-core
• I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

This change #6865 does not work and the modified Dockerfile.test is no longer built.

Expected Behavior

docker-compose run --rm test bash -l build and run successfully

Steps To Reproduce

1. run docker-compose run --rm test bash -l
2. docker build exit with error

Relevant log output

add-apt-repository (realy it is gpg) does not work without gpg-agent in first run:

gpg: error running '/usr/bin/gpg-agent': probably not installed
gpg: failed to start agent '/usr/bin/gpg-agent': Configuration error
gpg: can't connect to the agent: Configuration error
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/softwareproperties/shortcuthandler.py", line 423, in add_key

Adding key to /etc/apt/trusted.gpg.d/git-core-ubuntu-ppa.gpg with fingerprint E1DD270288B4E6030699E45FA1715D88E1DF1F24
    subprocess.run(cmd.split(), check=True, input=keys)
  File "/usr/lib/python3.11/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['gpg', '-q', '--no-options', '--no-default-keyring', '--batch', '--keyring', '/etc/apt/trusted.gpg.d/git-core-ubuntu-ppa.gpg', '--homedir', '/tmp/tmpip_5ksi6', '--import']' returned non-zero exit status 2.

apt install netcat does not install netcat because Ubuntu 23.04 changes defaults:

Package netcat is a virtual package provided by:
  netcat-traditional 1.10-47
  netcat-openbsd 1.219-1ubuntu1
You should explicitly select one to install.

E: Package 'netcat' has no installation candidate

Installing python from ppa:deadsnakes/ppa does not work because deadsnakes does not support Ubuntu 23.04:

+ add-apt-repository ppa:deadsnakes/ppa
Err:7 https://ppa.launchpadcontent.net/deadsnakes/ppa/ubuntu lunar Release
  404  Not Found [IP: 185.125.190.52 443]
Reading package lists...
E: The repository 'https://ppa.launchpadcontent.net/deadsnakes/ppa/ubuntu lunar Release' does not have a Release file.

From https://launchpad.net/~deadsnakes/+archive/ubuntu/ppa:

Supported Ubuntu and Python Versions
====================================

- Ubuntu 18.04 (bionic) Python2.3 - Python 2.6, Python 3.1 - Python 3.5, Python3.7 - Python3.11
- Ubuntu 20.04 (focal) Python3.5 - Python3.7, Python3.9 - Python3.11
- Ubuntu 22.04 (jammy) Python3.7 - Python3.9, Python3.11
- Note: Python2.7 (all), Python 3.6 (bionic), Python 3.8 (focal), Python 3.10 (jammy) are not provided by deadsnakes as upstream ubuntu provides those packages.

I can fix gpg and netcat install, but I dont know what I must do with deadsnakes.

Environment

- OS: Slackware 14.2
- Python: 3.7
- dbt: main 2ddf296a8e010ddbfc23339e554b66f141a7f304

Which database adapter are you using with dbt?

No response

Additional Context

No response

[dbeatty10]

Thanks for reporting this @eshkinkot !

I can confirm that neither of these worked for me either with ubuntu:23.04 in Dockerfile.test:

docker-compose run --rm test tox -e py

# this is equivalent to above
make unit USE_DOCKER=true

Since Ubuntu 23.04 (Lunar Lobster) is still in beta for another week and Ubuntu 22.04.2 LTS (Jammy Jellyfish) is a long-term supported release, I wonder if it makes most sense to solve this by reverting #6865 and keeping 22.04?

I didn't try out 22.04 to see if that works. Interested to hear if using ubuntu:22.04 works for you or not.

[eshkinkot]

Interested to hear if using ubuntu:22.04 works for you or not.

No :-) But it can be fixed:

diff --git a/Dockerfile.test b/Dockerfile.test
index 46effbf3c..fde36d9f4 100644
--- a/Dockerfile.test
+++ b/Dockerfile.test
@@ -3,13 +3,13 @@
 #  See `/docker` for a generic and production-ready docker file
 ##
 
-FROM ubuntu:23.04
+FROM ubuntu:22.04
 
 ENV DEBIAN_FRONTEND noninteractive
 
 RUN apt-get update \
     && apt-get install -y --no-install-recommends \
-    software-properties-common \
+    software-properties-common gpg-agent \
     && add-apt-repository ppa:git-core/ppa -y \
     && apt-get dist-upgrade -y \
     && apt-get install -y --no-install-recommends \
@@ -30,13 +30,9 @@ RUN apt-get update \
     unixodbc-dev \
     && add-apt-repository ppa:deadsnakes/ppa \
     && apt-get install -y \
-    python \
-    python-dev \
-    python3-pip \
-    python3.6 \
-    python3.6-dev \
+    python-is-python3 \
+    python-dev-is-python3 \
     python3-pip \
-    python3.6-venv \
     python3.7 \
     python3.7-dev \
     python3.7-venv \

with this patch it builds correctly and

dbt-core$ docker-compose up -d database
dbt-core$ docker-compose run test bash -l
/usr/app$ tox -e py

works as expected.

[dbeatty10]

Excellent @eshkinkot ! Thanks for opening up that PR 🏅