Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

steampipe in Docker/DockerFile --exit status 255 #209

Closed
astro44 opened this issue Feb 21, 2021 · 7 comments
Closed

steampipe in Docker/DockerFile --exit status 255 #209

astro44 opened this issue Feb 21, 2021 · 7 comments

Comments

@astro44
Copy link

astro44 commented Feb 21, 2021

after attempted install getting this message:
bash-4.2# steampipe plugin install aws
Installed plugin: aws v0.5.0
Documentation: https://hub.steampipe.io/plugins/turbot/aws
panic: x Initializing database... FAILED! exit status 255

goroutine 1 [running]:
github.com/turbot/steampipe/utils.FailOnErrorWithMessage(0x1047f60, 0xc0000e0540, 0xeba31e, 0x22)
/home/runner/work/steampipe/steampipe/utils/errors.go:28 +0x137
github.com/turbot/steampipe/db.EnsureDBInstalled()
/home/runner/work/steampipe/steampipe/db/install.go:91 +0xdba
github.com/turbot/steampipe/cmd.refreshConnections(0x0, 0x0)
/home/runner/work/steampipe/steampipe/cmd/plugin.go:364 +0x40
github.com/turbot/steampipe/cmd.runPluginInstallCmd(0xc00016ab00, 0xc0001136e0, 0x1, 0x1)
/home/runner/work/steampipe/steampipe/cmd/plugin.go:263 +0x62a
github.com/turbot/steampipe/cmdconfig.OnCmd.func1(0xc00016ab00, 0xc0001136e0, 0x1, 0x1)
/home/runner/work/steampipe/steampipe/cmdconfig/builder.go:30 +0x65
github.com/spf13/cobra.(*Command).execute(0xc00016ab00, 0xc0001136b0, 0x1, 0x1, 0xc00016ab00, 0xc0001136b0)
/home/runner/go/pkg/mod/github.com/spf13/cobra@v1.0.0/command.go:846 +0x29d
github.com/spf13/cobra.(*Command).ExecuteC(0x16e0980, 0x1, 0x1, 0x0)
/home/runner/go/pkg/mod/github.com/spf13/cobra@v1.0.0/command.go:950 +0x349
github.com/spf13/cobra.(*Command).Execute(...)
/home/runner/go/pkg/mod/github.com/spf13/cobra@v1.0.0/command.go:887
github.com/turbot/steampipe/cmd.Execute(0xc000488000, 0x1)
/home/runner/work/steampipe/steampipe/cmd/root.go:54 +0x6d
main.main()
/home/runner/work/steampipe/steampipe/main.go:28 +0xa0
@e-gineer
Copy link
Contributor

Hey @astro44 ... Thanks for reporting this, a couple of questions:

Are you trying to use / install steampipe as root in your container? Postgres refuses to run as root, and we don't catch / handle this case well yet (#167). If so, perhaps you can try setting your container as a non-root user?

We'd love to understand more about your use case for Steampipe? How are you trying to use it inside a container? (Helps us prioritize our features and approach.) Thanks!

@astro44
Copy link
Author

astro44 commented Feb 21, 2021 •
edited
Loading

@e-gineer Thanks for the fast response... it really doesn't matter what container. The objective is to get it to run in docker with any of these flavours, right now attempting Amazon Linux:
Amazon Linux 2
Alpine
CentOS
Debian
Ubuntu
if your team currently has a working dockerfile, example, with steampipe running that would be super helpfull.

Current attempt:

FROM amazonlinux

# Install apt dependencies
RUN yum install -y \
  gcc gcc-c++ freetype-devel yum-utils findutils openssl-devel
    
RUN yum -y groupinstall development

RUN curl https://www.python.org/ftp/python/3.7.10/Python-3.7.10.tar.xz | tar -xJ \
    && cd Python-3.7.10 \
    && ./configure --prefix=/usr/local --enable-shared \
    && make \
    && make install \
    && cd .. \
    && rm -rf Python-3.7.10

ENV LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH

RUN yum update

RUN /bin/sh -c "$(curl -fsSL https://github.com/turbot/steampipe/main/install.sh)"
RUN steampipe plugin install steampipe
RUN steampipe plugin install aws
RUN steampipe plugin install gcp
RUN steampipe plugin install azure
RUN steampipe plugin install git

@e-gineer
Copy link
Contributor

The key thing is to run all steampipe commands as a non-root user (see #167). For example, if you inject these extra lines into your Dockerfile it seemed to work for me:

RUN /bin/sh -c "$(curl -fsSL https://github.com/turbot/steampipe/main/install.sh)"

# Steampipe needs to run as a non-root user
RUN useradd -ms /bin/bash steampipe
USER steampipe
WORKDIR /home/steampipe

RUN steampipe plugin install steampipe
RUN steampipe plugin install aws
RUN steampipe plugin install gcp
RUN steampipe plugin install azure
RUN steampipe plugin install github

I've just built a quick working example for Ubuntu that may also be helpful:

FROM ubuntu:20.04

RUN apt-get update && \
    apt-get install -y curl && \
    useradd -ms /bin/bash steampipe

RUN /bin/sh -c "$(curl -fsSL https://github.com/turbot/steampipe/main/install.sh)"

USER steampipe
WORKDIR /home/steampipe

RUN steampipe plugin install steampipe aws gcp azure github

ENTRYPOINT [ "/usr/local/bin/steampipe" ]

CMD [ "query" ]

I was able to use it successfully as follows:

~/src/steampipe-docker $ docker build . --tag steampipe:latest
Sending build context to Docker daemon  2.048kB
Step 1/9 : FROM ubuntu:20.04
 ---> f63181f19b2f
Step 2/9 : RUN apt-get update &&     apt-get install -y curl &&     useradd -ms /bin/bash steampipe
 ---> Using cache
 ---> 0317ac1d3b9e
Step 3/9 : RUN /bin/sh -c "$(curl -fsSL https://github.com/turbot/steampipe/main/install.sh)"
 ---> Using cache
 ---> 8fbc5669907a
Step 4/9 : USER steampipe
 ---> Using cache
 ---> a5258d2049ca
Step 5/9 : WORKDIR /home/steampipe
 ---> Using cache
 ---> 55ce429d09e5
Step 6/9 : RUN steampipe plugin install steampipe aws gcp azure github
 ---> Using cache
 ---> f678dd34fd97
Step 7/9 : EXPOSE 9193
 ---> Using cache
 ---> c082d90ceb83
Step 8/9 : ENTRYPOINT [ "/usr/local/bin/steampipe" ]
 ---> Using cache
 ---> b70baa76f48c
Step 9/9 : CMD [ "query" ]
 ---> Using cache
 ---> a5705d7d3e02
Successfully built a5705d7d3e02
Successfully tagged steampipe:latest
~/src/steampipe-docker $ docker run -it steampipe:latest
Welcome to Steampipe v0.2.1
For more information, type .help
> select * from steampipe_registry_plugin
+---------------------+---------------------+---------------------+
| name                | create_time         | update_time         |
+---------------------+---------------------+---------------------+
| turbot/digitalocean | 2021-01-21 14:10:54 | 2021-02-18 19:34:13 |
| turbot/azure        | 2021-01-11 13:20:34 | 2021-02-18 19:01:02 |
| turbot/gcp          | 2021-01-21 13:51:19 | 2021-02-18 18:44:35 |
| turbot/aws          | 2020-12-03 17:52:34 | 2021-02-18 18:58:49 |
| turbot/whois        | 2021-01-21 14:26:20 | 2021-01-21 18:28:26 |
| turbot/chaos        | 2020-12-17 11:53:59 | 2021-01-22 10:49:38 |
| turbot/slack        | 2021-01-21 14:15:26 | 2021-02-18 19:26:08 |
| turbot/zendesk      | 2021-01-21 14:19:29 | 2021-02-18 18:59:55 |
| turbot/github       | 2021-01-21 14:40:16 | 2021-02-18 19:16:30 |
| turbot/ipstack      | 2021-01-21 14:21:12 | 2021-02-18 18:55:55 |
| turbot/steampipe    | 2021-01-06 22:12:14 | 2021-01-21 18:29:58 |
+---------------------+---------------------+---------------------+
> .exit
⠧ Shutting down... 
~/src/steampipe-docker $

@astro44
Copy link
Author

astro44 commented Feb 21, 2021 •
edited
Loading

@e-gineer Thank you... I still get this error (*same error w/ Ubuntu):

#=> ERROR [ 9/13] RUN steampipe plugin install steampipe 14.8s
#------
#> [ 9/13] RUN steampipe plugin install steampipe:
#Installed plugin: steampipe
#12 5.515 Documentation: https://hub.steampipe.io/plugins/turbot/steampipe
#panic: x Initializing database... FAILED! exit status 255
#12 14.04
#12 14.04 goroutine 1 [running]:
#12 14.04 github.com/turbot/steampipe/utils.FailOnErrorWithMessage(0x1047f60, 0xc000206060, 0xeba31e, 0x22)
#12 14.04 /home/runner/work/steampipe/steampipe/utils/errors.go:28 +0x137
#12 14.04 github.com/turbot/steampipe/db.EnsureDBInstalled()
#12 14.04 /home/runner/work/steampipe/steampipe/db/install.go:91 +0xdba
#12 14.04 github.com/turbot/steampipe/cmd.refreshConnections(0x0, 0x0)
#12 14.04 /home/runner/work/steampipe/steampipe/cmd/plugin.go:364 +0x40
#12 14.04 github.com/turbot/steampipe/cmd.runPluginInstallCmd(0xc00035eb00, 0xc000252230, 0x1, 0x1)
#12 14.04 /home/runner/work/steampipe/steampipe/cmd/plugin.go:263 +0x62a
#12 14.04 github.com/turbot/steampipe/cmdconfig.OnCmd.func1(0xc00035eb00, 0xc000252230, 0x1, 0x1)
#12 14.04 /home/runner/work/steampipe/steampipe/cmdconfig/builder.go:30 +0x65
#12 14.04 github.com/spf13/cobra.(*Command).execute(0xc00035eb00, 0xc000252200, 0x1, 0x1, 0xc00035eb00, 0xc000252200)
#12 14.04 /home/runner/go/pkg/mod/github.com/spf13/cobra@v1.0.0/command.go:846 +0x29d
#12 14.04 github.com/spf13/cobra.(*Command).ExecuteC(0x16e0980, 0x1, 0x1, 0x0)
#12 14.04 /home/runner/go/pkg/mod/github.com/spf13/cobra@v1.0.0/command.go:950 +0x349
#12 14.04 github.com/spf13/cobra.(*Command).Execute(...)
#12 14.04 /home/runner/go/pkg/mod/github.com/spf13/cobra@v1.0.0/command.go:887
#12 14.04 github.com/turbot/steampipe/cmd.Execute(0xc000408060, 0x1)
#12 14.04 /home/runner/work/steampipe/steampipe/cmd/root.go:54 +0x6d
#12 14.04 main.main()
#12 14.04 /home/runner/work/steampipe/steampipe/main.go:28 +0xa0
#------
#executor failed running [/bin/sh -c steampipe plugin install steampipe]: exit code: 2

using this dockerfile, based on your suggestions:

FROM amazonlinux

# Install apt dependencies
RUN yum install -y \
  gcc gcc-c++ freetype-devel yum-utils findutils openssl-devel


RUN yum -y groupinstall development

RUN curl https://www.python.org/ftp/python/3.7.10/Python-3.7.10.tar.xz | tar -xJ \
    && cd Python-3.7.10 \
    && ./configure --prefix=/usr/local --enable-shared \
    && make \
    && make install \
    && cd .. \
    && rm -rf Python-3.7.10

ENV LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH

RUN yum update -y

RUN /bin/sh -c "$(curl -fsSL https://github.com/turbot/steampipe/main/install.sh)"

# Steampipe needs to run as a non-root user
RUN useradd -ms /bin/bash steampipe
USER steampipe
WORKDIR /home/steampipe

RUN steampipe plugin install steampipe
RUN steampipe plugin install aws
RUN steampipe plugin install gcp
RUN steampipe plugin install azure
RUN steampipe plugin install github

@e-gineer
Copy link
Contributor

I think you may need to check your Dockerfile again? That one works for me. Here is my output, which includes 15 steps. Your error above seems to indicate 13 steps:

~/src/steampipe-docker $ cat Dockerfile
FROM amazonlinux

# Install apt dependencies
RUN yum install -y \
  gcc gcc-c++ freetype-devel yum-utils findutils openssl-devel

RUN yum -y groupinstall development

RUN curl https://www.python.org/ftp/python/3.7.10/Python-3.7.10.tar.xz | tar -xJ \
    && cd Python-3.7.10 \
    && ./configure --prefix=/usr/local --enable-shared \
    && make \
    && make install \
    && cd .. \
    && rm -rf Python-3.7.10

ENV LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH

RUN yum update -y

RUN /bin/sh -c "$(curl -fsSL https://github.com/turbot/steampipe/main/install.sh)"

# Steampipe needs to run as a non-root user
RUN useradd -ms /bin/bash steampipe
USER steampipe
WORKDIR /home/steampipe

RUN steampipe plugin install steampipe
RUN steampipe plugin install aws
RUN steampipe plugin install gcp
RUN steampipe plugin install azure
RUN steampipe plugin install github
~/src/steampipe-docker $ 
~/src/steampipe-docker $ docker build . --tag steampipe:issue-209
Sending build context to Docker daemon  4.608kB
Step 1/15 : FROM amazonlinux
 ---> d79e41188c15
Step 2/15 : RUN yum install -y   gcc gcc-c++ freetype-devel yum-utils findutils openssl-devel
 ---> Using cache
 ---> 315805f42fcf
Step 3/15 : RUN yum -y groupinstall development
 ---> Using cache
 ---> b20083bb3d09
Step 4/15 : RUN curl https://www.python.org/ftp/python/3.7.10/Python-3.7.10.tar.xz | tar -xJ     && cd Python-3.7.10     && ./configure --prefix=/usr/local --enable-shared     && make     && make install     && cd ..     && rm -rf Python-3.7.10
 ---> Using cache
 ---> deaf3969a0c4
Step 5/15 : ENV LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH
 ---> Using cache
 ---> 34bc91bf9165
Step 6/15 : RUN yum update -y
 ---> Using cache
 ---> 9f8c7a2fd628
Step 7/15 : RUN /bin/sh -c "$(curl -fsSL https://github.com/turbot/steampipe/main/install.sh)"
 ---> Using cache
 ---> 3827875817ba
Step 8/15 : RUN useradd -ms /bin/bash steampipe
 ---> Using cache
 ---> cd39dc213c93
Step 9/15 : USER steampipe
 ---> Using cache
 ---> 9059779aeef6
Step 10/15 : WORKDIR /home/steampipe
 ---> Using cache
 ---> 62cee3175cd1
Step 11/15 : RUN steampipe plugin install steampipe
 ---> Using cache
 ---> a2742fe4a4eb
Step 12/15 : RUN steampipe plugin install aws
 ---> Using cache
 ---> c2d8fe4bee63
Step 13/15 : RUN steampipe plugin install gcp
 ---> Using cache
 ---> 3b614cee4fb6
Step 14/15 : RUN steampipe plugin install azure
 ---> Using cache
 ---> 41d93d4cdea9
Step 15/15 : RUN steampipe plugin install github
 ---> Using cache
 ---> 78ae4460c767
Successfully built 78ae4460c767
Successfully tagged steampipe:issue-209
~/src/steampipe-docker $ 
~/src/steampipe-docker $ docker run -it steampipe:issue-209 steampipe query
Welcome to Steampipe v0.2.1
For more information, type .help
> select * from steampipe_registry_plugin
+---------------------+---------------------+---------------------+
| name                | create_time         | update_time         |
+---------------------+---------------------+---------------------+
| turbot/zendesk      | 2021-01-21 14:19:29 | 2021-02-18 18:59:55 |
| turbot/slack        | 2021-01-21 14:15:26 | 2021-02-18 19:26:08 |
| turbot/github       | 2021-01-21 14:40:16 | 2021-02-18 19:16:30 |
| turbot/chaos        | 2020-12-17 11:53:59 | 2021-01-22 10:49:38 |
| turbot/steampipe    | 2021-01-06 22:12:14 | 2021-01-21 18:29:58 |
| turbot/azure        | 2021-01-11 13:20:34 | 2021-02-18 19:01:02 |
| turbot/whois        | 2021-01-21 14:26:20 | 2021-01-21 18:28:26 |
| turbot/ipstack      | 2021-01-21 14:21:12 | 2021-02-18 18:55:55 |
| turbot/digitalocean | 2021-01-21 14:10:54 | 2021-02-18 19:34:13 |
| turbot/gcp          | 2021-01-21 13:51:19 | 2021-02-18 18:44:35 |
| turbot/aws          | 2020-12-03 17:52:34 | 2021-02-18 18:58:49 |
+---------------------+---------------------+---------------------+
> .exit
⠧ Shutting down... 
~/src/steampipe-docker $ 
~/src/steampipe-docker $

@astro44
Copy link
Author

astro44 commented Feb 22, 2021

@e-gineer got it working.... had to switch to intel based machine!! Thanks for all your help
Please close ticket.

@e-gineer
Copy link
Contributor

Glad you got it going!

ahirreddy pushed a commit to ahirreddy/steampipe that referenced this issue Jul 8, 2024
@bigdatasourav
Add table azure_data_lake_store. Closes turbot#209 (turbot#217)
f4e5356
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@astro44 @e-gineer