Private Crate Repository Issues

[josh-bpl]

Checklist

• I've looked through the issues and pull requests for similar reports

Describe your issue

I am trying to use cross to cross compile my project for a armv7-unknown-linux-gnueabihf (although this issue is also present when using cross to target the host target architecture) target using a Gitlab CI/CD shared runner. I have seen the Gitlab CI sample and the cross remote documentation, both of which I have implemented the best I can tell.

The issue I have is that my project contains dependencies to private Gitlab repositories. I saw the data volumes section of the cross remote documentation and have tried setting CROSS_REMOTE_COPY_REGISTRY and CROSS_REMOTE_COPY_CACHE environment variables, but whatever I do, cross always tries to fetch my private repos at compile time - and of course fails because it cannot authenticate. Before running the cross build command I have tried running cargo update and `` but still the same issue. It appears as though maybe it fails to copy over the cached packages?

I even tried adding this to my Cross.toml to use the job token and https to clone the private crates (with CARGO_NET_GIT_FETCH_WITH_CLI set.) but it still failed - almost like the pre-build command was not run.

[build.armv7-unknown-linux-gnueabihf]
build-std = false
xargo = false
pre-build = [
    "git config --global url.\"https://gitlab-ci-token:$CI_JOB_TOKEN@gitlab.com/\".insteadOf \"ssh://git@gitlab.com/\""
]

I have spent several days experimenting with all the options and am at the point where it looks like it might be an issue with cross - although my user error is always a possibility.

Related issues:
#401

What target(s) are you cross-compiling for?

armv7-unknown-linux-gnueabihf

Which operating system is the host (e.g computer cross is on) running?

• macOS
• Windows
• Linux / BSD
• other OS (specify in description)

What architecture is the host?

• x86_64 / AMD64
• arm32
• arm64 (including Mac M1)

What container engine is cross using?

• docker
• podman
• other container engine (specify in description)

cross version

cross 0.2.5

Example

.gitlab-ci.yml

image: "rust:latest"

stages:
  - build
  - test
  - doc

cargo:build_armv7l:
  stage: build
  services:
    - docker:19.03.12-dind
  before_script:
    - rustup target list --installed
    - export PATH=$PATH:$CARGO_HOME/bin/
    - apt -qq update
    - apt install -yqq git
    - apt-get install -yqq --no-install-recommends build-essential libudev-dev llvm-dev clang libclang-dev libusb-1.0-0-dev libudev-dev
  variables:
    CI_DEBUG_SERVICES: "true"
    GIT_SUBMODULE_STRATEGY: recursive
    # the host where the docker instance is running
    DOCKER_HOST: "tcp://docker:2375"
    DOCKER_DRIVER: "overlay2"
    # ensure cross knows it's running remotely
    CROSS_REMOTE: "1"
    # Copy cargo registry and cache to cross instance
    CROSS_REMOTE_COPY_REGISTRY: "1"
    DOCKER_TLS_CERTDIR: ""

  script:
    - 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client git -y )'
    - eval "$(ssh-agent)"
    - echo "$SSH_PRIVATE_KEY" | tr -d '\r' | ssh-add -
    - mkdir -p ~/.ssh
    - chmod 700 ~/.ssh
    - echo "$SSH_KNOWN_HOSTS" > ~/.ssh/known_hosts
    - chmod 644 ~/.ssh/known_hosts

    - apt -qq update
    - apt install -y apt-transport-https ca-certificates curl gnupg2 software-properties-common > /dev/null
    - curl -fsSL https://download.docker.com/linux/debian/gpg | apt-key add -
    - add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/debian $(lsb_release -cs) stable"
    - apt -qq update
    - apt install -y docker-ce > /dev/null

    - rustc --version && cargo --version # Print version info for debugging
    - cargo install cross
    - cargo clean
    - cargo metadata --format-version=1 --filter-platform armv7-unknown-linux-gnueabihf
    # kill the SSH agent, to ensure it works
    - eval 'ssh-agent -k'
    - cross build --verbose --armv7-unknown-linux-gnueabihf

  artifacts:
    when: always
    paths:
      - $CI_PROJECT_DIR/target/release/my_project

Job logs

job_log.txt

Additional information / notes

No response

[Emilgardis]

Discussion in matrix

2023-02-07
2023-02-08

[Emilgardis]

To confirm, you got the same behaviour when installing cross from current main?

The error for this sample is

    Updating git repository `ssh://git@gitlab.com/[REDACTED]/private_crate_1.git`
error: failed to get `private_crate_1` as a dependency of package `my_project v0.1.0 (/cross/project)`

Caused by:
  failed to load source for dependency `private_crate_1`

Caused by:
  Unable to update ssh://git@gitlab.com/[REDACTED]/private_crate_1.git?rev=355b98b051a7350c277994c8dcb1b9513e031890#355b98b0

Caused by:
  failed to clone into: /cargo/git/db/private_crate_1-74dc0eac1b1023aa

Caused by:
  error: unknown SSH host key
  The SSH host key for `gitlab.com` is not known and cannot be validated.

  To resolve this issue, add the host key to the `net.ssh.known-hosts` array in your Cargo configuration (such as /cargo/config.toml) or in your OpenSSH known_hosts file at /root/.ssh/known_hosts

Is the error the same when using CROSS_REMOTE_COPY_CACHE?

[josh-bpl]

Thanks for the reply, I have rerun the job with the only change being to add CROSS_REMOTE_COPY_CACHE: "1" to the job's variables - appears to be an identical error:

    Updating git repository `ssh://git@gitlab.com/[REDACTED]/private_crate_1.git`
error: failed to get `private_crate_1` as a dependency of package `my_project v0.1.0 (/cross/project)`

Caused by:
  failed to load source for dependency `private_crate_1`

Caused by:
  Unable to update ssh://git@gitlab.com/[REDACTED]/private_crate_1.git?rev=355b98b051a7350c277994c8dcb1b9513e031890#355b98b0

Caused by:
  failed to clone into: /cargo/git/db/private_crate_1-74dc0eac1b1023aa

Caused by:
  error: unknown SSH host key
  The SSH host key for `gitlab.com` is not known and cannot be validated.

  To resolve this issue, add the host key to the `net.ssh.known-hosts` array in your Cargo configuration (such as /cargo/config.toml) or in your OpenSSH known_hosts file at /root/.ssh/known_hosts

[Emilgardis]

The issue boiled down to two things

1. the copy copied some folders to the same path in the volume, ending up in cp -a /path/to/folder volume:/cross/path/to/folder, which meant that the path became volume:/cross/path/to/folder/folder
   Fixed by doing docker cp -a /path/to/folder/. ...
2. When copying $CARGO_HOME, we didn't make sure that the path existed, solved by mkdir -p /cross/path/to/cargo

[josh-bpl]

I really appreciate your help on this one @Emilgardis, thanks for looking into it and finding a fix so quickly! 🦾😎