podman manifest add disregarding architecture

[p5]

Issue Description

When trying to build a multi-arch OCI manifest with Podman, I am receiving a strange error where it appears Podman is looking for the incorrect image architecture.

$ podman manifest add test ghcr.io/rsturla/akmods/multiarch-nvidia:40-arm64 --arch=linux/arm64
Error: selecting image from manifest list in "docker://ghcr.io/rsturla/akmods/multiarch-nvidia:40-arm64": no image found in image index for architecture "amd64", variant "", OS "linux"

As you can see, I am explicitly telling Podman I wish to use an arm64 image in this build, however it still incorrectly looks to the registry to add an amd64 image.

Using a digest rather than the image tag, the command exits successfully and the image is added to the manifest.

For detailed reproduction steps, please see below.

Steps to reproduce the issue

Steps to reproduce the issue

1. podman manifest create test # successful
2. podman manifest add test ghcr.io/rsturla/akmods/multiarch-nvidia:40-x86_64 # successful
3. podman manifest add test ghcr.io/rsturla/akmods/multiarch-nvidia:40-arm64 # error
4. DIGEST=$(skopeo inspect --raw docker://ghcr.io/rsturla/akmods/multiarch-nvidia:40-arm64 | jq -r '.manifests[] | select(.platform.architecture == "arm64") | .digest') # successful
5. podman manifest add test ghcr.io/rsturla/akmods/multiarch-nvidia@$DIGEST # successful

➜  podman manifest create test
097522fc74713eb6d9f1533d7e811fb409b57a1192ac778d0ce60830b3cf9dd6
➜  podman manifest add test ghcr.io/rsturla/akmods/multiarch-nvidia:40-x86_64
097522fc74713eb6d9f1533d7e811fb409b57a1192ac778d0ce60830b3cf9dd6
➜  podman manifest add test ghcr.io/rsturla/akmods/multiarch-nvidia:40-arm64
Error: selecting image from manifest list in "docker://ghcr.io/rsturla/akmods/multiarch-nvidia:40-arm64": no image found in image index for architecture "amd64", variant "", OS "linux"
➜  DIGEST=$(skopeo inspect --raw docker://ghcr.io/rsturla/akmods/multiarch-nvidia:40-arm64 | jq -r '.manifests[] | select(.platform.architecture == "arm64") | .digest')
➜  podman manifest add test ghcr.io/rsturla/akmods/multiarch-nvidia@$DIGEST
097522fc74713eb6d9f1533d7e811fb409b57a1192ac778d0ce60830b3cf9dd6

Describe the results you received

Rather than Podman adding the correct ARM64 image, it looks to the registry for an AMD64 image, and fails when it cannot be found. Even after explicitly defining the architecture.

Describe the results you expected

I expect to be able to add both AMD64 and ARM64 images to the manifest using their tags.

podman info output

host:
  arch: amd64
  buildahVersion: 1.37.2
  cgroupControllers:
  - cpu
  - io
  - memory
  - pids
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-2.1.12-2.fc40.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.1.12, commit: '
  cpuUtilization:
    idlePercent: 95.26
    systemPercent: 1.16
    userPercent: 3.58
  cpus: 24
  databaseBackend: sqlite
  distribution:
    distribution: fedora
    variant: silverblue
    version: "40"
  eventLogger: journald
  freeLocks: 2028
  hostname: fedora
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 524288
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 524288
      size: 65536
  kernel: 6.10.6-200.fc40.x86_64
  linkmode: dynamic
  logDriver: journald
  memFree: 5875630080
  memTotal: 33363582976
  networkBackend: netavark
  networkBackendInfo:
    backend: netavark
    dns:
      package: aardvark-dns-1.12.2-2.fc40.x86_64
      path: /usr/libexec/podman/aardvark-dns
      version: aardvark-dns 1.12.2
    package: netavark-1.12.2-1.fc40.x86_64
    path: /usr/libexec/podman/netavark
    version: netavark 1.12.2
  ociRuntime:
    name: crun
    package: crun-1.15-1.fc40.x86_64
    path: /usr/bin/crun
    version: |-
      crun version 1.15
      commit: e6eacaf4034e84185fd8780ac9262bbf57082278
      rundir: /run/user/1000/crun
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +LIBKRUN +WASM:wasmedge +YAJL
  os: linux
  pasta:
    executable: /usr/bin/pasta
    package: passt-0^20240906.g6b38f07-1.fc40.x86_64
    version: |
      pasta 0^20240906.g6b38f07-1.fc40.x86_64
      Copyright Red Hat
      GNU General Public License, version 2 or later
        <https://www.gnu.org/licenses/old-licenses/gpl-2.0.html>
      This is free software: you are free to change and redistribute it.
      There is NO WARRANTY, to the extent permitted by law.
  remoteSocket:
    exists: false
    path: /run/user/1000/podman/podman.sock
  rootlessNetworkCmd: pasta
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: true
  serviceIsRemote: false
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.2.2-2.fc40.x86_64
    version: |-
      slirp4netns version 1.2.2
      commit: 0ee2d87523e906518d34a6b423271e4826f71faf
      libslirp: 4.7.0
      SLIRP_CONFIG_VERSION_MAX: 4
      libseccomp: 2.5.5
  swapFree: 8588050432
  swapTotal: 8589930496
  uptime: 24h 40m 30.00s (Approximately 1.00 days)
  variant: ""
plugins:
  authorization: null
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  - ipvlan
  volume:
  - local
registries:
  ghcr.io/rsturla:
    Blocked: false
    Insecure: false
    Location: ghcr.io/rsturla
    MirrorByDigestOnly: false
    Mirrors:
    - Insecure: true
      Location: localhost:5000/rsturla
      PullFromMirror: ""
    Prefix: ghcr.io/rsturla
    PullFromMirror: ""
  localhost:5000:
    Blocked: false
    Insecure: true
    Location: localhost:5000
    MirrorByDigestOnly: false
    Mirrors: null
    Prefix: localhost:5000
    PullFromMirror: ""
  search:
  - registry.fedoraproject.org
  - registry.access.redhat.com
  - docker.io
store:
  configFile: /var/home/admin/.config/containers/storage.conf
  containerStore:
    number: 4
    paused: 0
    running: 1
    stopped: 3
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /var/home/admin/.local/share/containers/storage
  graphRootAllocated: 1998678130688
  graphRootUsed: 731423617024
  graphStatus:
    Backing Filesystem: btrfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Supports shifting: "false"
    Supports volatile: "true"
    Using metacopy: "false"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 206
  runRoot: /run/user/1000/containers
  transientStore: false
  volumePath: /var/home/admin/.local/share/containers/storage/volumes
version:
  APIVersion: 5.2.2
  Built: 1724198400
  BuiltTime: Wed Aug 21 01:00:00 2024
  GitCommit: ""
  GoVersion: go1.22.6
  Os: linux
  OsArch: linux/amd64
  Version: 5.2.2

Podman in a container

No

Privileged Or Rootless

Rootless

Upstream Latest Release

Yes

Additional environment details

Additional environment details

Additional information

The issue appears similar to this - both are using scratch images and displaying the same error, however the issue linked was resolved through a patch.