syft is unable to locate image locally in k8s with containerd runtime

[ibreakthecloud]

What happened:
Running syft inside pod running in k8s with underlying containerd runtime always falls back to cloud registry.
I have a k8s cluster(containerd), with an application pod(image is on quay private repo) and a syft pod, syft pod has two sock path mounted(docker and containerd).
Running syft inside syft pod with quay private image fails, because it looks up image in quay registry although it is present locally.

$ syft quay.io/deepfenceio/deepfence_agent:3.6.0
1 error occurred:
        * failed to construct source from user input "quay.io/deepfenceio/deepfence_agent:3.6.0": could not fetch image "quay.io/deepfenceio/deepfence_agent:3.6.0": unable to use OciRegistry source: failed to get image descriptor from registry: GET https://quay.io/v2/deepfenceio/deepfence_agent/manifests/3.6.0: UNAUTHORIZED: access to the requested resource is not authorized; map[]

application image is present locally

$ ctr -n k8s.io i ls | grep quay.io/deepfenceio/deepfence_agent:3.6.0
quay.io/deepfenceio/deepfence_agent:3.6.0      application/vnd.docker.distribution.manifest.v2+json     sha256:b9c7d9ea537419177223f3b8e8d63ea1939d53e34ffef02e5dd0c9b80b9f9334 389.8 MiB linux/amd64        io.cri-containerd.image=managed 

What you expected to happen:
Look for image locally before falling back to registry in case of containerd

How to reproduce it (as minimally and precisely as possible):
Try scanning any image from pricate repo present in k8s cluster with containerd runtime

Anything else we need to know?:
Also, running syft inside pod in containerd k8s fails for few image but passes if ran on same image on ubuntu machine or mac

example:

failure inside pod

$ syft k8s.gcr.io/redis:e2e
1 error occurred:
        * failed to construct source from user input "k8s.gcr.io/redis:e2e": could not fetch image "k8s.gcr.io/redis:e2e": unable to use OciRegistry source: failed to get image from registry: unsupported MediaType: "application/vnd.docker.distribution.manifest.v1+prettyjws", see https://github.com/google/go-containerregistry/issues/377
        

success on vm(ubuntu):

$ syft k8s.gcr.io/redis:e2e
 ✔ Loaded image
 ✔ Parsed image
 ✔ Cataloged packages      [310 packages]
NAME                         VERSION                           TYPE
PyGObject                    3.12.0                            python
adduser                      3.113+nmu3ubuntu3                 deb
apt                          1.0.1ubuntu2.6                    deb
apt-utils                    1.0.1ubuntu2.6                    deb
...

Environment:

• Output of syft version:

Application:        syft
Version:            [not provided]
JsonSchemaVersion:  3.2.2
BuildDate:          [not provided]
GitCommit:          [not provided]
GitDescription:     [not provided]
Platform:           linux/amd64
GoVersion:          go1.18.1
Compiler:           gc

• OS (e.g: cat /etc/os-release or similar):

cat /etc/os-release
NAME="Container-Optimized OS"
ID=cos
PRETTY_NAME="Container-Optimized OS from Google"
HOME_URL="https://cloud.google.com/container-optimized-os/docs"
BUG_REPORT_URL="https://cloud.google.com/container-optimized-os/docs/resources/support-policy#contact_us"
KERNEL_COMMIT_ID=ccbab0481cec29d7f07947bcb6255f325b88513f
GOOGLE_CRASH_ID=Lakitu
GOOGLE_METRICS_PRODUCT_ID=26
VERSION=93
VERSION_ID=93
BUILD_ID=16623.102.23

• Kubectl get nodes

k get nodes -o wide
NAME                                             STATUS   ROLES    AGE    VERSION           INTERNAL-IP   EXTERNAL-IP      OS-IMAGE                             KERNEL-VERSION   CONTAINER-RUNTIME
gke-harsh-cve-agent-default-pool-f7a03977-bthn   Ready    <none>   7d2h   v1.22.8-gke.201   10.128.0.21   35.224.254.64    Container-Optimized OS from Google   5.10.90+         containerd://1.5.4
gke-harsh-cve-agent-default-pool-f7a03977-csq8   Ready    <none>   7d2h   v1.22.8-gke.201   10.128.0.24   34.122.100.211   Container-Optimized OS from Google   5.10.90+         containerd://1.5.4
gke-harsh-cve-agent-default-pool-f7a03977-htrc   Ready    <none>   7d2h   v1.22.8-gke.201   10.128.0.20   35.232.51.238    Container-Optimized OS from Google   5.10.90+         containerd://1.5.4

[ibreakthecloud]

Based on my understanding of syft, anchore/stereoscope needs to support this.
ref: anchore/stereoscope#67

[spiffcs]

Thanks for filing the issue @ibreakthecloud! We'll take a look when we have some time and talk about what it would take to support pulling from the local containerd runtime rather than having syft fall back to the registry each time

[spiffcs]

Hey @ibreakthecloud! We're going to close this issue and follow the support for when we get the change merged in anchore/stereoscope#67

[mabilgen]

we have the same issue. we would like to create SBOM from inside container/pod using syft and get image from containerd deamon. But when we use syft scan -vv containerd:my-local-image:latest -o json it would try to pull image from registry which is already available locally.

Is this resolved ?

[chrisplo]

@mabilgen I dont' think so, see in stereoscope pullImageIfMissing . . I think the docker.io is prefixed before checking local daemon