Image Scanning in Action: Securing Your Software Supply Chain

Introduction

“Are we using Log4j?”

This was the urgent question many IT security teams faced when the Log4Shell vulnerability (CVE-2021-44228) was disclosed. A Software Bill of Materials (SBOM) would have made answering it much easier.

With increasingly complex software supply chains, businesses and governments now require SBOMs to manage risk. For example, software vendors working with the federal government must provide them for transparency and security.

While using external components accelerates development and lowers costs, it exposes software to risks from common vulnerabilities (CVEs). Managing these evolving supply chains is nearly impossible without automation, given frequent updates, deployments, and new tools.

What is SBOM?

According to NTIA's SBOM FAQ, the official definition is as follows:

A Software Bill of Materials (SBOM) is a complete, formally structured list of components, libraries, and modules that are required to build (i.e. compile and link) a given piece of software and the supply chain relationships between them. These components can be open source or proprietary, free or paid, and widely available or restricted access.

• Source: NTIA SBOM FAQ
• More Information: NTIA Framing SBOM

Short Version:
A SBOM is a detailed list of all parts, libraries, and modules in software plus their supply chain ties—whether open source, paid, free, or restricted.

Importance of SBOM Scanning in Modern Software Development

It is much quicker and easier to scan a library of SBOMs than to scan your entire infrastructure from scratch. Tools like Trivy already support this. In the event of a zero-day—as we saw (and will continue to see) with Log4Shell—every minute counts.

SBOMs can also be leveraged by security teams to prioritize issues for remediation based on their presence and location and to create policies specific to component attributes such as vendor, version, or package type.

(Source: https://blog.gitguardian.com/what_is_sbom/)

Key Benefits of SBOM Scanning:

• Ensuring Vulnerability Management: Identifies and mitigates vulnerabilities to monitor and provide risk management.
• Regulatory Compliance: Simplifies meeting industry standards and audit readiness by providing detailed documentation of software components.
• Operational Efficiency: Streamlines dependency management, minimizes technical debt, and simplifies updates and patches.
• Supply Chain Transparency: Mitigates risks by revealing third-party component dependencies and potential vulnerabilities.

Objectives of This Blog Post

This blog post aims to provide an in-depth exploration of SBOM Scanning in the Wild, shedding light on its practical applications, challenges, and best practices in real-world scenarios. Specifically, we will:

• Demystify SBOM Scanning: Offer a clear understanding of what SBOM scanning entails, including its processes and benefits.
• Address Common Challenges: Identify and propose solutions to the typical obstacles faced when adopting SBOM scanning in diverse environments.
• Provide Best Practices and Recommendations: Share actionable insights and guidelines to help effectively implement and leverage SBOM scanning in the software development lifecycle.
• Real-World Example: Show a small example to integrate SBOM scanning into a microservice architecture.

Why SBOM?

Difficulty in Tracking Software Vulnerabilities
Modern applications rely on numerous dependencies, including hidden and transitive ones. Manually tracking vulnerabilities across all these components is impractical and error-prone. A SBOM provides a comprehensive inventory of all software components, making it easier to identify and manage vulnerabilities.

To effectively scan for known vulnerabilities, components within a SBOM are identified using standardized identifiers (such as CPE, PURL, or SWID). These identifiers are then analyzed against vulnerability databases to pinpoint known CVEs.

Deep Overview of Hidden Dependencies
A SBOM not only lists direct dependencies but also reveals hidden or indirect dependencies that might introduce vulnerabilities. This deep insight is crucial for a thorough security assessment to ensure no component is overlooked.

Automation and Standardization
SBOMs enable the automation of security scanning processes. By adhering to standardized formats and identifiers, tools can consistently interpret and analyze SBOMs across different platforms and ecosystems. This automation reduces manual effort and enhances the accuracy of vulnerability detection.

Scanning and Creation of SBOM
It is important to distinguish between creating and analyzing a SBOM:

• Creating a SBOM: Requires access to the host system or knowledge about the container layers. For example, you can add a SBOM to a container by generating it with BuildKit.
  (See: Docker Blog – “Generate SBOMs with BuildKit”)
• Analyzing or Scanning a SBOM: Involves using the SBOM text file and a continuously updated vulnerability database. The process compares identifiers in the SBOM with those in the database to detect potential vulnerabilities.

Understanding Vulnerability Management

A critical component of any secure software supply chain is clear visibility into detected vulnerabilities. An effective vulnerability management process helps organizations strengthen their security posture and reduce risk.

Goals of Vulnerability Management:

1. Early Detection:
    Identify vulnerabilities during development to reduce impact and cost.
2. Continuous Monitoring:
    Regularly scan code and infrastructure to catch new vulnerabilities.
3. Prioritization:
    Evaluate risks to focus on the most critical vulnerabilities first.
4. Automated Remediation:
    Automatically fix vulnerabilities to speed up response and ensure consistency.
5. Compliance and Reporting:
    Ensure adherence to standards and provide visibility into security status.

How SBOM Scanning Works in the DevOps Lifecycle

Integrating SBOM scanning into each phase of development strengthens security and ensures compliance. Below is an overview of key phases:

Software Component/Composition Analysis (SCA)

Software Composition Analysis automates application security by managing third-party and open-source components within the codebase. SCA identifies vulnerabilities in these components, mitigating risks such as supply chain attacks, and provides license information. Early implementation of SCA helps reduce security risks.

Incorporating SBOM scanning into the Integrated Development Environment (IDE) allows developers to catch vulnerabilities at the coding stage by interfacing with the language’s package manager. Practices like Git hooks, code linting, and secrets management complement this process.

Container Scanning

Containers have become a primary method for packaging and deploying applications. Container scanning focuses on:

• Detecting insecure containers (e.g., outdated libraries, misconfigurations, out-of-date operating systems)
• Validating compliance
• Recommending best practices

Note: The depth of scanning results varies by tool. Some organizations opt for distroless base images or paid services that offer vulnerability-free container registries.

Source

Image/Artifact Repositories

When images are pushed to a registry during CI, they are initially free of known vulnerabilities. However, as vulnerabilities evolve, continuous registry scanning is essential. Once new vulnerabilities are prioritized, images can be flagged or trigger remediation actions.

Live Production

Regular scanning in production is crucial since vulnerabilities may be discovered after deployment. Relying solely on build-stage scanning leaves a gap that continuous production monitoring can fill.

Key considerations include:

• What actions to take if a vulnerability is found in a container or host?
• How to manage runtime vulnerabilities (e.g., terminating vs. isolating a container)?
• Methods to isolate a vulnerable service.

For instance, a critical vulnerability might require an immediate redeployment with a patched image, whereas less severe issues might be monitored and patched during scheduled maintenance. In Kubernetes environments, strategies such as pod disruption budgets, canary deployments, or traffic shifting can minimize downtime during fixes.

Compliance and Reporting

SBOM scanning not only detects vulnerabilities but also supports compliance with regulatory and organizational requirements by:

1. Centralizing SBOM Inventory:
    - Automate SBOM generation at every build/deployment and store artifacts centrally.
    - Simplify audits with clear records of software components.
2. Aligning with Standards:
    - Demonstrate ongoing compliance with frameworks like NIST, ISO 27001, or PCI DSS by mapping SBOM findings to required controls.
    - Streamline reporting during security reviews or audits.
3. Automated Alerts and Dashboards:
    - Integrate SBOM scanning with project management and alert systems.
    - Present high-level metrics (e.g., vulnerabilities over time) along with detailed breakdowns for teams.
4. Real-Time Reports and Audit Trails:
    - Generate reports detailing discovered vulnerabilities, remediation actions, and timelines.
    - Provide evidence of continuous security monitoring for regulators and stakeholders.
5. Evidence for Legal and Regulatory Needs:
    - Archive SBOMs, scan logs, and incident reports to demonstrate due diligence during audits or breach investigations.

By embedding SBOM scanning into your DevOps workflow, you ensure continuous visibility, rapid remediation, and clear documentation—key pillars of a robust, compliant software supply chain.

A Short Overview of SBOM Tools in the DevOps Lifecycle

A number of tools can generate and analyze SBOMs, integrate scanning into CI/CD pipelines, and continuously monitor vulnerabilities. Your choice depends on your technology stack, container strategy, integration needs, and organizational constraints.

Selected Tools Include:

• Trivy: Open Source; unified scanning for vulnerabilities, IaC misconfigurations, and secrets; simple CLI; broad resource coverage.
• Syft: Open Source; specializes in generating SBOMs.
• Grype: Open Source; specializes in analyzing SBOMs.
• DependencyTrack: Open Source; focused on the CycloneDX standard with compliant reporting.
• Snyk: Commercial; developer-friendly with extensive integrations.

Sources:
• Trivy GitHub Repository
• Syft GitHub Repository
• Grype GitHub Repository
• DependencyTrack GitHub Repository
• Snyk CLI GitHub

Hands-on Trivy with k3d

Prerequisites:

• Docker installed
• k3d installed
• kubectl installed
• Helm installed

1. Create a k3d Cluster Configuration

Create a custom configuration file named k3d-config.yaml with the following content:

# k3d-config.yaml
apiVersion: k3d.io/v1alpha5
kind: Simple
metadata:
  name: trivy-demo
servers: 1
agents: 1
ports:
  - port: 1080:80
    nodeFilters:
      - loadbalancer

Create the cluster

The Trivy operator needs at least 500mb RAM, to prevent OOM exceptions the nodes are specified with 1GB memory.

k3d cluster create --config k3d-config.yaml --agents-memory 1G --servers-memory 1G

Verify the cluster is running:

k3d cluster list
kubectl get nodes

2. Install the Trivy Operator

Add the Helm chart repository for Aqua Security and update:

helm repo add aqua https://aquasecurity.github.io/helm-charts/
helm repo update

Install the Trivy operator:

helm install trivy-operator aqua/trivy-operator \
  --namespace trivy-system \
  --create-namespace

Check that the operator is running:

kubectl get pods -n trivy-system

You should see an output similar to:

NAME                                  READY   STATUS    RESTARTS   AGE
trivy-operator-xxxx-xxxxx             1/1     Running   0          1m

3. Deploy a Test Application with Known Vulnerabilities

Create a file named deployment.yaml with the following content:

# deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: vulnerable-nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app: vulnerable-nginx
  template:
    metadata:
      labels:
        app: vulnerable-nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.14.2
        ports:
        - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: vulnerable-nginx
spec:
  type: NodePort
  selector:
    app: vulnerable-nginx
  ports:
  - port: 80
    targetPort: 80
    nodePort: 30080

Apply the deployment:

kubectl apply -f deployment.yaml

Verify the pods (if using a specific namespace, adjust accordingly):

kubectl get pods -n vulnerable-nginx-ns

4. Check the Vulnerability Reports

4.1 List the Reports:

kubectl get vulnerabilityreports -n vulnerable-nginx-ns -o wide

or

kubectl get configauditreports -n vulnerable-nginx-ns -o wide

You should see a report similar to:

NAME                                                       REPOSITORY        TAG     SCANNER   AGE
vulnerable-nginx-xxxxxxx-default-vulnerable-nginx          nginx             1.14.2  Trivy     1m

4.2 Inspect a Specific Report:

kubectl describe vulnerabilityreports <report-name> -n vulnerable-nginx-ns

For example:

kubectl describe vulnerabilityreports vulnerable-nginx-xxxxxxx-default-vulnerable-nginx -n vulnerable-nginx-ns

This will display details such as the image (e.g., nginx:1.14.2), CVE IDs, severity, package versions, and available fixes.

5. Viewing SBOM (Software Bill of Materials)

List SBOM reports:

kubectl get sbomreports

Inspect a SBOM report:

kubectl describe sbomreports <report-name>

6. Clean Up

To clean up the environment, run:

# Remove test resources
kubectl delete -f deployment.yaml

# Uninstall Trivy operator
helm uninstall trivy-operator -n trivy-system
kubectl delete ns trivy-system

# Delete k3d cluster
k3d cluster delete trivy-demo

7. Repository Structure

A suggested structure for the demo project:

.
├── k3d-config.yaml
└── deployment.yaml

8. Summary

In this demo, you have:

1. Created a local Kubernetes cluster using k3d.
2. Installed the Trivy operator for vulnerability and SBOM scanning.
3. Deployed a vulnerable nginx container.
4. Reviewed the vulnerability reports and SBOMs.

This setup demonstrates real-time SBOM and vulnerability scanning in an environment that simulates production.

What is the Future of SBOMs?

(Source: https://scribesecurity.com/blog/insights-from-cisa-new-guide-the-future-of-sbom/)

SBOMs are evolving alongside modern computing, with several upcoming trends:

• Automation and Integration:
   Emerging SBOMs (e.g., ML-BOM for machine learning tools) will heavily rely on automation.
• Expanded BOM Types:
   New formats such as KBOM (Kubernetes BOM), IBOM (Infrastructure BOM), and others are being developed to better categorize and prioritize complex software structures.
   (See: KBOM)
• Industry Standardization:
   Ongoing efforts to standardize SBOM formats (like CycloneDX) will simplify data exchange and interpretation across the software ecosystem.
• CycloneDX Innovations:
   Projects under CycloneDX are developing specialized BOMs (e.g., HBOM, OBOM) and are working on new ways to analyze and exchange vulnerability information, including:
  • Vulnerability Exploitability eXchange (VEX)
  • Vulnerability Disclosure Report (VDR)
  • Bill of Vulnerabilities (BOV)
  • BOM-Link
• Cryptographic BOM (CBOM):
   An important development for ensuring security in a post-quantum world.