Kubernetes News

The Kubernetes blog is used by the project to communicate new features, community reports, and any news that might be relevant to the Kubernetes community.
The Kubernetes project logo

  • Kubernetes v1.37: KubeletInUserNamespace (aka Rootless mode) Graduates to Beta

    Kubernetes v1.37 promotes the KubeletInUserNamespace feature gate to beta. With this feature enabled, all of the node components (kubelet, CRI and OCI runtimes, CNI plugins, and kube-proxy) can run as a non-root user on the host, using a Linux user namespace. This technique is also known as rootless mode. The work started as an experiment in 2018, and was merged into Kubernetes v1.22 (2021) as an alpha feature (Kubernetes Enhancement Proposal KEP-2033).

    This feature should not be confused with user namespaces for pods (hostUsers: false with the UserNamespacesSupport feature gate, GA since v1.36), which puts pods in user namespaces but still runs the node components as root. These two features do not conflict. Moreover, they can be combined to nest Kubernetes inside Kubernetes without resorting to the full privileged: true.

    Why run the node components in a user namespace?

    Because the node components have historically had container-breakout vulnerabilities that could compromise full root privileges on the host.

    Examples of such vulnerabilities include:

    • CVE-2022-0811 ("cr8escape"): CRI-O could be tricked into setting arbitrary sysctls, such as kernel.core_pattern, resulting in arbitrary code execution as root on the host
    • CVE-2023-27561: runc could be tricked into bypassing the masked paths of a container via a volume mount race, exposing the host's procfs files (a regression of CVE-2019-19921)
    • CVE-2024-10220: the kubelet could be made to execute arbitrary commands as root via gitRepo volumes (gitRepo volumes had a similar vulnerability, CVE-2018-11235, back in 2018 too)
    • CVE-2025-31133: runc could be tricked into bind-mounting attacker-controlled paths and writing to the host's procfs files, such as /proc/sysrq-trigger and /proc/sys/kernel/core_pattern
    • CVE-2026-53488: containerd could be tricked into executing arbitrary commands on the host, via crafted labels in a container image

    By running the node components in a user namespace, the potential damage is confined to the non-root user's account. Notably, an attacker cannot conceal their intrusion by modifying the kernel, the boot loader, or the firmware.

    It should still be noted that user namespaces are not effective for mitigating vulnerabilities in the kernel itself. User namespaces should be used in conjunction with traditional hardening measures such as seccomp to prevent containers from invoking unnecessary system calls.

    Use cases

    • Production clusters: mitigate potential container-breakout vulnerabilities.
    • Shared machines (e.g., HPC): users can deploy Kubernetes without asking the machine administrator for root privileges, and without the risk of accidentally breaking other users' environments.
    • Laptops: prevent a local cluster from accidentally breaking the host system configuration, e.g., the host iptables rules used for VPNs.
    • AI sandbox: a Kubernetes application developer may create a dedicated local user account for running an AI coding agent and a test Kubernetes cluster. This setup is useful for preventing the AI agent from breaking the host when it is deceived by malicious information on the Internet.
    • Kubernetes-in-Kubernetes: a nested cluster can run inside a parent cluster as a user-namespaced pod (hostUsers: false), isolating workloads more strictly than Kubernetes API namespaces do.
    • Bootstrapping: a temporary unprivileged cluster can be used to bootstrap an actual cluster, e.g., with Cluster API.

    How does it work?

    A Linux kernel user namespace maps a host level non-root user (e.g., UID 1000) to a fake root user inside the namespace. The UID 0 privileges are limited to the inside of the namespace. The fake root is enough for most of the node components' tasks: mounting volumes, creating cgroups, and configuring the network namespaces of pods. It still comes with some caveats that may break compatibility with specific CNI and CSI drivers, though.

    The user namespace has to be created outside of Kubernetes. For example, Rootless Docker can be used to prepare the user namespace in which Kubernetes runs.

    The KubeletInUserNamespace feature gate itself is quite "boring": basically it just lets the kubelet ignore permission errors that occur when setting some sysctl values (e.g., vm.overcommit_memory and kernel.panic) and when watching kernel messages via /dev/kmsg.

    See Running Kubernetes Node Components as a Non-root User for further information.

    What changed from Alpha to Beta?

    • The KubeletInUserNamespace feature gate is now enabled by default. Enabling the gate does not put the kubelet into a user namespace automatically, so nothing changes for existing "rootful" clusters.
    • kubectl get nodes -o yaml now reports whether nodes are running in a user namespace via the runningInUserNamespace property. A cluster administrator can use this property to set node labels or taints, to avoid scheduling workloads that need real root privileges (e.g., some CNI plugin installers) onto rootless nodes.
    • For Kubernetes' own CI/CD testing, the node conformance end to end tests now run on a rootless cluster (ci-kubernetes-e2e-kind-rootless).

    Several related improvements have also happened outside the promotion of the feature gate itself:

    • Linux kernel v6.3 (2023): added support for idmapped tmpfs.
    • Kubernetes v1.33 (2025): enabled the UserNamespacesSupport feature gate by default, allowing user-namespaced pods (hostUsers: false) to be created without extra configuration.
    • containerd v2.1 (2025): added support for writable cgroups.

    With these improvements, a Kubernetes cluster with KubeletInUserNamespace can now also be nested inside Kubernetes pods with hostUsers: false (UserNamespacesSupport).

    How to use it

    kind

    The easiest way is to use kind (a Kubernetes SIG Testing project) to run a Kubernetes cluster in rootless Docker, rootless nerdctl, or rootless Podman:

    # Example using Docker
dockerd-rootless-setuptool.sh install
kind create cluster

    Depending on the host configuration, you may need additional configuration for systemd, kernel modules, sysctl, etc.

    See the Docker documentation and the kind documentation for further information.

    minikube

    minikube (a Kubernetes SIG Cluster Lifecycle project) also supports running a Kubernetes cluster in rootless Docker or rootless Podman:

    dockerd-rootless-setuptool.sh install
minikube start --driver=docker

    See the minikube documentation for further information.

    Usernetes

    Usernetes (a third-party project) is a distribution of rootless Kubernetes, maintained by the author of this article. The project began in 2018, and it is where the KubeletInUserNamespace feature gate originally came from.

    Unlike kind and minikube, Usernetes supports creating a cluster with multiple rootless Docker / Podman / nerdctl nodes, connected using VXLAN via the Flannel CNI plugin.

    Usernetes also experimentally supports a Kubernetes-in-Kubernetes mode.

    k3s

    k3s (a CNCF Sandbox project) also supports rootless mode. Unlike kind, minikube, and the current generation of Usernetes, rootless k3s does not rely on an external runtime such as rootless Docker.

    What's next?

    Depending on feedback and adoption, the Kubernetes project plans to graduate this feature to General Availability (GA) in a future release. If you have feedback on this feature, please open an issue in the kubernetes/kubernetes repository.

    The project is also discussing several Kubernetes Enhancement Proposals that may contribute to simplifying Kubernetes-in-Kubernetes with this feature:

    Getting involved

    We always welcome new contributors. If you would like to get involved, you can join the Node Special Interest Group (SIG Node).

    If you would like to share feedback, you can do so on our public Slack channel (visit https://slack.k8s.io/ for an invitation if you need one).

    Special thanks to everyone who helped design and implement this feature, including but not limited to (in alphabetical order):

  • Kubernetes v1.37: DRA Updates

    Kubernetes 1.37 is here and Dynamic Resource Allocation (DRA) keeps pushing past where it started! This release brings DRA Extended Resource support to GA, a milestone the team has been building toward for three straight releases. Several more features graduate to Beta or GA. A fresh batch of alpha features rounds out the release.

    I'll dive into what's new for DRA in Kubernetes 1.37!

    What's stable in 1.37

    DRA Extended Resource support has graduated to GA. This is the mechanism that lets DRA drivers satisfy requests made through the traditional extended resource API, think example.com/gpu in a Pod spec, without requiring a separate device plugin alongside the DRA driver. An extended resource name can be set directly on a DeviceClass, and Pods requesting it get matched to a device through DRA with no ResourceClaim needed on the workload's part.

    It's been on a steady path since KEP acceptance in 1.34. Alpha landed in 1.35, Beta in 1.36, and now it's Stable. For cluster operators, this is what makes DRA adoption gradual. Existing workloads written against extended resources keep working unmodified while the backend allocation logic moves over to DRA.

    ResourceClaims status with possible standardized network interface data adds a devices field to ResourceClaim .status, letting DRA drivers report per-device status, including, for network devices, the interface name, MAC address, and IP addresses. This gives users and controllers visibility into device state that was previously invisible once a device was configured in a Pod, and makes it possible to build things like network services that rely on a device's reported IPs.

    DRA: device taints and tolerations is now Stable; DRA drivers can mark devices as tainted so they're skipped for new Pod scheduling, and cluster admins can apply the same taints cluster-wide via a DeviceTaintRule, without reconfiguring drivers. Pods already using a tainted device can be evicted automatically, unless their ResourceClaim explicitly tolerates the taint. This mirrors node taints and tolerations, letting operators take a single device offline for maintenance or mark it degraded, without disrupting the rest of the cluster.

    Standard numaNode device attribute standardizes resource.kubernetes.io/numaNode as a shared attribute name, so devices from different drivers can be compared on the same NUMA node instead of each driver inventing its own name for it. It landed directly as stable in 1.37, since it's a naming/registration KEP with no feature gate or in-tree behavior change.

    Feature promoted to Beta

    ResourceClaim support for workloads graduates to Beta behind the DRAWorkloadResourceClaims feature gate, which stays disabled by default. In a cluster that has the feature enabled, Workloads and PodGroups can reference ResourceClaims directly, so a single claim can be shared across an entire group of Pods. This is instead of claims being capped at 256 Pods through the old per-Pod reservation limit.

    The DRA Device Attributes Downward API is aimed at supporting device injection into KubeVirt VMs. Drivers populate a Metadata field when preparing a claim, and the framework writes it to a JSON file mounted into the container via CDI, letting workloads read a device's PCI bus address, MAC address, and other attributes directly instead of requiring custom controllers to watch and translate ResourceClaims and ResourceSlices.

    Alpha features

    List types for attributes moved into a second Alpha in 1.37, letting a device attribute hold more than one value instead of a single scalar, such as a CPU that's adjacent to more than one PCIe root. This makes it possible to match or distinguish devices based on overlapping or non-overlapping sets of values, while single-value attributes keep working as they do today.

    Node allocatable resource requests moved into Alpha 2. It lets the scheduler and kubelet treat DRA-managed CPU, memory, and similar node resources the same way they treat ordinary resource requests, so a node doesn't get oversubscribed and users no longer have to duplicate the same request in both a ResourceClaim and the pod spec.

    Resource availability visibility moved to a second Alpha in Kubernetes 1.37. Users create a ResourcePoolStatusRequest to get a point-in-time availability snapshot. To refresh it, delete and recreate the request; it is not a continuous monitoring API.

    DRA: Optional Node Operations lets a driver skip kubelet's prepare and unprepare calls for allocations that don't need any setup on the node. This makes it possible to avoid an unnecessary dependency on the driver for allocations where there's genuinely nothing for it to do locally.

    Derived Attributes is a new feature that lets you use CEL expressions to match up devices based on your own custom rules. Before this, pairing devices from different vendors (like a GPU/TPU and a NIC on the same NUMA node) only worked if both drivers used the exact same attribute name. If one used numa and the other used numaNode, the scheduler couldn't pair them together. Now, you can easily bridge these differences yourself inside your manifest, meaning you don't have to wait for hardware vendors to agree on standardized attribute names. Beyond just fixing naming differences, you can also use CEL to handle more complex scenarios like slicing a specific ID out of a long, monolithic topology string, or grouping devices into custom performance tiers based on their available capacity.

    DRA Device Compatibility Groups lets drivers tag partitions of a device, like MIG vs vGPU profiles on the same GPU, with compatibility groups, so the scheduler rejects incompatible combinations up front instead of the driver failing at node preparation time. It's controlled by the DRADeviceCompatibilityGroups feature gate, disabled by default.

    PreQueueingHint extension point is new as Alpha in 1.37. DRA ResourceClaim events used to trigger a full scan of every unschedulable pod, an O(N²) cost during large scale-ups. The DRA plugin now uses a pod informer index to narrow that to just the pods actually affected, cutting the requeue path to O(1) and roughly doubling scheduling throughput in early benchmarks. Controlled by the SchedulerPreQueueingHints feature gate.

    DRA Consumable Capacity now supports fractional values in CapacityRequestPolicyRange, enabling more precise capacity requests and allocation for devices with fractional resources. This improves flexibility for workloads that require fine-grained resource allocation. The enhancement is gated by the DRAFractionalCapacityRange feature gate, which is in Beta in 1.37.

    What’s next

    DRA continues to mature with every release. Several features currently in Alpha and Beta are on track to progress in the coming releases, and the community keeps working on DRA's performance, scalability, and reliability. Expect another ambitious set of DRA features in Kubernetes 1.38.

    Getting involved

    A good starting point is joining the WG Device Management Slack channel and meetings which happens at US/EU and EU/APAC friendly time slots.

    Not all enhancement ideas are tracked as issues yet, so come talk to us if you want to help or have some ideas yourself! We have work to do at all levels, from difficult core changes to usability enhancements in kubectl which could be picked up by newcomers.

    Acknowledgments

    The following KEP owners added or promoted a feature in the 1.37 release (in alphabetic order):

    This would not have been possible without the help of the reviewers and approvers. So a huge thanks to everyone else who helped shape this release, in ways big and small. Given enough eyeballs, all bugs are shallow and this release had plenty of them, watching closely and caring enough to make things better. DRA got better this cycle because of all of you.

  • Kubernetes v1.37: Scale Workloads to Zero with HorizontalPodAutoscaler

    Kubernetes v1.37 includes API support for horizontal autoscaling of workloads down to zero replicas. This feature is now Beta and enabled by default. A HorizontalPodAutoscaler (HPA) that uses a suitable object metric or external metric can now scale a workload to zero replicas, then bring it back when the metric changes.

    Before v1.37, you needed an add-on or external component, or you had to enable the Alpha feature gate, to scale from zero. It is now part of core Kubernetes.

    Scaling to zero removes the last idle Pod from workloads such as queue consumers and batch processors. The savings are largest when each Pod reserves expensive resources, including dedicated CPUs or GPUs.

    The trade-off is cold-start time: the HPA must observe the metric, schedule a Pod, and start the application. This works well when work can wait in a durable queue.

    Kubernetes Services do not buffer requests while no Pods are ready, so HTTP and other request-driven workloads need a separate buffering layer.

    Why scaling from zero needs a different metric

    The HPA commonly scales on CPU or memory usage. Both metrics come from running Pods. Once the replica count reaches zero, there are no Pods left to measure and no signal that can tell the HPA to scale back up.

    Object and external metrics do not have that limitation. A queue length, for example, exists independently of the workers that consume it. The HPA can continue reading the queue length while no workers are running.

    The following example scales a queue consumer to and from zero using an external metric.

    Configure an external metric

    The following example uses a Prometheus metric named queue_consumer_lag. It assumes that Prometheus already collects a series similar to this one:

    queue_consumer_lag{namespace="default",name="worker_tasks"}

    Kubernetes needs a metrics adapter to make that value available through the External Metrics API. One implementation is the Prometheus Adapter, which can expose the series using an externalRules entry:

    externalRules:
- seriesQuery:'{__name__="queue_consumer_lag",name!=""}'
metricsQuery:sum(<<.Series>>{<<.LabelMatchers>>}) by (name)
resources:
overrides:
namespace:
resource:namespace

    The exact adapter installation and discovery rules depend on your monitoring setup. See the Prometheus Adapter guide to external metrics for the full configuration options.

    Before creating the HPA, you can verify that Kubernetes can read the metric:

    kubectl get --raw \
 '/apis/external.metrics.k8s.io/v1beta1/namespaces/default/queue_consumer_lag?labelSelector=name%3Dworker_tasks'

    The request should return the current value for worker_tasks. If it does not, fix the metrics pipeline before configuring the HPA. An HPA cannot scale from zero when its metric is unavailable.

    Configure the HPA

    The following HPA targets a Deployment named queue-worker. It allows between zero and ten replicas, with one replica requested for each 30 queued tasks:

    apiVersion:autoscaling/v2
kind:HorizontalPodAutoscaler
metadata:
name:queue-worker
annotations:
kubernetes.io/description:"Scales queue-worker based on the number of queued tasks"
spec:
scaleTargetRef:
apiVersion:apps/v1
kind:Deployment
name:queue-worker
minReplicas:0
maxReplicas:10
metrics:
- type:External
external:
metric:
name:queue_consumer_lag
selector:
matchLabels:
name:worker_tasks
target:
type:Value
value:"30"

    When the queue is empty, the HPA can reduce the Deployment to zero replicas. When tasks arrive, the external metric remains available and the HPA calculates a new replica count, capped at ten by maxReplicas.

    Start the Deployment with at least one replica. Manually setting a Deployment to zero has always paused autoscaling. The HPA preserves that behavior and will not wake a workload that it did not scale down itself.

    Normal HPA behavior still applies. In particular, the default downscale stabilization window is five minutes. The window prevents a short drop in queue length from immediately removing all workers. You can configure the window through spec.behavior.scaleDown if your workload needs different behavior.

    How the HPA distinguishes zero from paused

    Scaling from zero creates an ambiguity. A replica count of zero can mean that the HPA scaled the workload down, or that an operator manually paused it.

    The controller resolves this with a ScaledToZero status condition. When the HPA scales a workload from one or more replicas to zero, it records ScaledToZero=True. The condition tells later reconciliation loops that the controller owns the zero state and should continue evaluating object or external metrics.

    After scaling the workload back up, the controller changes the condition to ScaledToZero=False with the reason NotScaledToZero. A workload at zero without the ScaledToZero=True condition remains paused.

    You can inspect the conditions with:

    kubectl describe hpa queue-worker

    If the adapter cannot return the configured metric, the HPA reports ScalingActive=False with a reason such as FailedGetExternalMetric. Restore the metric or manually scale the workload to recover capacity.

    Before upgrading or rolling back

    In Kubernetes v1.37, the HPAScaleToZero feature gate is enabled by default on both the kube-apiserver and kube-controller-manager. The API server accepts minReplicas: 0; the controller manager performs the condition-based scaling.

    During a version-skewed control plane upgrade, wait until both components support the feature and have it enabled before creating HPAs with minReplicas: 0. A controller manager with the feature disabled treats replicas: 0 as a manual pause and may leave a workload at zero.

    Before disabling the feature gate or downgrading to a version without the condition-based implementation:

    • Change affected HPAs to minReplicas: 1 or higher.
    • Scale any workload currently at zero to at least one replica.

    minReplicas: 0 also requires at least one object or external metric. The API server rejects an HPA that only contains resource metrics such as CPU or memory.

    From Alpha to Beta

    The first Alpha implementation shipped in Kubernetes v1.16. Kubernetes v1.36 added the ScaledToZero condition and the controller behavior needed to distinguish an automatic scale-down from a manual pause.

    Kubernetes v1.37 enables the feature by default after adding integration and end-to-end coverage for scaling down to zero and back up from an external metric. The next step is to gather operational feedback before considering graduation to GA.

    How can I learn more?

    How to get involved

    This feature is owned by SIG Autoscaling. Join Kubernetes Slack and the #sig-autoscaling channel to share feedback from Beta usage.

    Acknowledgements

    Thanks to the SIG Autoscaling contributors who took this feature from the original v1.16 implementation to the condition-based redesign and Beta graduation. Thanks also to Guy Templeton and Adrian Moisey for reviewing the KEP, and to the release, documentation, and production-readiness reviewers who helped prepare it for Kubernetes v1.37.

  • Kubernetes v1.37: etcd RangeStream Cuts Memory Use on Large List Reads

    I am excited to announce that etcd RangeStream is graduating to beta in Kubernetes v1.37. Paired with etcd v3.7, it reduces the memory the API server and etcd need to read a large collection, and makes peak usage more predictable.

    The cost of large reads

    The API server serves most list and watch requests from its in-memory watch cache. Populating that cache requires reading a resource's full state from etcd, at startup and on every re-initialization. For a resource with many objects, or large ones, such as Pods, that read is expensive.

    The API server already paginated these reads, asking etcd for a fixed number of keys at a time rather than the whole collection at once. But a page bounded by key count has no awareness of object size, so a page of large objects can still be very large. That makes memory usage hard to predict, and a bad combination of object size and concurrent reads can be enough to trigger an OOM. etcd's unary Range assembles each page in full before sending it, and the API server holds it while decoding, so the same payload sits in memory on both sides at once. Most of that cost lands on etcd, which is also where streaming helps most.

    Streaming reads with RangeStream

    etcd v3.7 adds a streaming version of that read, the RangeStream RPC. It takes the same RangeRequest as Range and returns the same result set, but instead of building the whole response up front, etcd splits it into chunks and streams them. Chunk size is tuned adaptively to the values being returned, so a collection of large objects is bounded by bytes rather than by a key count, and memory is freed as the stream progresses instead of being held until a whole page is assembled.

    When the feature is enabled, the API server uses RangeStream wherever it reads a whole collection out of etcd. This includes watch cache initialization, and the fallback paths where a list request cannot be served from the cache and reads etcd directly. In either case the API server decodes each chunk as it arrives and releases it before pulling the next one, so neither side ever holds the whole collection.

    Requirements

    • Kubernetes v1.37 or later
    • etcd v3.7 or later

    RangeStream is used when the EtcdRangeStream feature gate is enabled on the kube-apiserver, which is beta and on by default in v1.37, and etcd is v3.7 or later. The API server resolves etcd's support at startup and also falls back at runtime if a call returns Unimplemented, so an API server paired with an older etcd keeps using the paginated Range path on its own. To turn it off, disable the gate:

    --feature-gates=EtcdRangeStream=false

    Confirming RangeStream is in use

    The API server records streamed reads under their own operation label on its etcd metrics. A non-zero count here means RangeStream is in use:

    etcd_request_duration_seconds_count{operation="listStream"}

    If it stays at zero, the API server is still using the paginated Range path, most likely because etcd is older than v3.7.

    Learn more

    If you have questions or feedback, join the #sig-etcd channel on Kubernetes Slack.

  • Kubernetes v1.37: Storage Version Migration Enabled by Default

    I am excited that storage version migration (SVM) has graduated to General Availability (GA) in Kubernetes v1.37!

    After a number of releases of work and testing, the built-in StorageVersionMigration API (storagemigration.k8s.io/v1) and control plane controller are now fully stable and enabled by default across all v1.37 Kubernetes clusters.

    The problem with stale storage versions

    In Kubernetes, stored API resources are written using a specific storage version (schema representation). The way Kubernetes interacts with object storage fundamentally requires mutation of a resource in order to ensure that the latest storage version is used for all resources. This creates problems when you want to change the storage version of a resource.

    One example of a scenario where you may want to change the storage version of a resource is when you are promoting a CRD to drop an older API version (such as v1alpha1) to a newer version (leaving just v1beta1 and v1). It's a problem to drop the older API version whilst there are still resources stored with the old alpha version.

    To avoid problems, you designate v1 as the new storage version; but, on it's own, that's not enough. While new writes are stored as v1, any existing resource could remain stored as v1alpha1 or v1beta1 in storage. You cannot safely remove v1alpha1 from the CRD's .status.storedVersions or drop serving support until every single resource in storage has been re-written to not be serialized and stored with the alpha version.

    Another relevant example is encryption at rest and, related, key rotation. When you configure encryption at rest or rotate encryption keys, existing resources in storage remain unencrypted (or encrypted under old keys) until they are actively re-written through the Kubernetes API server.

    Historically, cluster administrators and CRD authors had to rely on manual kubectl get / kubectl replace scripts, or to deploy the out-of-tree kube-storage-version-migrator component to force re-writes. These approaches were often tedious, error-prone, and difficult to monitor.

    How storage version migration works

    Initiating a storage version migration is as simple as creating a declarative StorageVersionMigration object. The built-in StorageVersionMigrator controller in the Kubernetes control plane watches for these objects, and automatically migrates existing resources to the default storage version for that API.

    Example: Migrating a custom resource API

    Suppose you have updated a CustomResourceDefinition (crontabs.example.com) to use v1 as its storage version. To migrate all existing stored resources off older versions, create a StorageVersionMigration:

    apiVersion:storagemigration.k8s.io/v1
kind:StorageVersionMigration
metadata:
name:crontabs-migration
spec:
resource:
group:example.com
resource:crontabs

    Apply the manifest using kubectl:

    kubectl apply -f crontabs-migration.yaml

    Monitoring and verifying migrations

    The StorageVersionMigrator controller updates the status of the StorageVersionMigration object as migration progresses. You can inspect the migration status using kubectl:

    kubectl get storageversionmigration.storagemigration.k8s.io/crontabs-migration -o yaml

    A successful migration will report a Succeeded condition set to True:

    status:
conditions:
- type:Running
status:"False"
lastUpdateTime:"2026-08-02T10:05:00Z"
reason:StorageVersionMigrationInProgress
- type:Succeeded
status:"True"
lastUpdateTime:"2026-08-02T10:05:00Z"
reason:StorageVersionMigrationSucceeded

    Once the migration has succeeded, you can be confident that all instances of the resource in storage are stored in the current storage version. For CRDs, the stored version should be updated in the CRD's .status.storedVersions to only contain the preferred version. If the .status.storedVersions is not updated following a successful migration then that means that the CRD was updated during the migration. In that case, the migration should be retried in order to safely deprecate an older storage version.

    Including migrations in your CRD manifests

    Because StorageVersionMigration is a standard declarative Kubernetes API, CRD authors can bundle or trigger migrations directly alongside CRD upgrades. For example, you can include the migration in the same manifest as your updated CustomResourceDefinition:

    apiVersion:apiextensions.k8s.io/v1
kind:CustomResourceDefinition
metadata:
name:crontabs.example.com
spec:
group:example.com
# Updated versions list where v1 has storage: true
...
---
apiVersion:storagemigration.k8s.io/v1
kind:StorageVersionMigration
metadata:
name:crontabs-migration
spec:
resource:
group:example.com
resource:crontabs

    What's next?

    SIG API Machinery would love to hear your feedback as you adopt built-in Storage Version Migration in your clusters. Reach out to us on the #sig-api-machinery Slack channel or participate in our community discussions!