BlogIf the Pod Never Restarts, Is It Still the Same Se...
kubernetesobservabilityin-place resizeDRA

If the Pod Never Restarts, Is It Still the Same Service?

K
August 21, 2026·5 min read

Kubernetes 1.37 is landing in the late-August 2026 window, the next cut on the project's four-month cadence after 1.34. The behavior that matters is not another KEP in the notes. It is this: a pod can change its CPU limit, its memory request, and its GPU claim while remaining Running. No restart. No reschedule. No new object for a classic check to notice.

In-place resize and Dynamic Resource Allocation are moving from feature-gated experiment toward default cluster behavior. Vertical Pod Autoscaler can write a new envelope onto a live container. A DRA ResourceClaim can bind a different device slice than the one you admitted. The process keeps its PID. kubelet reports Ready. The liveness probe still returns 200.

That is the missed signal.

Running is not an identity

Most cluster health models treat a stable pod as a stable service. Same UID. Same container ID. Restart count of zero. Phase Running. Ready true. If those hold, the workload is the thing you scheduled.

1.37 breaks that assumption without breaking the process.

The workload you admitted might have been 2 CPU, 4Gi, and an exclusive GPU. Twenty minutes later it can be 500m, 1Gi, and a shared MIG slice on a node that also hosts a training job. Nothing crashed. Nothing hit OOMKilled. The identity your alerts trust is still green. The envelope under the process is not the envelope you admitted.

We already argued that an autonomous system can stay reachable while its control surface disappears, in No Steering Wheel? Where Is Your Automation's Control?. Kubernetes is doing the equivalent at the cgroup layer. The same miss appears when monitoring keys off a name that no longer means what it used to, as in Can a Microsoft Rename Break Your Monitoring?. Here the name is Ready. The meaning moved.

What your rules still watch

Release roundups will list KEPs. They will skip the operational break: a large share of production alert rules still key off process death.

  • kube_pod_container_status_restarts_total increasing
  • reason CrashLoopBackOff
  • OOMKilled in lastState
  • liveness probe failures
  • pod phase not Running
  • Deployment available replicas dropping

Those signals fit the failure mode Kubernetes had for a decade. If the envelope is wrong, the kubelet kills the container, the replica set replaces the pod, and you notice because something restarted.

In-place mutation inverts that. The kubelet applies a new memory.max and cpu.max on the live cgroup. A DRA driver reallocates a device without deleting the pod. Checks that poll "is this process up?" keep succeeding. Prometheus rules stay silent because their predicates never became true.

kube-state-metrics will still export requests and limits. cAdvisor will still show usage. Usage against a moving limit is not the same as noticing that the limit moved. If you only graph CPU throttling after the fact, you are reading a symptom of a mutation you never recorded as a change.

Watch the live envelope, not the PID

Stop treating process existence as the health object. Treat the live resource envelope, and the DRA claims bound to it, as first-class health.

For every production pod you care about, answer four questions at any moment:

  1. What CPU and memory did we admit at schedule time?
  2. What CPU and memory does the kubelet currently enforce on the cgroup?
  3. Which ResourceClaims are bound, which devices are allocated, and are those the DeviceClasses we expected?
  4. Did any of those change while restart count stayed at zero?

If you cannot answer the fourth question, you cannot tell a live mutation from a stable service.

You already have the data. Compare spec.containers[].resources with status.containerStatuses[].resources and the pod resize condition. Treat kube_pod_container_resource_requests and kube_pod_container_resource_limits as time series, not gauges you ignore until a restart. Scrape ResourceClaim allocation and ResourceSlice updates. Read cgroup v2 memory.max and cpu.max from kubelet or node-exporter. Include neighbor claims on the same ResourceSlice. A shared GPU slice is a noisy-tenant problem even when your process is healthy.

The signal you want is not "pod not ready." It is "admitted envelope is not the live envelope, and no restart occurred." A second signal: DRA allocatedDevices drifted from the claim you bound at admit time.

A useful rule shape:

changes(kube_pod_container_resource_limits{resource="memory"}[15m]) > 0
and increase(kube_pod_container_status_restarts_total[15m]) == 0

That is a starting point, not a complete policy. You still have to separate intended mutations, a VPA you own, from unintended ones, a claim that silently lost a device. Intended still has to be visible. If VPA resized a latency-sensitive API from 2 CPU to 250m, the probe may keep passing while p99 falls off a cliff. A healthy probe on a smaller cgroup is not the same service.

For DRA, watch the claim, not only the pod. A pod can remain Running with a ResourceClaim that is allocated, pending, or bound to a different slice after a driver reconcile. Your inference pod can keep serving while a training job lands on the same GPU and tokens per second drop by half. No OOMKill. No restart. The missed signal is the claim graph, not the container state.

Flip the gates after you can see the mutation

The 1.37 notes and upgrade threads are landing this week. Feature-gate defaults get decided on the first staging cluster, not at the next freeze. Once in-place mutation is on, restart-centric monitors go quiet without anyone editing an alert.

Before you enable InPlacePodVerticalScaling or DRA on a cluster that serves customers:

  • Snapshot admitted requests, limits, and ResourceClaims for every Deployment and Job you treat as production. Store that snapshot as the baseline identity of the workload, not the pod UID.
  • Turn the four questions into a board a human can read in thirty seconds. If it cannot show that this Running pod is not the pod you admitted, it is not a 1.37 board.
  • Mark VPA in-place updates as explicit, named changes. A resize without an annotation or event you retain is silent drift, even if you wanted VPA on.
  • Put neighboring DRA consumers in the same view. Shared device health is cluster health, not a per-pod check.
  • Keep the old restart and OOM rules. They still catch the old failure mode. They are no longer sufficient.

The buyer question is not whether you have cluster metrics. You have those. kube-prometheus-stack has been shipping them for years. The question is whether you can see that this running workload is not the same workload you admitted.

If your health model cannot detect a live mutation, 1.37 does not add power. It adds silent drift.

If you already run Tink on a box, ask it for the live requests, limits, and device claims, not whether the process exists. Envelope identity is the 1.37 question.

If you are turning 1.37 on this week, start with the envelope check, not the restart graph.

Try Tink on your server

One command to install. Watches your server, explains problems, guides fixes.

Get started freeRead the docs

← Back to all posts