Filesystem Persistence Across Code Sandbox Restarts

Separating filesystem storage from compute lifecycle prevents agent work from vanishing on restart.

· 8 min read
Cover illustration for “Filesystem Persistence Across Code Sandbox Restarts”
Sandboxed Code Execution · October 23, 2026 · 8 min read · 1,811 words

Filesystem persistence across code sandbox restarts depends on separating the filesystem lifecycle from the compute lifecycle. Persistent AI agent infrastructure preserves accumulated work while compute stops or moves, whereas serverless functions alone do not provide that continuity. Long-running agents fail when a timeout or restart destroys their workspace, even if the next invocation starts successfully.

Why treating the sandbox as the unit of state creates a hidden failure mode

An agent clones a repository, installs dependencies, and works through a fix, only to resume after a restart in an empty workspace. The sandbox is available, but its work has disappeared. Nothing needs to crash for this failure to occur. PingCAP describes ephemeral sandboxes as tying the file layer to compute, so a timeout or crash can erase hours of agent work without warning.

The root cause is an architectural assumption. A disposable execution environment has been treated as the owner of durable work, including repositories, configurations, logs, patches, and cached dependencies. Rebuilding that workspace spends tokens and compute on ground already covered. Compute should be disposable so that untrusted code can start, run in isolation, and stop. The filesystem needs a separate lifecycle because it contains the work that the next execution must reuse.

Three distinct things that can be "lost" in a restart, and why saving one does not save the others

A restart can destroy filesystem state, process state, and in-memory state together, but these are separate losses. Filesystem state includes installed packages, cloned repositories, output artifacts, and credentials cached to disk. Process state covers the process tree that is running, background jobs, open sockets, and a REPL with context already loaded. In-memory state covers variables, loaded models, and conversation buffers that were never flushed to disk. The boundaries overlap operationally, but their recovery guarantees differ.

Saving one layer does not save the others. A persistent volume can retain a REPL’s files while losing the running interpreter, its variables, and its open connections. Conversely, preserving memory without disk persistence does not retain installed packages. Even a memory snapshot that preserves process state may leave an external connection needing to be re-established. A claim that an agent “resumes” therefore says little until it identifies which layers survive and which must be reconstructed.

How snapshot-restore decouples the filesystem from the compute boundary

Snapshot-restore makes retained state independent of a running compute instance. A full sandbox snapshot captures guest memory, virtual device states, CPU registers, and disk files, then writes that state to durable storage. The compute instance can be destroyed while the snapshot remains available for restoration on a suitable host. This is more than keeping a directory intact. It preserves a picture of execution as well as the files that execution produced.

Restoration also differs from booting an empty environment. Memory can be initialized through mmap rather than reading the full memory file upfront, which enables fast restoration. A cold Firecracker boot takes meaningfully longer than a snapshot restore, so those measurements are not interchangeable when evaluating an interactive agent. The delay to measure is the time from a wake request until code runs again. For agents that wake often, this lets compute be released between interactions without a full cold-boot path each time they return.

Persistent standby versus hibernation: two different answers to the same problem, optimized for different workloads

Persistent standby and hibernation offer different ways to retain a usable session. With persistent standby, the VM stays fully resident on its host, ready for the next turn but consuming memory and CPU while idle. With hibernation, the sandbox is snapshotted and paused, then restored when another request arrives. Hibernation favors lower compute cost between bursts, at the expense of restore latency.

A single tool call, stateless function evaluation, or one-shot code-interpreter answer has no retained session waiting for another turn, so ephemeral execution is appropriate. An iterating coding agent, live browser session, or REPL-driven analysis keeps a session for the next turn. Frequent but unpredictably timed interactions favor fast-restore hibernation when idle periods dominate. Across a fleet, that choice lets host count follow active concurrency rather than retained session count, because idle sandboxes can be evicted and empty hosts removed.

Copy-on-write storage makes snapshot-based persistence affordable at real filesystem sizes

Copying an entire disk at every checkpoint would make snapshotting expensive as workspaces grow. Copy-on-write storage saves data that has not changed just once. Content that has not changed stays shared, while a new snapshot saves the parts that changed and points to the rest. Checkpoint size and the cost of retaining a session depend on the session’s changes.

Cortex hashes and uploads only dirty chunks at snapshot time, then publishes a new version of the session’s manifest. Untouched offsets continue to reference shared base chunks. In the described case, a session that dirties a small number of chunks adds at most 320 MiB of new disk data even when the guest sees a 128 GiB disk. Tencent Cloud’s CubeSandbox v0.3.0 uses a similar principle through its CubeCoW engine, combining a reflink volume with soft-dirty memory tracking.

What the billing model reveals about whether a platform has truly separated storage from compute

The billing model exposes whether pausing an agent actually releases compute. If idle time runs the same meter as active execution, keeping state still means paying for the running sandbox. A longer timeout does not change that relationship. Genuine separation allows compute billing to stop while storage billing continues for persisted state. The decisive distinction is therefore between a running-but-idle environment and one whose compute has been released.

Vercel Sandbox bills active CPU at a per-vCPU-hour rate only while code executes, while provisioned memory is billed for the full wall-clock duration that the sandbox runs, including idle time. Once the sandbox stops, only snapshot storage is billed. For platform evaluation, the lesson is to identify every meter that remains active during the idle interval. Wall-clock, active-CPU, and memory-based billing can produce different costs for the same pattern of agent activity without any difference in the work completed.

How durable volumes and shared agent storage extend persistence beyond the sandbox boundary

Restart persistence is narrower than deletion persistence. A snapshot can preserve a sandbox across stops and restores while remaining coupled to that sandbox’s identity. Deleting or replacing the sandbox may leave its snapshot lost or inaccessible to a successor. Durable block volumes instead attach at mount time and persist independently, so deleting the compute instance leaves the volume available to a replacement. PingCAP describes this stronger separation as allowing a second runtime to reopen the same workspace.

Shared storage lets multiple runtimes use that independent storage at the same time. An agent that checkpoints disk state and fans out into exploration branches needs storage that its continuations can access concurrently when they share a workspace. Snapshot-restore alone does not provide that shared access. Without a shared layer, separate full filesystem copies multiply storage use and introduce divergence at merge time. Persistence across a restart, survival after deletion, and concurrent access are consequently distinct capabilities.

Isolation model and filesystem persistence are not independent choices

Persistence retains sensitive working state as well as useful artifacts. Keeping it safe therefore depends on the boundary that keeps another tenant out of those files. Container-based sandboxes share the host kernel, so a container escape can put the host filesystem and other tenants’ persisted data at risk. Snyk identified CVE-2024-21626, “Leaky Vessels,” as enabling container escapes that gain unauthorized access to the underlying host operating system.

MicroVMs use a separate kernel per workload. A guest-to-host escape must defeat hardware virtualization rather than only a namespace boundary, but isolation still depends on how the VM is assigned. Reusing a sandbox for a different user or task can expose files retained from its previous use, so the per-user model calls for one session per VM. Persisted state also sharpens risks from filesystem access outside the intended scope, arbitrary network egress, the host kernel’s syscall surface, cross-tenant leakage, and secret exposure through environment variables or the /proc filesystem. A durable workspace is useful only within its intended access boundary.

Per-user agent fleets: how the storage/compute separation makes per-tenant persistence economically viable

A per-user fleet makes the separation operational through a snapshot-and-restore scheduler. Each user gets a sandbox for the session’s full run, and its state is kept between interactions. When idle, the sandbox is evicted with its state preserved. The scheduler captures its state, releases its host resources, and restores it in under a second when the next request arrives. Copy-on-write storage limits the saved delta, while separate storage billing prevents every retained user session from requiring resident compute.

AWS Lambda’s Firecracker architecture also reduces idle compute cost by evicting idle environments under memory pressure through LRU warm pools, although warm microVMs may remain briefly available for reuse. Managing resources this way does not by itself give agents durable workspaces. The distinction between persistent AI agent infrastructure and serverless functions is not simply whether either uses microVMs, but whether retained state has an explicit lifecycle independent of invocation compute. A production per-user fleet also needs private filesystems, per-tenant MCP capability scoping, and per-tenant egress rules. Together, these tie workspace continuity, tool access, and outbound permissions to the tenant they belong to, independently of whichever worker is free.

Practical checklist for auditing whether your sandbox infrastructure has genuinely decoupled storage from compute

Write a file, restart the sandbox, and read it back. If the file is missing, filesystem state is tied to the execution environment in the path that was tested.

Next, test deletion separately from restart. Delete the sandbox, create a replacement, and try to reopen the workspace using its durable volume or persistent disk. Standby snapshots may pass the restart test but fail this one.

For the cost test, read the billing documentation for the paused state. Identify whether CPU, memory, wall-clock time, or only persisted storage remains billable. A pause that retains the active-execution meter has not delivered the intended economic separation.

For the latency test, measure from the wake request to first code execution. Do not use a cold-boot figure or a snapshot-loading measurement that ends before code runs in its place. For interactive agents, the diagnostic target is a wake within one second.

For tenant isolation, verify that one user’s sandbox cannot read another user’s persisted files. Review both the runtime boundary and volume access in the architecture documentation. Shared volumes accessible across users invalidate filesystem privacy even if their processes are separately contained.

To check what can be recovered, test installed packages and process state separately. Test background jobs, the REPL, and in-memory variables for survival or the need to relaunch and reload them. File recovery alone is not execution recovery.

Finally, check whether snapshot storage grows with the data that changed or with the total disk size. The architecture documentation should distinguish copy-on-write or content-addressable snapshots from full copies.

Sources

  1. Don't Let AI Agents YOLO Your Files: Information and Control in Agent-Native Filesystems
  2. Firecracker

More in Sandboxed Code Execution