Durability and Drift-Resolution Guarantees

This page writes down, in plain prose, what protonfs actually guarantees about not losing your data and about how it decides what needs syncing. Every claim below is grounded in a specific mechanism in the source, referenced by module — where a guarantee has a boundary or a known limitation, it is stated explicitly rather than glossed over. For the command-by-command behavior, see Command Reference; for the frozen CLI contract (exit codes, option names), see Stability Promise (M4.1).

Durability

Atomic index and config writes

The local sync manifest (.protonfs/index.json) and the resumable-refresh frontier (.protonfs/refresh-state.json) are never edited in place. Both are written via the same idiom (IndexStore.save in src/protonfs/index.py, refreshstate.save_frontier in src/protonfs/refreshstate.py): the new document is written to a temp file in the same directory (so the following rename is a true same-filesystem atomic operation), flushed and fsync’d, then swapped onto the real path with os.replace. A reader — or a crash at any point during the write — only ever sees the complete old file or the complete new one; a torn or truncated index is not a failure mode this design permits. After the rename, IndexStore.save also fsync’s the directory, so the rename itself survives a power cut (best effort: some FUSE and network mounts refuse to fsync a directory). Since 3.1.0 the index is written as compact JSON, about a fifth smaller than the indented form and faster to build.

On some network and FUSE mounts (glusterfs, observed) the temp file is occasionally gone by the time of the rename. The content is still in memory, so IndexStore.save writes a fresh temp file and retries, a few times with a short backoff, before it raises (#170).

Config files (.protonfs/config.json, config.local.json, and the global ~/.config/protonfs/config.json) are written the same way by src/protonfs/config.py.

Repo locking

Every command that read-modifies-writes the index (push, pull, rm, restore, refresh, offload) takes an exclusive, non-blocking POSIX flock on .protonfs/lock for its duration (src/protonfs/locking.py). A second concurrent protonfs process on the same repo fails immediately with an instructive message rather than blocking or silently interleaving writes with the first. Because flock is tied to the open file description, a crashed process releases its lock automatically — there is no stale lock file to clean up by hand.

Boundary: this lock is advisory and POSIX-only (fcntl). On a platform without fcntl (documented in the module as the case pending issue #9 — in practice non-POSIX systems), it degrades to a no-op rather than blocking progress; two processes on such a platform are not protected against each other.

Excluded files are never touched

A file the store’s .protonfs/ignore/include rules exclude (IgnoreMatcher in src/protonfs/ignore.py) is outside protonfs’s scope in both directions: it is never uploaded, downloaded, indexed by refresh, offloaded, or trashed, and status/ls do not report it. Changing the rules so that files you already hold become excluded removes nothing: the local and Drive copies stay, and so do their index entries, so bringing the files back into scope resumes where they left off. Before 3.1.0, pull and refresh brought excluded files down whenever they were missing locally, so a file excluded on purpose came back after every deletion.

Resumable, idempotent push and pull

push and pull (src/protonfs/commands/push.py, src/protonfs/commands/pull.py) both group their work by parent directory and record each group in the index once it is confirmed (an upload seen in a Drive listing, a download found in its own staging directory), not just once at the end. A download is checked there, before it replaces anything: a file proton-drive reported failed, did not deliver, or (when pull --resolve has a Drive listing) delivered with a plaintext size or sha1 other than the listing’s is never moved into place, so the local file stays as it was, and it counts as failed (under-delivered when it did not land or did not match). The index is saved part-way through at most every CHECKPOINT_INTERVAL_S (10 s), and always at the end. Combined with the atomic-write guarantee above, this makes an interrupted run (Ctrl-C, a dropped connection, a killed process) crash-safe: the next invocation sees the index as it stood at the last save and acts on what remains, repeating at most those few seconds of work. Before 3.1.0 the index was saved after every group; each save rewrites the whole file (1.6 s at 100,000 entries), so a run over thousands of folders spent most of its time saving.

These progress saves are checkpoints (IndexStore.checkpoint): if one fails, the failure is logged and the run carries on, because the in-memory index still holds everything. The save at the end of the command persists it, and is the one that fails the command if it cannot (#170).

offload is stricter, because deleting a local copy cannot be undone. It records the verified files as metadata-only and saves the index before deleting them, in batches at most every CHECKPOINT_INTERVAL_S; a failed save is retried once, and a second failure stops offload with the batch still on disk. An interruption can therefore never leave the index saying a deleted file is still held here, which push --deletions trash would take for a local deletion of the only copy.

Re-running push/pull on files that are already synced is a no-op — they classify as locally-indexed and are excluded from the transfer set — so retrying a command after a partial failure is always safe to do.

Resumable refresh with throttle backoff

refresh walks the remote tree breadth-first (DriveClient.walk in src/protonfs/drive.py). Each directory listing goes through list_with_backoff, which retries a timeout or throttle-shaped error with exponential backoff (bounded by PROTONFS_LIST_RETRIES / PROTONFS_LIST_BACKOFF / PROTONFS_LIST_BACKOFF_CAP) before giving up on that call, so a single wedged directory doesn’t blow up an otherwise-healthy walk. Beyond that, the walk’s own frontier (the queue of directories not yet listed) is persisted via the atomic writer described above (src/protonfs/refreshstate.py), always together with the index holding the metadata-only entries seeded so far (commands/refresh.py:_seed_directory), at most every CHECKPOINT_INTERVAL_S and when the walk is interrupted. The frontier is never saved ahead of the index, so a resumed walk never skips a directory whose entries were not saved. If a run is interrupted partway — by a throttle giving up, or by the process dying — the next refresh invocation resumes from the saved frontier instead of restarting the whole tree from the root (and re-triggering the same throttle).

Boundary: change/deletion detection (a file classified remote-changed or remote-deleted) requires a complete pass over the remote listing, since a deletion can only be inferred from a file’s absence from the whole tree. A resumed (partial) pass still seeds newly-discovered files but does not run change/deletion detection; that only happens once a full pass completes without being interrupted.

Verify-push-against-remote (never trust “transferred”)

proton-drive can report a file as successfully transferred without the bytes actually landing on Drive (an “under-delivery”). push never records a file as synced on trust alone: after each upload batch it re-lists the destination directory via DriveClient.remote_identities and only indexes a file once it is confirmed present there and its plaintext claimedSize matches the local byte size exactly (src/protonfs/commands/push.py). This deliberately compares against Proton’s decrypted claimedSize/claimedDigests, never the encrypted totalStorageSize (which runs larger due to encryption overhead and would never match a plaintext local size). A file that fails this check is left unindexed and reported as a distinct under-delivered failure (not a conflict) — the fix is a plain retry on the next push, not a --resolve strategy.

That retry has to get past the short copy the failed upload left on Drive: without a strategy proton-drive rejects the name as taken, and the file is not indexed, so the copy looks like any other file’s. Two checks recognise it as this file’s own older copy, and then the local file is uploaded as a new revision of it (#168):

  • the remote copy is a byte-prefix of the local file: its plaintext size is no larger, and its sha1 equals the sha1 of that many leading local bytes. This is what a file pushed while still being appended to leaves behind;

  • or the failed push recorded the remote revision it uploaded (the underdelivered key in index.json, beside the entries), and the node’s active revision is still that one, so nothing has been uploaded over it since. Push only records a node it wrote to, never one a skip/keep-both strategy left alone.

A remote copy that passes neither check is still a conflict.

A file this machine pushed before and has since changed locally is uploaded as a new revision of its existing Drive node (proton-drive’s merge strategy), so Drive’s version history keeps the copy it supersedes. Without a strategy proton-drive rejects any upload onto an existing name, so this is what makes a re-push of a changed file possible at all. It is only chosen when the remote copy is not provably different from what the index recorded (its plaintext size and sha1 do not disagree with the entry); a remote copy that changed too is a real conflict and is reported as one, never buried under a new revision. A path this machine never held locally (a metadata-only entry) is never treated as a revision of the remote file.

A remote copy whose listing carries no plaintext claimedSize proves only that a file of that name exists, so it is never accepted as verification. The upload still happens, but the file is reported as an unverified failure (push exits 1) and is not indexed — a file already in the index keeps its previous entry. status therefore never counts it as locally-indexed, and the next push retries it; once the listing reports a size again, that retry verifies (or adopts) it normally. The same identity is one offload refuses to delete on, so push can no longer write an index entry that offload would decline to trust.

offload reuses the identical verify-against-remote idiom before it deletes any local bytes (below). A batch proton-drive reports skips for is verified file by file rather than trusted or discarded: a skip is only reported as an aggregate count with no per-file attribution, so each file in that batch must match the remote strictly (size, and sha1 where both sides have one) before it is indexed.

A local move that push applies on Drive (filesystem move/rename, see push) is held to the same rule. It counts as moved only when a listing of the target folder taken afterwards shows a node at the new name, absent before, with the indexed size (and sha1 where both sides have one). Otherwise the file is uploaded as a new file, and the old path’s index entry is left alone, or pointed at the node’s actual location when a move landed and the rename after it did not.

proton-drive also fails a call as a whole: it reads a local path holding *, ?, [ or { as a glob pattern, and a pattern matching nothing (or a file gone since the scan) stops the whole call before anything moves, while still printing an all-zero summary. protonfs escapes those characters in every path it passes, and treats a failed call that names no failed file as an error carrying proton-drive’s own message, never as a call with nothing to do (3.1.1). A file whose path holds a backslash, or a {a,b} brace list, cannot be escaped for proton-drive at all; it is reported as a failure that asks you to rename it, and the rest of the batch goes ahead.

On download proton-drive replaces each of < > : " | ? * and control characters in a file’s name with _ (t 12:00.dat lands as t 12_00.dat). pull expects that name and moves the file into place under its real one, and two files that would land under the same name are fetched in separate calls, so one can never overwrite the other (3.1.1).

Stricter checks reach entries already recorded

Each index entry was judged by the checks of the protonfs that wrote it, and later releases made those checks stricter (real size verification in 1.11.3, revisions for appended files in 1.12.3). Without more, an entry recorded under a weaker check would stay trusted indefinitely, including one offload has since deleted locally, where Drive holds the only copy (#169).

So the index records the check level its entries meet (check_level in index.json; an index an earlier release wrote reads as 0). upgrade re-verifies an index below the current level against Drive, one listing per remote directory, and only then records the new level. The repairs it makes never discard a copy: a local file whose Drive copy does not match is dropped from the index so the next push uploads it, and a Drive-only copy that is shorter, different or gone is reported and left untouched rather than rewritten to look consistent. A release that makes a check stricter raises the level in the same change (see CONTRIBUTING.md). verify --index runs the same comparison on demand, read-only unless --repair.

Remote manifest: a cache, never an authority

A repo can keep a manifest of what protonfs verified on Drive at <remote_root>/.protonfs/manifest.json (see verify), so a host can learn what is on the remote from one small download instead of a walk. It is held to these rules (src/protonfs/manifest.py):

  • It lags, never leads. An entry is written only after the upload it describes was verified against a live listing (the same check as above), and an unverified upload is never recorded. The manifest is only ever created from a full listing (verify --repair); push/rm only update an existing one, so it never looks complete while missing history. A crash between an upload and the manifest write leaves it behind, which verify reports and verify --repair fixes.

  • It never authorises a delete. offload keeps its own live verification and never reads the manifest. Nothing under the remote or local .protonfs/ directory is ever treated as data: not seeded by refresh, not classified, not offloaded.

  • Its readers only read. pull on an empty index seeds from it, says that files it does not list are not included, and verifies every download itself, so a stale entry costs a reported failure, never a silent one.

  • Concurrent writers. Each write checks the manifest’s Drive revision first and replays its changes onto a newer copy another host wrote, then confirms its own revision is the active one afterwards. The proton-drive CLI has no conditional write, so a narrow race can still drop another host’s update from the newest revision. That leaves the manifest behind, which is the direction it is allowed to be wrong in, and Drive’s version history keeps every revision written.

  • Maintenance is on by default (defaults.manifest) and a manifest failure never fails the push/rm that triggered it: it is reported as a warning, and the command’s own result stands.

Moving files out of git keeps a copy at every step

protonfs import (src/protonfs/commands/gitmove.py) stops git tracking files only after it has proved each one is on Drive. It pushes them, then reads every file once, whatever defaults.low_io says, and requires that the bytes match the index entry push wrote, the blob HEAD records for the file, and a fresh listing of its Drive directory (size and sha1). A Drive copy with no sha1 (uploaded by another client) is downloaded to a staging directory and compared byte for byte, since size alone cannot rule out a same-size different version. If any file fails, git is not changed at all; running the command again uploads only what is missing.

When every file verifies, it writes one literal, anchored line per file into a marked block in the store’s .gitignore, runs git rm --cached, and stages the .gitignore. It never commits, never rewrites history and never changes or deletes an imported file, so the bytes are still on disk, in git history and on Drive afterwards. A failing git command undoes every git change it made; if undoing fails too, the error lists the commands that finish the job by hand.

Another clone that pulls the import commit loses the files from its work tree (git removes paths a commit untracks). Its index still lists them, so they look deleted locally; push never trashes a path listed in the import block, whatever --deletions says, so those Drive copies stay, and a default pull brings the files back.

protonfs export moves files the other way with the same rules. It downloads any offloaded file, then reads every file once and requires that the bytes match the index entry and that the index matches a fresh listing of the file’s Drive directory (a copy without a sha1 is downloaded and compared, as for import), so git is given the version Drive holds and never a stale copy that would replace a newer one another clone pushed. If any file fails, git is not changed. Git’s side goes first: the import line is removed from .gitignore and the files are staged, and only then does the .protonfs/ignore export block that stops protonfs syncing them get written and staged. An interruption therefore leaves each file tracked by git, synced by protonfs, or both, never neither, and a failing git command undoes every change. The Drive copies stay unless --trash-remote is given, which trashes them only after the git step succeeded, so the file is then on disk and staged in git. Export never commits, never rewrites history, and never changes or deletes a local file. A Ctrl-C reports the state it left: during the git step every change is undone; after it, the files are staged and no longer synced, and any Drive copies not yet trashed are named as possibly still there.

Settle window: nothing still being written is deleted

A file can still be in use when no process holds it open, so offload (and prune, which deletes only through it) never removes a file whose local modification time is within --min-age (default one day) of the run (src/protonfs/commands/offload.py, src/protonfs/retention.py). This check comes before every other one. A file that was pushed while it was still inside that window is re-verified against the live listing before deletion, even under --no-verify, because the record written at push time described a file that was still changing. prune additionally keeps the newest --keep files of each directory whatever their age, and it can only narrow the set offload considers, never widen it.

Git-LFS pointer-stub protection

A repo migrated off git-LFS can still contain un-smudged pointer stubs (tiny text files starting with the line version https://git-lfs.github.com/spec/v1, checked by is_pointer_stub in src/protonfs/lfs.py) — for instance a clone that never ran git lfs pull. Such a stub’s own content hash has nothing to do with the real tracked file, so naively diffing it against the index or remote would either mass-false-conflict or, worse, classify it as pushable and let a normal push overwrite the real Drive object with a ~131-byte placeholder.

This is guarded in two independent places: classify (src/protonfs/diff.py) short-circuits any local file that parses as a pointer stub to a dedicated lfs-pointer state before any hash comparison runs, so it is never treated as local-only or conflicting; and push (src/protonfs/commands/push.py) independently re-checks every file it’s about to upload and refuses any that parse as a pointer stub, regardless of how it got into the candidate set. The second check exists specifically as defense-in-depth in case the first is ever wrong.

SHA-512-pinned binary installs

protonfs install-drive downloads the official proton-drive binary and verifies its SHA-512 digest against a pinned checksum (pinned_sha512 in src/protonfs/install.py) before writing it to disk — a mismatch raises an error and the binary is not installed. For a platform/version combination without a built-in pin, PROTONFS_DRIVE_SHA512 supplies the checksum to verify against explicitly; there is no path that installs a binary without checking it against some expected digest.

Drift resolution

How status/ls classify sync state

Both commands are thin renderers over the same classifier, classify() in src/protonfs/diff.py, which compares up to three views of each path — the local scan, the local index, and (when available) a live remote listing — into one of the SyncState values:

Without a remote view (plain status/ls, the default) nothing is checked on Drive, so every state below is a statement about the local files and the index — what protonfs last recorded — and never about the remote. The states marked (remote view) are only produced by status --remote/ls --remote, which walk Drive first.

  • locally-indexed — local matches the index (and, with a remote view, the remote has not diverged from it). Called synced before 2.0, a name that read as “verified on Drive” when nothing on Drive had been checked.

  • local-only — present locally, no index entry (never pushed).

  • metadata-only — an index entry this device deliberately never materialized (e.g. after refresh or offload); with a remote view, the remote still matches it.

  • local-deleted (remote view) — a file this device held is gone locally, and the remote still lists it.

  • conflict — local diverged from the index and there is no remote view to attribute a direction to. push sends it as a new revision when the remote still holds the indexed copy and reports a real conflict otherwise; status --remote says which before anything is sent.

  • remote-only — absent locally, with the copy on Drive: listed on Drive and absent from the index (remote view), or held by this device and now gone locally, as the index records (no remote view).

  • local-modified (remote view) — local content diverged from the index; the remote has not.

  • remote-modified (remote view) — the remote diverged from the index; local has not.

  • both-modified (remote view) — both sides diverged from the index (a resolvable divergence).

  • remote-changed (remote view) — a metadata-only entry whose remote copy moved.

  • remote-deleted (remote view) — an index entry with nothing at that path in a full remote walk.

  • lfs-pointer — an un-smudged git-LFS pointer stub (see above); deliberately inert, never treated as actionable drift.

A push that could not verify a file on Drive (under-delivered or unverified) never moves it to locally-indexed: it stays local-only or conflict until a later push verifies it.

Remote divergence itself (_remote_diverged) prefers comparing Proton’s plaintext claimedDigests.sha1 against the index’s stored sha1 when both sides know it; when either is unknown (trust-on-first-use for a value protonfs has never observed), it falls back to comparing plaintext claimedSize against the index’s size — again, never the encrypted size, for the same reason as the push-verification guarantee above.

status’s exit code (0 clean / 1 drift / 2 conflict, conflict outranking drift) is a direct function of these states, not a separate classification — see status_exit_code in src/protonfs/commands/status.py and the full table in Stability Promise (M4.1).

pull --resolve: never silently overwrite a local edit

A bare pull (no --resolve) only ever brings down files that are absent locally (metadata-only and remote-only; with --no-restore, not the remote-only files this machine held and has since deleted or moved) — it cannot overwrite a local file, and a path the scan skipped that still exists on disk is never downloaded over because it never considers a locally-present file as a pull candidate without a remote view. A file that changed on both sides since the last sync (both-modified) or that locally diverged with no provable remote view (conflict) is left completely untouched and reported as a failure; pull exits non-zero rather than guessing.

--resolve picks a policy applied only to genuinely resolvable divergence (both-modified, i.e. a remote copy exists to reconcile against):

  • remote — download the remote copy, overwriting local (file_strategy= "replace" against Drive).

  • local — keep the local file untouched; it stays queued for the next push.

  • both — fetch the remote copy into a sibling file under a .remote suffix (deliberately not indexed — it is scratch for a manual merge), and leave the original local file alone.

With any policy, a file that changed only on Drive (remote-modified: the local copy still matches the index) is downloaded over the local copy, since no local edit is at stake; the policy decides only both-modified files. A file this machine held that is gone locally (local-deleted) is restored under any policy, as a bare pull restores it, unless --no-restore is given.

Plain conflict entries (no remote view available to attribute direction) are never resolved by --resolve — they are reported regardless, because there is no proof of what “remote” even means for them without a live listing.

offload’s remote-verified guarantee before deleting local bytes

Before any local file is unlinked, offload (src/protonfs/commands/ offload.py) applies two checks, the first of which is unconditional (it holds even under --no-verify):

  1. The file’s live content hash must match what the index recorded at last sync. A file edited locally since it was last synced has content that is not on Drive; deleting it would destroy the only copy of that edit, and a same-size remote object would even pass a size-only verification — so this check compares actual content hash, not just size, and cannot be turned off.

  2. By default (unless --no-verify), the remote parent directory is freshly re-listed and the file must be present there with a plaintext claimedSize matching the local byte size — the same idiom as push’s post-upload verification, applied here before a delete instead of after an upload.

A file failing either check is left on disk and reported (skipped_modified/skipped_unverified) rather than silently kept or force-deleted; this is not treated as command failure. --no-verify only disables check 2 (trusting the index’s record of what was last confirmed on Drive); it never disables check 1.

Trash-not-delete semantics

rm (src/protonfs/commands/rm.py) trashes the remote node by default — this is always reversible via restore. rm -f/--force additionally attempts a permanent delete of the now-trashed node, but only as a best-effort second step after the (always-succeeding) trash.

Boundary — duplicate basenames: proton-drive’s filesystem delete addresses a trashed node only by its path under /trash (/trash/<basename>); there is no working way to target a specific trashed node by its stable UID (as of proton-drive 2026-07-16, both /trash/<uid> and a bare UID are rejected, despite --help advertising UID addressing elsewhere — a live probe test in the test suite flags it automatically if a future proton-drive lifts this). Consequence: when two or more trashed items share a basename, rm -f cannot safely tell which one you meant to permanently delete, so it refuses the permanent-delete step and leaves the item trashed (still reversible), reporting the ambiguity so you can resolve it via the Proton Drive app/web UI instead. Nothing is ever lost by this refusal — the item stays exactly where trashing put it.

Boundary — restore ambiguity: restore (DriveClient.restore in src/protonfs/drive.py) tries original-path restore first (accepted by proton-drive 0.4.6). On proton-drive 0.5.0+, which removed that form, it falls back to resolving the trashed entry by decrypted name under /trash. Because 0.5.0 restores by name with first-match-wins and offers no UID disambiguation under /trash either, this fallback explicitly checks that the resolved same-named trash entry’s original parent matches the path being restored; if more than one trashed item shares that basename and the first match did not come from the expected parent, restore refuses rather than risk restoring the wrong node, raising an error that names the ambiguity and points at the Drive web UI (or permanently deleting the older same-named entries first) as the way out. This is an upstream proton-drive limitation (tracked as issue #56 in the codebase), not a choice protonfs makes — see the full detail in DriveClient.restore’s docstring.

See also