Durability and Drift-Resolution Guarantees¶
This page writes down, in plain prose, what protonfs actually guarantees about
not losing your data and about how it decides what needs syncing. Every claim
below is grounded in a specific mechanism in the source, referenced by module —
where a guarantee has a boundary or a known limitation, it is stated explicitly
rather than glossed over. For the command-by-command behavior, see
Command Reference; for the frozen CLI contract (exit codes, option names),
see Stability Promise (M4.1).
Durability¶
Atomic index and config writes¶
The local sync manifest (.protonfs/index.json) and the resumable-refresh
frontier (.protonfs/refresh-state.json) are never edited in place. Both are
written via the same idiom (IndexStore.save in src/protonfs/index.py,
refreshstate.save_frontier in src/protonfs/refreshstate.py): the new
document is written to a temp file in the same directory (so the following
rename is a true same-filesystem atomic operation), flushed and fsync’d, then
swapped onto the real path with os.replace. A reader — or a crash at any
point during the write — only ever sees the complete old file or the complete
new one; a torn or truncated index is not a failure mode this design permits.
After the rename, IndexStore.save also fsync’s the directory, so the
rename itself survives a power cut (best effort: some FUSE and network mounts
refuse to fsync a directory). Since 3.1.0 the index is written as compact
JSON, about a fifth smaller than the indented form and faster to build.
On some network and FUSE mounts (glusterfs, observed) the temp file is
occasionally gone by the time of the rename. The content is still in memory, so
IndexStore.save writes a fresh temp file and retries, a few times with a
short backoff, before it raises (#170).
Config files (.protonfs/config.json, config.local.json, and the global
~/.config/protonfs/config.json) are written the same way by
src/protonfs/config.py.
Repo locking¶
Every command that read-modifies-writes the index (push, pull, rm,
restore, refresh, offload) takes an exclusive, non-blocking POSIX
flock on .protonfs/lock for its duration (src/protonfs/locking.py). A
second concurrent protonfs process on the same repo fails immediately with an
instructive message rather than blocking or silently interleaving writes with the
first. Because flock is tied to the open file description, a crashed process
releases its lock automatically — there is no stale lock file to clean up by
hand.
Boundary: this lock is advisory and POSIX-only (fcntl). On a platform
without fcntl (documented in the module as the case pending issue #9 — in
practice non-POSIX systems), it degrades to a no-op rather than blocking
progress; two processes on such a platform are not protected against each other.
Excluded files are never touched¶
A file the store’s .protonfs/ignore/include rules exclude (IgnoreMatcher
in src/protonfs/ignore.py) is outside protonfs’s scope in both directions: it is
never uploaded, downloaded, indexed by refresh, offloaded, or trashed, and
status/ls do not report it. Changing the rules so that files you already hold
become excluded removes nothing: the local and Drive copies stay, and so do their index
entries, so bringing the files back into scope resumes where they left off. Before
3.1.0, pull and refresh brought excluded files down whenever they were missing
locally, so a file excluded on purpose came back after every deletion.
Resumable, idempotent push and pull¶
push and pull (src/protonfs/commands/push.py,
src/protonfs/commands/pull.py) both group their work by parent directory and
record each group in the index once it is confirmed (an upload seen in a Drive
listing, a download found in its own staging directory), not just once at the end.
A download is checked there, before it replaces anything: a file proton-drive
reported failed, did not deliver, or (when pull --resolve has a Drive listing)
delivered with a plaintext size or sha1 other than the listing’s is never moved into
place, so the local file stays as it was, and it counts as failed
(under-delivered when it did not land or did not match). The index is saved
part-way through at most every CHECKPOINT_INTERVAL_S (10 s), and always at the
end. Combined with the atomic-write guarantee above, this makes an interrupted run
(Ctrl-C, a dropped connection, a killed process) crash-safe: the next invocation
sees the index as it stood at the last save and acts on what remains, repeating at
most those few seconds of work. Before 3.1.0 the index was saved after every
group; each save rewrites the whole file (1.6 s at 100,000 entries), so a run over
thousands of folders spent most of its time saving.
These progress saves are checkpoints (IndexStore.checkpoint): if one fails,
the failure is logged and the run carries on, because the in-memory index still
holds everything. The save at the end of the command persists it, and is the one
that fails the command if it cannot (#170).
offload is stricter, because deleting a local copy cannot be undone. It records
the verified files as metadata-only and saves the index before deleting them, in
batches at most every CHECKPOINT_INTERVAL_S; a failed save is retried once, and
a second failure stops offload with the batch still on disk. An interruption can
therefore never leave the index saying a deleted file is still held here, which
push --deletions trash would take for a local deletion of the only copy.
Re-running push/pull on files that are already synced is a no-op — they
classify as locally-indexed and are excluded from the transfer set — so
retrying a command after a partial failure is always safe to do.
Resumable refresh with throttle backoff¶
refresh walks the remote tree breadth-first (DriveClient.walk in
src/protonfs/drive.py). Each directory listing goes through
list_with_backoff, which retries a timeout or throttle-shaped error with
exponential backoff (bounded by PROTONFS_LIST_RETRIES /
PROTONFS_LIST_BACKOFF / PROTONFS_LIST_BACKOFF_CAP) before giving up on
that call, so a single wedged directory doesn’t blow up an otherwise-healthy
walk. Beyond that, the walk’s own frontier (the queue of directories not yet
listed) is persisted via the atomic writer described above
(src/protonfs/refreshstate.py), always together with the index holding the
metadata-only entries seeded so far (commands/refresh.py:_seed_directory), at
most every CHECKPOINT_INTERVAL_S and when the walk is interrupted. The
frontier is never saved ahead of the index, so a resumed walk never skips a
directory whose entries were not saved. If a run is interrupted partway — by
a throttle giving up, or by the process dying — the next refresh invocation
resumes from the saved frontier instead of restarting the whole tree from the
root (and re-triggering the same throttle).
Boundary: change/deletion detection (a file classified remote-changed or
remote-deleted) requires a complete pass over the remote listing, since a
deletion can only be inferred from a file’s absence from the whole tree. A
resumed (partial) pass still seeds newly-discovered files but does not run
change/deletion detection; that only happens once a full pass completes without
being interrupted.
Verify-push-against-remote (never trust “transferred”)¶
proton-drive can report a file as successfully transferred without the bytes
actually landing on Drive (an “under-delivery”). push never records a file as
synced on trust alone: after each upload batch it re-lists the destination
directory via DriveClient.remote_identities and only indexes a file once it
is confirmed present there and its plaintext claimedSize matches the
local byte size exactly (src/protonfs/commands/push.py). This deliberately
compares against Proton’s decrypted claimedSize/claimedDigests, never the
encrypted totalStorageSize (which runs larger due to encryption overhead and
would never match a plaintext local size). A file that fails this check is left
unindexed and reported as a distinct under-delivered failure (not a
conflict) — the fix is a plain retry on the next push, not a --resolve
strategy.
That retry has to get past the short copy the failed upload left on Drive:
without a strategy proton-drive rejects the name as taken, and the file is
not indexed, so the copy looks like any other file’s. Two checks recognise it
as this file’s own older copy, and then the local file is uploaded as a new
revision of it (#168):
the remote copy is a byte-prefix of the local file: its plaintext size is no larger, and its sha1 equals the sha1 of that many leading local bytes. This is what a file pushed while still being appended to leaves behind;
or the failed push recorded the remote revision it uploaded (the
underdeliveredkey inindex.json, beside the entries), and the node’s active revision is still that one, so nothing has been uploaded over it since. Push only records a node it wrote to, never one askip/keep-bothstrategy left alone.
A remote copy that passes neither check is still a conflict.
A file this machine pushed before and has since changed locally is uploaded as a
new revision of its existing Drive node (proton-drive’s merge
strategy), so Drive’s version history keeps the copy it supersedes. Without a
strategy proton-drive rejects any upload onto an existing name, so this is
what makes a re-push of a changed file possible at all. It is only chosen when
the remote copy is not provably different from what the index recorded (its
plaintext size and sha1 do not disagree with the entry); a remote copy that
changed too is a real conflict and is reported as one, never buried under a new
revision. A path this machine never held locally (a metadata-only entry) is
never treated as a revision of the remote file.
A remote copy whose listing carries no plaintext claimedSize proves only
that a file of that name exists, so it is never accepted as verification. The
upload still happens, but the file is reported as an unverified failure (push
exits 1) and is not indexed — a file already in the index keeps its previous
entry. status therefore never counts it as locally-indexed, and the next push
retries it; once the listing reports a size again, that retry verifies (or
adopts) it normally. The same identity is one offload refuses to delete on,
so push can no longer write an index entry that offload would decline to trust.
offload reuses the identical verify-against-remote idiom before it deletes
any local bytes (below). A batch proton-drive reports skips for is verified
file by file rather than trusted or discarded: a skip is only reported as an
aggregate count with no per-file attribution, so each file in that batch must
match the remote strictly (size, and sha1 where both sides have one) before it
is indexed.
A local move that push applies on Drive (filesystem move/rename, see
push) is held to the same rule. It counts as moved only when a listing
of the target folder taken afterwards shows a node at the new name, absent before,
with the indexed size (and sha1 where both sides have one). Otherwise the file is
uploaded as a new file, and the old path’s index entry is left alone, or pointed at
the node’s actual location when a move landed and the rename after it did not.
proton-drive also fails a call as a whole: it reads a local path holding
*, ?, [ or { as a glob pattern, and a pattern matching nothing (or a
file gone since the scan) stops the whole call before anything moves, while still
printing an all-zero summary. protonfs escapes those characters in every path it
passes, and treats a failed call that names no failed file as an error carrying
proton-drive’s own message, never as a call with nothing to do (3.1.1). A
file whose path holds a backslash, or a {a,b} brace list, cannot be escaped
for proton-drive at all; it is reported as a failure that asks you to rename
it, and the rest of the batch goes ahead.
On download proton-drive replaces each of < > : " | ? * and control
characters in a file’s name with _ (t 12:00.dat lands as t 12_00.dat).
pull expects that name and moves the file into place under its real one, and
two files that would land under the same name are fetched in separate calls, so
one can never overwrite the other (3.1.1).
Stricter checks reach entries already recorded¶
Each index entry was judged by the checks of the protonfs that wrote it, and later
releases made those checks stricter (real size verification in 1.11.3, revisions for
appended files in 1.12.3). Without more, an entry recorded under a weaker check would
stay trusted indefinitely, including one offload has since deleted locally, where
Drive holds the only copy (#169).
So the index records the check level its entries meet (check_level in
index.json; an index an earlier release wrote reads as 0). upgrade
re-verifies an index below the current level against Drive, one listing per remote
directory, and only then records the new level. The repairs it makes never discard a
copy: a local file whose Drive copy does not match is dropped from the index so the
next push uploads it, and a Drive-only copy that is shorter, different or gone is
reported and left untouched rather than rewritten to look consistent. A release that
makes a check stricter raises the level in the same change (see CONTRIBUTING.md).
verify --index runs the same comparison on demand, read-only unless --repair.
Moving files out of git keeps a copy at every step¶
protonfs import (src/protonfs/commands/gitmove.py) stops git tracking files
only after it has proved each one is on Drive. It pushes them, then reads every file
once, whatever defaults.low_io says, and requires that the bytes match the index
entry push wrote, the blob HEAD records for the file, and a fresh listing of its
Drive directory (size and sha1). A Drive copy with no sha1 (uploaded by another client)
is downloaded to a staging directory and compared byte for byte, since size alone cannot
rule out a same-size different version. If any file fails, git is not
changed at all; running the command again uploads only what is missing.
When every file verifies, it writes one literal, anchored line per file into a
marked block in the store’s .gitignore, runs git rm --cached, and stages the
.gitignore. It never commits, never rewrites history and never changes or deletes
an imported file, so the bytes are still on disk, in git history and on Drive afterwards.
A failing git command undoes every git change it made; if undoing fails too, the error
lists the commands that finish the job by hand.
Another clone that pulls the import commit loses the files from its work tree (git
removes paths a commit untracks). Its index still lists them, so they look deleted
locally; push never trashes a path listed in the import block, whatever
--deletions says, so those Drive copies stay, and a default pull brings the
files back.
protonfs export moves files the other way with the same rules. It downloads any
offloaded file, then reads every file once and requires that the bytes match the index
entry and that the index matches a fresh listing of the file’s Drive directory (a copy
without a sha1 is downloaded and compared, as for import), so git
is given the version Drive holds and never a stale copy that would replace a newer one
another clone pushed. If any file fails, git is not changed. Git’s side goes first: the
import line is removed from .gitignore and the files are staged, and only then does
the .protonfs/ignore export block that stops protonfs syncing them get written and
staged. An interruption therefore leaves each file tracked by git, synced by protonfs,
or both, never neither, and a failing git command undoes every change. The Drive
copies stay unless --trash-remote is given, which trashes them only after the git
step succeeded, so the file is then on disk and staged in git. Export never commits,
never rewrites history, and never changes or deletes a local file. A Ctrl-C reports the
state it left: during the git step every change is undone; after it, the files are
staged and no longer synced, and any Drive copies not yet trashed are named as possibly
still there.
Settle window: nothing still being written is deleted¶
A file can still be in use when no process holds it open, so offload (and
prune, which deletes only through it) never removes a file whose local
modification time is within --min-age (default one day) of the run
(src/protonfs/commands/offload.py, src/protonfs/retention.py). This check
comes before every other one. A file that was pushed while it was still inside
that window is re-verified against the live listing before deletion, even under
--no-verify, because the record written at push time described a file that
was still changing. prune additionally keeps the newest --keep files of
each directory whatever their age, and it can only narrow the set offload
considers, never widen it.
Git-LFS pointer-stub protection¶
A repo migrated off git-LFS can still contain un-smudged pointer stubs (tiny
text files starting with the line version
https://git-lfs.github.com/spec/v1, checked by is_pointer_stub in
src/protonfs/lfs.py) — for instance a clone that never ran git lfs pull.
Such a stub’s own content hash has nothing to do with the real tracked file, so
naively diffing it against the index or remote would either mass-false-conflict
or, worse, classify it as pushable and let a normal push overwrite the real
Drive object with a ~131-byte placeholder.
This is guarded in two independent places: classify (src/protonfs/diff.py)
short-circuits any local file that parses as a pointer stub to a dedicated
lfs-pointer state before any hash comparison runs, so it is never treated as
local-only or conflicting; and push (src/protonfs/commands/push.py)
independently re-checks every file it’s about to upload and refuses any that
parse as a pointer stub, regardless of how it got into the candidate set. The
second check exists specifically as defense-in-depth in case the first is ever
wrong.
SHA-512-pinned binary installs¶
protonfs install-drive downloads the official proton-drive binary and
verifies its SHA-512 digest against a pinned checksum (pinned_sha512 in
src/protonfs/install.py) before writing it to disk — a mismatch raises an
error and the binary is not installed. For a platform/version combination
without a built-in pin, PROTONFS_DRIVE_SHA512 supplies the checksum to verify
against explicitly; there is no path that installs a binary without checking it
against some expected digest.
Drift resolution¶
How status/ls classify sync state¶
Both commands are thin renderers over the same classifier,
classify() in src/protonfs/diff.py, which compares up to three views of
each path — the local scan, the local index, and (when available) a live remote
listing — into one of the SyncState values:
Without a remote view (plain status/ls, the default) nothing is checked
on Drive, so every state below is a statement about the local files and the index
— what protonfs last recorded — and never about the remote. The states marked
(remote view) are only produced by status --remote/ls --remote, which
walk Drive first.
locally-indexed— local matches the index (and, with a remote view, the remote has not diverged from it). Calledsyncedbefore 2.0, a name that read as “verified on Drive” when nothing on Drive had been checked.local-only— present locally, no index entry (never pushed).metadata-only— an index entry this device deliberately never materialized (e.g. afterrefreshoroffload); with a remote view, the remote still matches it.local-deleted(remote view) — a file this device held is gone locally, and the remote still lists it.conflict— local diverged from the index and there is no remote view to attribute a direction to.pushsends it as a new revision when the remote still holds the indexed copy and reports a real conflict otherwise;status --remotesays which before anything is sent.remote-only— absent locally, with the copy on Drive: listed on Drive and absent from the index (remote view), or held by this device and now gone locally, as the index records (no remote view).local-modified(remote view) — local content diverged from the index; the remote has not.remote-modified(remote view) — the remote diverged from the index; local has not.both-modified(remote view) — both sides diverged from the index (a resolvable divergence).remote-changed(remote view) — a metadata-only entry whose remote copy moved.remote-deleted(remote view) — an index entry with nothing at that path in a full remote walk.lfs-pointer— an un-smudged git-LFS pointer stub (see above); deliberately inert, never treated as actionable drift.
A push that could not verify a file on Drive (under-delivered or
unverified) never moves it to locally-indexed: it stays local-only or
conflict until a later push verifies it.
Remote divergence itself (_remote_diverged) prefers comparing Proton’s
plaintext claimedDigests.sha1 against the index’s stored sha1 when both
sides know it; when either is unknown (trust-on-first-use for a value protonfs
has never observed), it falls back to comparing plaintext claimedSize
against the index’s size — again, never the encrypted size, for the same
reason as the push-verification guarantee above.
status’s exit code (0 clean / 1 drift / 2 conflict, conflict
outranking drift) is a direct function of these states, not a separate
classification — see status_exit_code in src/protonfs/commands/status.py
and the full table in Stability Promise (M4.1).
pull --resolve: never silently overwrite a local edit¶
A bare pull (no --resolve) only ever brings down files that are absent
locally (metadata-only and remote-only; with --no-restore, not the
remote-only files this machine held and has since deleted or moved) — it
cannot overwrite a local file, and a path the scan skipped that still exists on disk
is never downloaded over
because it never considers a locally-present file as a pull candidate without a
remote view. A file that changed on both sides since the last sync
(both-modified) or that locally diverged with no provable remote view
(conflict) is left completely untouched and reported as a failure; pull
exits non-zero rather than guessing.
--resolve picks a policy applied only to genuinely resolvable divergence
(both-modified, i.e. a remote copy exists to reconcile against):
remote— download the remote copy, overwriting local (file_strategy= "replace"against Drive).local— keep the local file untouched; it stays queued for the next push.both— fetch the remote copy into a sibling file under a.remotesuffix (deliberately not indexed — it is scratch for a manual merge), and leave the original local file alone.
With any policy, a file that changed only on Drive (remote-modified: the local
copy still matches the index) is downloaded over the local copy, since no local edit
is at stake; the policy decides only both-modified files. A file this machine
held that is gone locally (local-deleted) is restored under any policy, as a bare
pull restores it, unless --no-restore is given.
Plain conflict entries (no remote view available to attribute direction) are
never resolved by --resolve — they are reported regardless, because
there is no proof of what “remote” even means for them without a live listing.
offload’s remote-verified guarantee before deleting local bytes¶
Before any local file is unlinked, offload (src/protonfs/commands/
offload.py) applies two checks, the first of which is unconditional (it
holds even under --no-verify):
The file’s live content hash must match what the index recorded at last sync. A file edited locally since it was last synced has content that is not on Drive; deleting it would destroy the only copy of that edit, and a same-size remote object would even pass a size-only verification — so this check compares actual content hash, not just size, and cannot be turned off.
By default (unless
--no-verify), the remote parent directory is freshly re-listed and the file must be present there with a plaintextclaimedSizematching the local byte size — the same idiom as push’s post-upload verification, applied here before a delete instead of after an upload.
A file failing either check is left on disk and reported
(skipped_modified/skipped_unverified) rather than silently kept or
force-deleted; this is not treated as command failure. --no-verify only
disables check 2 (trusting the index’s record of what was last confirmed
on Drive); it never disables check 1.
Trash-not-delete semantics¶
rm (src/protonfs/commands/rm.py) trashes the remote node by default —
this is always reversible via restore. rm -f/--force additionally
attempts a permanent delete of the now-trashed node, but only as a best-effort
second step after the (always-succeeding) trash.
Boundary — duplicate basenames: proton-drive’s filesystem delete
addresses a trashed node only by its path under /trash (/trash/<basename>);
there is no working way to target a specific trashed node by its stable UID (as
of proton-drive 2026-07-16, both /trash/<uid> and a bare UID are rejected,
despite --help advertising UID addressing elsewhere — a live probe test in
the test suite flags it automatically if a future proton-drive lifts this).
Consequence: when two or more trashed items share a basename, rm -f cannot
safely tell which one you meant to permanently delete, so it refuses the
permanent-delete step and leaves the item trashed (still reversible), reporting
the ambiguity so you can resolve it via the Proton Drive app/web UI instead.
Nothing is ever lost by this refusal — the item stays exactly where trashing put
it.
Boundary — restore ambiguity: restore (DriveClient.restore in
src/protonfs/drive.py) tries original-path restore first (accepted by
proton-drive 0.4.6). On proton-drive 0.5.0+, which removed that form, it falls
back to resolving the trashed entry by decrypted name under /trash. Because
0.5.0 restores by name with first-match-wins and offers no UID disambiguation
under /trash either, this fallback explicitly checks that the resolved
same-named trash entry’s original parent matches the path being restored; if
more than one trashed item shares that basename and the first match did not come
from the expected parent, restore refuses rather than risk restoring the
wrong node, raising an error that names the ambiguity and points at the Drive web
UI (or permanently deleting the older same-named entries first) as the way out.
This is an upstream proton-drive limitation (tracked as issue #56 in the
codebase), not a choice protonfs makes — see the full detail in
DriveClient.restore’s docstring.
See also¶
Command Reference for the full command-by-command reference.
Stability Promise (M4.1) for the frozen exit-code/option contract these guarantees sit underneath.
Syncing across machines for the day-to-day workflow these mechanisms support.