Nocturne application deployed on the PC

The owner authorized deployment of the latest merged Nocturne code. Application revision 33f4c25f from PR #548 is now running on CaptainKangapoo with its existing read-only NFS setup. Incremental NAS discovery remains disabled.

Initial deployment verification

• Built the merged application and frontend, then added the existing nfs-common runtime dependency. Installed image: nocturne-gallery:6d167587-pc, image ID sha256:02a90a5ad3b31024cd61cf13968ae39eb1f8f7ea6aeede8d191baa6629f3337e.
• Ran all backend tests against the installed image in isolation: GALLERY_TEST_BACKEND=/app python -m pytest /tests -q -p no:cacheprovider: 150 passed. Tests and tools were mounted read-only; production state was not mounted. The first harness run omitted /tools; its two missing-file failures were resolved by mounting the required tooling and rerunning the full suite.
• Used SQLite backup to create a consistent private catalog snapshot. Migration on a separate copy passed integrity checking and preserved the annotation checksum, media IDs/paths, tags, albums, and availability counts. A final backup while the sole writer was stopped also passed integrity checking.
• Retained 380,410 available and 33,656 unavailable records at cutover. SQLite remains in DELETE journaling mode. Existing login/session and authenticated browse/status/failure APIs pass; the new polling diagnostics are present.
• Container health passes. Its /app/library.py checksum matches the merged source, with no scanner-code bind mount. The NFS mount is read-only; the public HTTPS route still returns the expected Authentik redirect without a session.
• Ten workers are active. Completed preview jobs increased after replacement; the scheduler reports no error. Effective settings are 300-second legacy polling and a 60-GiB cache budget, retaining the twelve-CPU/six-GiB limits.
• Persisted the exact source and launch configuration on local WSL storage and tested the launcher without recreating the already healthy container.

flowchart LR
    Source[Reviewed merged source] --> Image[Tested PC image]
    Catalog[Stopped sole writer] --> Backup[Verified private backup]
    Image --> PC[PC gallery writer]
    Backup --> PC
    PC --> ReadOnly[Read-only NFS media]
    PC --> Local[Local catalog and cache]
    Browser --> Authentik --> PC

Recovery and remaining boundaries

The previous image, container configuration, launcher, and catalog backups remain on local disk. See the current deployment and recovery locations. Rollback must handle the new eviction markers before using an older queue. No backup artifacts were deleted or moved.

At the initial deployment checkpoint, the first full scan was still in progress; last-success diagnostics remained unset until a healthy scan could finish. Some originals report permission errors; 1,771 failed jobs already existed before cutover. No source permission, scraper, export, firewall, or authentication-policy changes were made. Successful browsing and job progress do not establish that every original is readable. NAS shadow-trial and feed activation gates remain open.

Hash scheduling follow-up

The owner approved the scheduling repair, its measured rollout, and NAS shadow preparation. PR #546 merged as 82528c7a after all 27 current-head checks passed. The first repository-wide Python run failed in an unchanged Stirling test that compared access timestamps; the rerun passed. The exact candidate image nocturne-gallery:4af34f93-pc passed all 156 backend regressions before deployment. A new stopped-writer backup passed integrity and annotation checks.

Duplicate hashes no longer consume derivative queue capacity. Hashing is limited to one claimed job and yields during scans/feed recovery. Thumbnails now queue and complete despite the duplicate backlog. No schema migration was introduced. The ten-worker setting remains; CPU alone does not justify increasing NAS reads.

Recorded short live samples showed about 13 versus 19 scanned items/second, with post-change CPU around 3.2 cores and I/O stalls around 25 percent instead of roughly 73 percent. These are observational windows at different scan positions/cache states, not a controlled speedup or acceptance result.

Worker comparison and retained setting

PR #548 merged as 33f4c25f after all 27 current-head checks passed. Its tested head, 5953ea3b, is deployed as nocturne-gallery:5953ea3b-pc. The exact image passed all 160 backend tests before activation. A stopped-writer backup passed integrity and annotation checks. The code ceiling is now sixteen workers; the default remains four and the deployed setting remains ten.

The owner-approved ten-to-twelve-to-ten comparison completed three approximately 94-second windows after warm-up. At that checkpoint there were 381,758 available catalog records. Aggregate measurements record the following:

Phase	Scan items/second	Derivatives/second	CPU cores	NFS MiB/second	API probe seconds
Ten before	38.16	10.82	1.95	10.33	0.75
Twelve	34.15	13.82	3.34	15.11	0.39
Ten after	50.70	13.49	2.15	14.93	0.54

The ten-worker baselines were unstable and twelve failed the conservative scan gate. Ten was retained. Different scan positions, warming caches and uncontrolled NAS/scraper load prevent a causal speedup or regression claim. Scraper throughput was not measured credibly; the five-percent impact gate remains unverified. The provisional twofold reconciliation target has not been established.

NAS shadow preparation and diagnosed failure

The original tested image loaded on the NAS. Its kernel rejected CFS quotas; the explicit CPU-affinity variant subsequently verified one logical CPU and 512 MiB without changing host CPU assignments or removing resource limits. SSH forwarding was prohibited by existing policy, so a fixed-endpoint SSH command transport was tested for the separate shadow consumer. SSH policy is unchanged.

The helper then repeatedly reset its epoch on Watchman permission warnings for excluded host staging paths. This was an adapter classification failure, not evidence of notification overflow. The helper was stopped with state retained; no healthy initial inventory or twenty-four-hour observation completed.

PR #547, commit 36692777, recognizes only the pinned permission-warning format for demonstrably excluded scopes. Eligible, untrusted, mixed, recrawl and overflow warnings still force recovery. All 183 integrated backend tests passed. The corrected exact image passed an actual-Watchman fixture with 300,000 files and an unreadable excluded directory, including read-only enforcement, settling, rename, deletion and restart recovery, under one CPU and 512 MiB. Quiet-folder discovery was 4.45 seconds in this local fixture; that is not NAS acceptance evidence.

The owner started the corrected image and its resource probe verified one logical CPU and 512 MiB. Excluded warnings remained in one epoch while the helper published up to 166,800 changes. These are event counts, including settling transitions, rather than distinct media totals. No healthy inventory completed during about one hour, so the shadow consumer never started its twenty-four-hour observation.

Memory-limit hits rose from 73,467 to 470,962. Swap rose from about seventy to 183 MiB, with 39,610 major faults recorded at the final sample. The guardian confirmed the helper stopped on request; the private observer also stopped and all state was retained. Counters include the then-in-container health probes. Component attribution and OOM history were not established. See the sanitized resource evidence.

A separate opt-in candidate in PR #547 keeps one logical CPU, allows one GiB of RAM, and disables swap. Its exact-image local fixture passed with 400,000 eligible and 100,000 excluded files, including settling, read-only enforcement, rename, deletion and restart recovery. Initial inventory took 433 seconds; quiet-folder discovery took about five seconds. These are fixture results, not NAS acceptance. The installer's resource probe passed and rejected a swap-enabled container. Default, affinity and larger-trial Compose renderings passed; the actual NAS Compose also rendered the larger trial correctly. The NAS reported about fourteen GiB available at staging. See the larger fixture.

The owner approved and started the one-GiB candidate. The NAS verified CPU affinity [7], RAM limit 1,073,741,824 bytes and the same combined memory/swap limit. However, the newly added zero-byte swap guard stopped it after a single 4-KiB swapped page. At that sample, memory use was 420,511,744 bytes and limit hits were zero. This did not establish another capacity failure. No healthy inventory or twenty-four-hour observation completed.

The guard assumed the combined cgroup limit guaranteed absolute zero swap. That assumption was incorrect for this kernel. The Linux 4.4 memory-controller documentation permits host-wide reclaim to swap pages within the combined cap. The guard-stop evidence records the distinction. The installer now reports a hard swap ban only on cgroup v2, and accurately reports the combined cap on the NAS's cgroup v1.

The corrected candidate is staged at commit 4c01006c with verified source and launcher checksums. It reuses the existing pinned image. Its guard permits incidental global paging and stops after three consecutive samples with at least 64 MiB swapped, 4,096 limit hits and 128 new major faults per sample. Tests covered the observed single page, global-only paging, inactive swapped pages, and sustained pressure, including persistent sample state. OOM/restart, repeated epoch, one-hour initial-recovery and twenty-six-hour lifetime guards remain.

The owner explicitly accepted this old-kernel policy and started the corrected retry on 2026-10-30. The total cap remains one GiB; absolute zero swap is not guaranteed. The NAS verified affinity [7], RAM and combined RAM/swap limits of 1,073,741,824 bytes, and the expected absence of a hard swap ban on cgroup v1. The helper and isolated Windows controller are running initial recovery with retained state. Early samples showed no limit hits, continued metadata publication and 16 KiB of incidental paging with no new major faults. The sustained-pressure guard remained clear. An initial Watchman timeout is still reported as recovery; no complete inventory or twenty-four-hour observation is claimed. No host swap setting is changed. Host-side diagnostics stay outside the constrained helper. Retained state, warm caches and diagnostic placement prevent treating the retry as an isolated memory-performance comparison.

The Windows observer is prepared with a separate local catalog and no original or production-catalog mounts. Its clock starts after NAS discovery is healthy. Actual scraper access paths, throughput impact, controlled NAS fixtures, completed-inventory comparison, recovery/rollback and normal-operation discovery gates remain unverified. Production remains legacy with ten workers. No scraper configuration, source permissions, exports, firewall or SSH policy were changed.

Bounded validation and remaining discovery work

The owner replaced the twenty-four-hour observation requirement with bounded spot validation. The explicit short policy targets a ten-minute observation; all other evidence checks remain enforced. The live fixture test took about two minutes and passed additions (about sixteen seconds), growth, settling, rename with preserved identity and annotations, confirmed deletion and duplicate replay into a separate partial catalog. Production received zero fixture rows; zero derivative jobs and source cleanup were verified. See aggregate NAS evidence.

The one-GiB trial then reached its one-hour initial-recovery deadline without a healthy full inventory. The controller and helper stopped with state retained. All 110 resource samples showed zero memory-limit hits; the stop does not prove RAM-capacity exhaustion. No completed full inventory, ten-minute observation or full-catalog equivalence is claimed. This supersedes the running-state checkpoint above. Production remains legacy with ten workers.

The next performance work is fresh-clock delta delivery during reconciliation, cached watcher metadata with fewer source stats, removal of duplicate initial metadata passes, bounded snapshot finalization and inexpensive queue health. These are implementation targets, not measured speedups or production activation.

Repository CI repair

The documentation PR exposed an unrelated source-verifier defect: reading a reviewed file could update its access time and be rejected as source tampering. A disposable-file reproduction changed only access time. PR #549 fixed the comparison while retaining identity, ownership, mode, link count, size and nanosecond modification/change checks. It merged as e04bbc3c after owner approval and all twenty-seven checks passed at d10ad7c0. The affected Python 3.13 suites passed all sixty-three tests, with one opt-in Compose test skipped. This source repair performed no native collection or service deployment.