perf: batch E2EE queries, batch dir rename, hash lookup, larger Windows watcher buffer#9993
Open
qoole wants to merge 1 commit intonextcloud:masterfrom
Open
perf: batch E2EE queries, batch dir rename, hash lookup, larger Windows watcher buffer#9993qoole wants to merge 1 commit intonextcloud:masterfrom
qoole wants to merge 1 commit intonextcloud:masterfrom
Conversation
…sh lookup - Batch E2EE path queries: replace N serial getFileRecord calls with single IN query for getRootE2eFolderRecord and findEncryptedAncestorForRecord. With deeply nested encrypted folders this reduces sync startup latency by collapsing per-level DB round trips into a single statement. - Batch directory rename: collect child records first, then process mutations after the streaming query completes. Avoids interleaved read/write on the same statement and lets the journal commit cleanly. Commit the journal once before the batch loop so each child rename starts from a clean transaction boundary instead of accumulating in the previous batch's transaction. - Remove Instantiator destroy/recreate hack on tray open / menu close (eliminates full delegate teardown + rebuild on every interaction). - O(n^2) to O(1) folder adjustment: QHash lookup replaces std::find_if in adjustDeletedFoldersWithNewChildren. Builds a name index once and reuses it instead of rescanning per item. - Windows file watcher buffer: 40KB->256KB start, 64KB->2MB max. Larger initial buffer prevents buffer overflow under heavy file churn, which was triggering expensive full-tree rescans. Signed-off-by: Qoole <2862661+qoole@users.noreply.github.com>
7fe0d3d to
c260fac
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
A set of related performance fixes touching sync startup, propagation, and the tray.
Batch E2EE path queries (
syncjournaldb.cpp)getRootE2eFolderRecordandfindEncryptedAncestorForRecordpreviously walked up the path one level at a time, issuing a separategetFileRecordper level. For a path 8 levels deep that's 8 round trips to SQLite on every E2EE-related lookup.Replaced with a single
WHERE path IN (...)query that fetches all candidate ancestors at once. The result is then walked locally to find the deepest e2e-encrypted match.Batch directory rename (
propagatorjobs.cpp)PropagateLocalRename::startpreviously processed child journal entries inline while iterating a streamingSELECT. Mixing reads and writes on the same SQLite statement is fragile and prevented batching. Refactored to:SELECTis open.This makes the operation safer and lets the journal batch its writes.
Remove
Instantiatordestroy/recreate hack (MainWindow.qml,CurrentAccountHeaderButton.qml)The tray was destroying and recreating its account
Instantiatoron every menu open/close as a workaround for an old binding glitch. This causes a full delegate teardown and rebuild every time the user opens the tray. The hack is no longer needed; removed it.O(n²) -> O(1) folder adjustment (
owncloudpropagator.cpp)adjustDeletedFoldersWithNewChildrenlooked up each item withstd::find_ifover the full vector, making the function O(n²). Replaced with aQHash<QString, int>index built once and reused for each lookup.Windows file watcher buffer (
folderwatcher_win.cpp)The
ReadDirectoryChangesWbuffer was 40KB initial / 64KB max, which overflows easily under heavy file churn (e.g. large extracts, IDE save cascades). On overflow Windows reports a single "buffer overflow" event and the watcher falls back to a full-tree rescan, which is extremely expensive on large folders.Bumped to 256KB initial / 2MB max. The buffer only grows as needed.
Checklist
AI (if applicable)