You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
Tomáš Dvořák
cf688634c4
feat: hash-based duplicate detection and instant upload (#191)
* feat: hash-based duplicate detection and instant upload
Clients send a client-computed sha256 with the upload session request;
when upload_dedup_scope is enabled and a completed entity with the same
hash and size exists within scope, the file is materialized by linking
the existing entity (refcount++) instead of transferring bytes.
- entity.hash column + (hash,size) index; stored on entity creation
- FileClient.FindEntityByHash: completed version entities only
(reference_count > 0, no upload session), owner-scoped unless global
- CreateFile LinkedEntityID branch: attach entity, bump refcount, set
primary entity + logical size, charge owner quota via storage diff
- DBFS.PrepareUpload rapid path: dedup lookup after validation, commits
file + emits upload event, skips session metadata/lock/credentials
- manager short-circuits rapid sessions before driver token + KV store
- postProcessRapidUpload queues media-meta / FTS tasks per file
- upload_dedup_scope setting (off/owner/global, default owner) surfaced
in admin FileSystem parameters and basic site config
- Frontend streams SHA-256 via hash-wasm (no whole-file buffering),
skips hashing for encrypted policies, short-circuits on rapid_uploaded
- Tests: FindEntityByHash scoping/isolation, linked CreateFile, and
full PrepareUpload rapid path incl. scope=off
Implements upstream cloudreve/cloudreve#3044 (dedup detection);
addresses fork issues #66/#15.
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* docs: roadmap — dedup landed (#191), multi-file share scoped
---------
Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
|
2 weeks ago |
| .. |
|
ar-AR
|
feat: monorepo restructure — vendor frontend + desktop, scaffold android, rewrite CI/CD
|
2 weeks ago |
|
de-DE
|
feat: monorepo restructure — vendor frontend + desktop, scaffold android, rewrite CI/CD
|
2 weeks ago |
|
en-US
|
feat: hash-based duplicate detection and instant upload (#191)
|
2 weeks ago |
|
es-ES
|
feat: monorepo restructure — vendor frontend + desktop, scaffold android, rewrite CI/CD
|
2 weeks ago |
|
fr-FR
|
feat: monorepo restructure — vendor frontend + desktop, scaffold android, rewrite CI/CD
|
2 weeks ago |
|
it-IT
|
feat: monorepo restructure — vendor frontend + desktop, scaffold android, rewrite CI/CD
|
2 weeks ago |
|
ja-JP
|
feat: monorepo restructure — vendor frontend + desktop, scaffold android, rewrite CI/CD
|
2 weeks ago |
|
ko-KR
|
feat: monorepo restructure — vendor frontend + desktop, scaffold android, rewrite CI/CD
|
2 weeks ago |
|
pl-PL
|
feat: monorepo restructure — vendor frontend + desktop, scaffold android, rewrite CI/CD
|
2 weeks ago |
|
pt-BR
|
feat: monorepo restructure — vendor frontend + desktop, scaffold android, rewrite CI/CD
|
2 weeks ago |
|
ru-RU
|
feat: monorepo restructure — vendor frontend + desktop, scaffold android, rewrite CI/CD
|
2 weeks ago |
|
zh-CN
|
feat: hash-based duplicate detection and instant upload (#191)
|
2 weeks ago |
|
zh-TW
|
feat: monorepo restructure — vendor frontend + desktop, scaffold android, rewrite CI/CD
|
2 weeks ago |