diff --git a/CLAUDE.md b/CLAUDE.md index 8ab04e3dc..3e8b69247 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -63,9 +63,8 @@ scheduler → event emitters), `initHTTPServer()` (Fastify plugins, auth, routes `SystemIds` passed to each model's `init()` during first-run seeding. - `modules/` — pluggable extensions, discovered from disk. Each module is a directory with a `definition.yml` (key, title, props/config schema) plus its implementation — e.g. - `modules/authentication/local/`. `modules/storage/*` is definition-only so far: the admin area - stores a configuration per site and module, but no `storage.ts` exists yet and nothing reads or - writes content through a target — pages and assets go straight to the database. + `modules/authentication/local/`. `modules/storage/*` ships `db` and `disk` — see + [Storage targets](#storage-targets). - `tasks/simple/` — jobs run in-process by the scheduler; each exports `task()`. File name is kebab-case, the task key is its camelCase form. - `tasks/workers/` — CPU-bound jobs run in a worker thread via `worker.ts`, which boots a minimal @@ -162,6 +161,23 @@ npm run build # rollup → blocks/compiled/ The API is browsable via Swagger UI at `http://localhost:3000/_api` in a running instance. Default admin login is `admin@example.com` / `12345678`. +### How far to go verifying a change + +Match the check to the size of the change. `npm run build`, `npx oxlint` and `npm run typecheck` are +seconds each and are the right check for nearly everything. + +**Do not stand up a throwaway instance and drive a headless browser to look at a small change.** That +means booting a backend against a scratch database, seeding it, and screenshotting through +`/usr/bin/chromium` — a good ten minutes of setup that a moved border, a colour, a spacing tweak or a +renamed label does not earn. Read the rule you wrote, trust the build, and say what you changed. + +It is worth the setup for a **new** piece of UI whose markup has to meet a stylesheet written +elsewhere, where being wrong means shipping something visibly broken — a component reusing existing +content classes is the case that has actually gone wrong. Also for a flow with real state to exercise +(a login, an upload, a save), where a screenshot answers a question reading cannot. + +See the `wikijs-isolated-test-instance` memory for how to boot one when it IS warranted. + ## TypeScript (backend) The backend is entirely **TypeScript 7** (the native Go compiler — `tsc` is a platform binary, not a @@ -382,6 +398,111 @@ Consequences worth knowing: [Utilities and dates](#utilities-and-dates); the `lodash-es` and `luxon` still present in older files are on their way out. +### Storage targets + +A storage target is one module from `modules/storage//` configured for one site. Two modules +ship, both with a real `storage.ts`: `db`, enabled on every site and impossible to turn off, and +`disk`, which mirrors the wiki's tree at `///` — the configured root is +the site's own folder, since a target belongs to one site, so two sites must not share a path. S3, +Azure, GCS, git and SFTP were removed rather than left as definitions with nothing behind them; a +new module is a directory with a `definition.yml` and a `storage.ts` exporting the `StorageModule` +contract, and `hasImplementation` gates both dispatch and the admin area's action buttons on the +latter existing. + +**Content is written to every target that claims it, and read from one.** Those are two separate +questions with two separate answers, and conflating them is the way to get this wrong: + +- **Written** — a target's `contentTypes.activeTypes` says what is stored there, and a site may store + the same kind in several places at once. An upload goes to *all* of them; the admin area's + **Targets** tab is where that is set, per target. +- **Read** — `assetDelivery.servedTypes` names the content types a reader's request is answered from + that target, at most one target per type across the site. The **Content Delivery** tab sets it, and + a target may only be nominated for a type it also stores (`validateTarget` refuses the pair). Pages + are never nominated: a page is read from its own row, always. + + **Only an explicit nomination moves delivery.** A type nobody has been nominated for is served + from the database (`deliveryTargetsFor`), never from whichever other target happens to be enabled + — enabling one says where content is *written*, and a target enabled after an upload holds none of + the existing files anyway. Disabling a target therefore puts delivery back, and `updateTarget` + clears `servedTypes` as it goes so that re-enabling it later does not silently take the content + type with it. The database gives the role up only by not holding the type at all. + +So neither an asset nor a page records where its bytes went — there is no single place — and each +target derives where its own copy sits from the tree, the same way for both. `resolveTargetFor` and +`assets.storageInfo` are gone; `writeTargetsFor` and `deliveryTargetsFor` replace them. + +**Which content type a file is, is one answer per site, not per target.** `large` is a category of +its own rather than a modifier — that is what lets a target take the 40 MB video without also taking +every thumbnail — and the size at which it starts lives in the site's config as +`storage.largeThreshold` (`storage.largeThresholdFor`, the admin area's **Configuration** tab). It +has to be shared: a file the disk target called large and the database called an image would be +claimed by neither target, or by both. The whole storage configuration of a site — the site-wide +settings and every target — is read and written as one, through `GET`/`PUT /sites/:siteId/storage`. + +Everything else follows from that: + +- **A write must succeed everywhere; a read may fall back.** `storage.putAsset` fans out and throws if + any target refuses, failing the upload — an asset may have no database copy, so a half-stored one + must not be reported as saved. `getAsset` tries the nominated source and then every other target + holding the content, database last, because a target enabled after an upload never received it. + Pages are gentler still: `mirrorPage` / `removePage` / `relocatePage` log a target that could not + keep up and carry on, since the database always has the page. +- **The disk target's `exportAll` is a copy, not a move.** It writes out everything it is configured + to hold, overwriting, and touches neither the database nor any record — which is how content that + predates the target being enabled gets onto it. +- **The move is the database target's `offloadUnchecked`.** Turning a target on only affects what is + uploaded from then on, so a site that unticks a content type on the database is still carrying + every file of that kind ever uploaded — and carrying it unreachably, since a target is only read + for a type it stores. That action reads each of those out of its row, writes it to every enabled + target holding the type, **reads it back to check** and only then clears the `data` column. An + asset with no destination keeps its copy and is reported as stranded: nothing is cleared that is + not known to be somewhere else, because this is the only copy of the bytes. Metadata is untouched + throughout — `data` is the one column it empties. +- **Renames have files to move on every target.** `assets.relocateAssets` takes the old location from + the caller and reads the new one off the tree; `tree.renameFolder` does the same for everything + beneath a renamed folder, pages included (moved, not rewritten — nothing changed). +- **Thumbnails always stay in the database** (`assets.preview`) — the file manager asks for a + screenful at a time, and a slow target must not cost a wiki its file browser. +- **`pages.adoptStoredPage` and `assets.adoptStoredFile` are the way back in**, for the disk + target's two import actions. Both take an `overwrite` flag, and it is the only thing separating + them: `importAll` leaves a path the wiki already has alone, because reconciling a file changed on + both sides is a merge and belongs to a target with history, while `importAllOverwrite` lets the + folder win — for a restore, where there is nothing to reconcile. A page is replaced through + `updatePage`, so its previous version is in its history; an asset has none, and `replace` also + dispatches the new bytes to every write target, since the copy a reader is served is usually the + database's. Imported content is rendered with **no script or style permission** whoever ran the + import, since the file need not have been written by them. +- **On import a file is a page if its extension is reserved, or if it declares an `editor`** in its + front matter. A text page is front matter plus the source; a **JSON** page — a redirection today — + is one JSON document with the metadata at its top level and the source under `content`. + +**Pages and assets share one folder, and the site's `pageExtensions` is what keeps them apart.** A +page is stored under its editor's extension (`md`, `html`, `adoc`, `json`), while the tree holds its +name without one — so page `notes/readme` and an attachment called `readme.md` are different names to +the wiki and the same file on disk. Three rules stop them ever meeting, and all three live in the +models rather than in the storage module: + +- **`pageExtensions` are reserved.** `assets.upload` refuses an attachment using one — a `.md` file is + a page, so uploading it as an attachment is a mistake rather than a collision. This is the whole of + it on a default site (`md,html,txt`). +- **Both sides check anyway.** For an extension a site has taken *off* that list, + `assets.guardAgainstPageCollision` and `pages.guardAgainstAssetCollision` refuse whichever arrives + second. Extensions must match to collide: `readme.pdf` sits happily beside the page `readme`. +- **Nothing guesses at a name.** `StoragePageRef` carries `contentType`, so a delete or a move touches + exactly one file — a target that tried each extension in turn would delete the attachment next door. + +**A target's `state` column is how it is behaving, not how it is configured.** `{ status: 'healthy' | +'warning' | 'error', message, updatedAt }`, written only by `storage.recordState` as the model +dispatches to a module, absent from `StorageTargetInput`, and reported by the Status card. `error` is +a failure that was raised to whoever asked (a refused upload); `warning` is one that was swallowed +because the request succeeded anyway (a page copy that could not be written). The last operation +wins — a later success clears an earlier failure, which is what makes a full disk that gets emptied +stop reporting itself without anybody dismissing anything. Nothing probes a target proactively, so a +misconfigured one reads healthy until something is actually asked of it. + +Not to be confused with `/cache/files`, the serving cache in the assets model. That one is +derived and swept; a storage target is where content actually lives. + ### Icons Icons come from **Iconify** and are referenced the way Iconify references them — `:`, diff --git a/backend/api/schemas/storage.ts b/backend/api/schemas/storage.ts index fa750299a..ca43f0640 100644 --- a/backend/api/schemas/storage.ts +++ b/backend/api/schemas/storage.ts @@ -1,4 +1,4 @@ -import { CONTENT_TYPES } from '../../models/storage.ts' +import { CONTENT_TYPES, STORAGE_TARGET_STATUSES } from '../../models/storage.ts' import type { FastifyInstance } from 'fastify' export async function registerSchemas(app: FastifyInstance): Promise { @@ -40,7 +40,8 @@ export async function registerSchemas(app: FastifyInstance): Promise { }, contentTypes: { type: 'object', - description: 'Which kinds of content this target holds.', + description: + 'Which kinds of content are written to this target. Not a choice between targets: a site may store the same kind in several places at once, and every one of them receives a copy.', properties: { activeTypes: { type: 'array', @@ -48,10 +49,6 @@ export async function registerSchemas(app: FastifyInstance): Promise { type: 'string', enum: [...CONTENT_TYPES] } - }, - largeThreshold: { - type: 'string', - description: 'Size above which an asset counts as a large file, e.g. `5MB`.' } } }, @@ -71,42 +68,15 @@ export async function registerSchemas(app: FastifyInstance): Promise { }, directAccess: { type: 'boolean' - } - } - }, - versioning: { - type: 'object', - description: - 'Whether past versions are kept. `isForceEnabled` marks a module where versioning is inherent, such as git.', - properties: { - isSupported: { - type: 'boolean' }, - isForceEnabled: { - type: 'boolean' - }, - enabled: { - type: 'boolean' - } - } - }, - setup: { - type: 'object', - description: - 'Only present for a module that has a setup process and an implementation to run it.', - properties: { - handler: { - type: 'string', - description: 'Which setup flow the admin area should walk through, e.g. `github`.' - }, - state: { - type: 'string', - enum: ['notconfigured', 'pendinginstall', 'configured'] - }, - values: { - type: 'object', - additionalProperties: true, - description: 'Values the setup form starts from.' + servedTypes: { + type: 'array', + description: + 'The content types a request for a file is answered from this target. A subset of `contentTypes.activeTypes`, since a target can only serve back what it was asked to store, and across a site each type names at most one target.', + items: { + type: 'string', + enum: [...CONTENT_TYPES] + } } } }, @@ -147,6 +117,26 @@ export async function registerSchemas(app: FastifyInstance): Promise { } } } + }, + state: { + type: 'object', + description: + 'How the target is behaving, as opposed to how it is configured. Read-only and absent from `StorageTargetInput`: it records the outcome of the last operation the wiki asked of this target, not anything an administrator sets. `warning` is an operation that failed without being refused - a page copy that could not be written - and `error` is one that was reported to whoever asked, such as a failed upload. A subsequent success clears either.', + properties: { + status: { + type: 'string', + enum: [...STORAGE_TARGET_STATUSES] + }, + message: { + type: 'string', + description: 'What went wrong. Empty when healthy.' + }, + updatedAt: { + type: ['string', 'null'], + description: + 'When the status was last written, or null for a target that has not been asked to do anything yet.' + } + } } } }) @@ -166,7 +156,7 @@ export async function registerSchemas(app: FastifyInstance): Promise { isEnabled: { type: 'boolean', description: - 'The database target cannot be disabled, and a target with a pending setup cannot be enabled.' + 'The database target cannot be disabled, and a module without an implementation cannot be enabled.' }, contentTypes: { type: 'object', @@ -177,10 +167,6 @@ export async function registerSchemas(app: FastifyInstance): Promise { type: 'string', enum: [...CONTENT_TYPES] } - }, - largeThreshold: { - type: 'string', - maxLength: 32 } } }, @@ -193,16 +179,14 @@ export async function registerSchemas(app: FastifyInstance): Promise { }, directAccess: { type: 'boolean' - } - } - }, - versioning: { - type: 'object', - description: - 'Ignored by a module that does not support versioning or that forces it on — the module decides, not the client.', - properties: { - enabled: { - type: 'boolean' + }, + servedTypes: { + type: 'array', + description: 'Refused for a content type this target is not also configured to store.', + items: { + type: 'string', + enum: [...CONTENT_TYPES] + } } } }, diff --git a/backend/api/storage.ts b/backend/api/storage.ts index c2f1b60e1..4af278536 100644 --- a/backend/api/storage.ts +++ b/backend/api/storage.ts @@ -6,18 +6,18 @@ import type { StorageTargetInput } from '../models/storage.ts' */ async function routes(app: FastifyInstance) { /** - * LIST SITE STORAGE TARGETS + * GET SITE STORAGE CONFIGURATION */ app.get<{ Params: { siteId: string } }>( - '/sites/:siteId/storage/targets', + '/sites/:siteId/storage', { config: { permissions: ['manage:system'] }, schema: { - summary: 'List the storage targets of a site', + summary: 'Get the storage configuration of a site', description: - 'One target per storage module installed in `modules/storage`, whether or not it has ever been enabled. Configuration values include any credentials a module stores, hence the `manage:system` requirement. Note that no module ships an implementation yet: a target holds configuration, and nothing reads or writes content through it.', + 'The site-wide settings, plus one target per storage module installed in `modules/storage`, whether or not it has ever been enabled. Configuration values include any credentials a module stores, hence the `manage:system` requirement. Where a given file is written and where it is read from are both derived from this configuration rather than recorded per file, so changing it changes where content is looked for, not where it already sits.', tags: ['Storage'], params: { type: 'object', @@ -31,9 +31,19 @@ async function routes(app: FastifyInstance) { }, response: { 200: { - description: 'List of storage targets', - type: 'array', - items: { $ref: 'StorageTarget#' } + description: 'Storage configuration of the site', + type: 'object', + properties: { + largeThreshold: { + type: 'string', + description: + 'Size at or above which an asset counts as a large file, e.g. `5MB`. One answer for the whole site: a file has to be the same kind of thing to every target.' + }, + targets: { + type: 'array', + items: { $ref: 'StorageTarget#' } + } + } } } } @@ -43,23 +53,29 @@ async function routes(app: FastifyInstance) { if (!site) { return reply.notFound('Site does not exist.') } - return WIKI.models.storage.getSiteTargets(req.params.siteId) + return { + largeThreshold: WIKI.models.storage.largeThresholdFor(req.params.siteId), + targets: await WIKI.models.storage.getSiteTargets(req.params.siteId) + } } ) /** - * UPDATE SITE STORAGE TARGETS + * UPDATE SITE STORAGE CONFIGURATION */ - app.put<{ Params: { siteId: string }; Body: { targets: StorageTargetInput[] } }>( - '/sites/:siteId/storage/targets', + app.put<{ + Params: { siteId: string } + Body: { largeThreshold?: string; targets?: StorageTargetInput[] } + }>( + '/sites/:siteId/storage', { config: { permissions: ['manage:system'] }, schema: { - summary: 'Update the storage targets of a site', + summary: 'Update the storage configuration of a site', description: - 'Only the targets listed are affected, and within each of them only the fields provided. Every target is validated before any of them is written, so a rejected request changes nothing.', + 'Only the targets listed are affected, and within each of them only the fields provided. Everything is validated before any of it is written, so a rejected request changes nothing.', tags: ['Storage'], params: { type: 'object', @@ -73,8 +89,12 @@ async function routes(app: FastifyInstance) { }, body: { type: 'object', - required: ['targets'], properties: { + largeThreshold: { + type: 'string', + maxLength: 32, + description: 'A size such as `5MB`. Applies to every target of the site.' + }, targets: { type: 'array', items: { $ref: 'StorageTargetInput#' } @@ -83,7 +103,7 @@ async function routes(app: FastifyInstance) { }, response: { 200: { - description: 'Storage targets updated successfully', + description: 'Storage configuration updated successfully', type: 'object', properties: { ok: { @@ -110,9 +130,13 @@ async function routes(app: FastifyInstance) { // -> Validated as a whole first: a partially applied storage configuration is worse than a // refused one, since the admin area saves every target at once + const invalidConfig = WIKI.models.storage.validateSiteConfig(req.body) + if (invalidConfig) { + return reply.badRequest(invalidConfig) + } const current = await WIKI.models.storage.getSiteTargets(req.params.siteId) const patches = [] - for (const patch of req.body.targets) { + for (const patch of req.body.targets ?? []) { const target = current.find((t) => t.id === patch.id) if (!target) { return reply.notFound(`Storage target ${patch.id} does not exist.`) @@ -124,6 +148,8 @@ async function routes(app: FastifyInstance) { patches.push({ target, patch }) } + await WIKI.models.storage.updateSiteConfig(req.params.siteId, req.body) + let updated = 0 for (const { target, patch } of patches) { if (await WIKI.models.storage.updateTarget(req.params.siteId, target, patch)) { @@ -133,7 +159,7 @@ async function routes(app: FastifyInstance) { return { ok: true, - message: 'Storage targets updated successfully.', + message: 'Storage configuration updated successfully.', updated } } @@ -151,7 +177,7 @@ async function routes(app: FastifyInstance) { schema: { summary: 'Run an action on a storage target', description: - 'The actions a target offers are listed with it. Only an enabled target can run one, and only a module with an implementation offers any — so every action currently fails, no module having one yet.', + 'The actions a target offers are listed with it, and only an enabled target can run one. An action runs to completion before the request is answered, so moving a large amount of content can take a while.', tags: ['Storage'], params: { type: 'object', @@ -201,103 +227,18 @@ async function routes(app: FastifyInstance) { if (!target.actions.some((act) => act.handler === req.params.action)) { return reply.badRequest(`${target.title} has no "${req.params.action}" action.`) } - - try { - await WIKI.models.storage.executeAction(target, req.params.action) - } catch (err: any) { - WIKI.logger.warn(err) - return reply.badRequest(err.message) - } - - return { - ok: true, - message: 'Action completed successfully.' - } - } - ) - - /** - * RUN STORAGE TARGET SETUP STEP - */ - app.post<{ - Params: { siteId: string; targetId: string } - Body: Record - }>( - '/sites/:siteId/storage/targets/:targetId/setup', - { - config: { - permissions: ['manage:system'] - }, - schema: { - summary: 'Advance the setup process of a storage target', - description: - 'For modules that cannot be configured by hand, such as one backed by an app installed on a provider. The body is passed to the module as-is, and what comes back tells the admin area what to do next. Only a module with an implementation has a setup process — none does yet.', - tags: ['Storage'], - params: { - type: 'object', - properties: { - siteId: { - type: 'string', - format: 'uuid' - }, - targetId: { - type: 'string', - format: 'uuid' - } - }, - required: ['siteId', 'targetId'] - }, - body: { - type: 'object', - required: ['step'], - additionalProperties: true, - properties: { - step: { - type: 'string', - maxLength: 255, - description: 'Which step of the process to run, as named by the module.' - } - } - }, - response: { - 200: { - description: 'Setup step completed successfully', - type: 'object', - properties: { - ok: { - type: 'boolean' - }, - message: { - type: 'string' - }, - state: { - type: 'object', - additionalProperties: true, - description: 'What the module wants done next, e.g. `{ nextStep, url }`.' - } - } - } - } - } - }, - async (req, reply) => { - const target = await WIKI.models.storage.getSiteTargetById( - req.params.siteId, - req.params.targetId - ) - if (!target) { - return reply.notFound('Storage target does not exist.') - } - if (!target.setup) { - return reply.badRequest(`${target.title} has no setup process.`) + // -> An action may create content, and content records who authored it. An API key is not a + // who, so these are reserved to a logged-in administrator. + const actorId = req.session?.authenticated ? req.session.user?.id : null + if (!actorId) { + return reply.unauthorized('Running a storage action requires a logged in user.') } try { - const state = await WIKI.models.storage.runSetup(target, req.body) + const message = await WIKI.models.storage.executeAction(target, req.params.action, actorId) return { ok: true, - message: 'Setup step completed successfully.', - state + message: message ?? 'Action completed successfully.' } } catch (err: any) { WIKI.logger.warn(err) @@ -305,76 +246,6 @@ async function routes(app: FastifyInstance) { } } ) - - /** - * DESTROY STORAGE TARGET SETUP - */ - app.delete<{ Params: { siteId: string; targetId: string } }>( - '/sites/:siteId/storage/targets/:targetId/setup', - { - config: { - permissions: ['manage:system'] - }, - schema: { - summary: 'Reset the setup of a storage target', - description: - 'Undoes what the setup process configured, so that it can be started over. What that involves is up to the module.', - tags: ['Storage'], - params: { - type: 'object', - properties: { - siteId: { - type: 'string', - format: 'uuid' - }, - targetId: { - type: 'string', - format: 'uuid' - } - }, - required: ['siteId', 'targetId'] - }, - response: { - 200: { - description: 'Setup reset successfully', - type: 'object', - properties: { - ok: { - type: 'boolean' - }, - message: { - type: 'string' - } - } - } - } - } - }, - async (req, reply) => { - const target = await WIKI.models.storage.getSiteTargetById( - req.params.siteId, - req.params.targetId - ) - if (!target) { - return reply.notFound('Storage target does not exist.') - } - if (!target.setup) { - return reply.badRequest(`${target.title} has no setup process.`) - } - - try { - await WIKI.models.storage.destroySetup(target) - } catch (err: any) { - WIKI.logger.warn(err) - return reply.badRequest(err.message) - } - - return { - ok: true, - message: 'Setup reset successfully.' - } - } - ) } export default routes diff --git a/backend/db/migrations/20260809235619_init/migration.sql b/backend/db/migrations/20260809235619_init/migration.sql index 73a4aa433..441bde36f 100644 --- a/backend/db/migrations/20260809235619_init/migration.sql +++ b/backend/db/migrations/20260809235619_init/migration.sql @@ -41,7 +41,6 @@ CREATE TABLE "assets" ( "updatedAt" timestamp DEFAULT now() NOT NULL, "data" bytea, "preview" bytea, - "storageInfo" jsonb, "authorId" uuid NOT NULL, "siteId" uuid NOT NULL ); @@ -316,7 +315,6 @@ CREATE TABLE "storage" ( "isEnabled" boolean DEFAULT false NOT NULL, "contentTypes" jsonb DEFAULT '{}' NOT NULL, "assetDelivery" jsonb DEFAULT '{}' NOT NULL, - "versioning" jsonb DEFAULT '{}' NOT NULL, "config" jsonb DEFAULT '{}' NOT NULL, "state" jsonb DEFAULT '{}' NOT NULL, "siteId" uuid NOT NULL @@ -456,4 +454,4 @@ ALTER TABLE "tags" ADD CONSTRAINT "tags_siteId_sites_id_fkey" FOREIGN KEY ("site ALTER TABLE "tree" ADD CONSTRAINT "tree_siteId_sites_id_fkey" FOREIGN KEY ("siteId") REFERENCES "sites"("id");--> statement-breakpoint ALTER TABLE "userGroups" ADD CONSTRAINT "userGroups_userId_users_id_fkey" FOREIGN KEY ("userId") REFERENCES "users"("id") ON DELETE CASCADE;--> statement-breakpoint ALTER TABLE "userGroups" ADD CONSTRAINT "userGroups_groupId_groups_id_fkey" FOREIGN KEY ("groupId") REFERENCES "groups"("id") ON DELETE CASCADE;--> statement-breakpoint -ALTER TABLE "userKeys" ADD CONSTRAINT "userKeys_userId_users_id_fkey" FOREIGN KEY ("userId") REFERENCES "users"("id"); \ No newline at end of file +ALTER TABLE "userKeys" ADD CONSTRAINT "userKeys_userId_users_id_fkey" FOREIGN KEY ("userId") REFERENCES "users"("id"); diff --git a/backend/db/migrations/20260809235619_init/snapshot.json b/backend/db/migrations/20260809235619_init/snapshot.json index 5e25489f0..622ffad33 100644 --- a/backend/db/migrations/20260809235619_init/snapshot.json +++ b/backend/db/migrations/20260809235619_init/snapshot.json @@ -651,19 +651,6 @@ "schema": "public", "table": "assets" }, - { - "type": "jsonb", - "typeSchema": null, - "notNull": false, - "dimensions": 0, - "default": null, - "generated": null, - "identity": null, - "name": "storageInfo", - "entityType": "columns", - "schema": "public", - "table": "assets" - }, { "type": "uuid", "typeSchema": null, @@ -3306,19 +3293,6 @@ "schema": "public", "table": "storage" }, - { - "type": "jsonb", - "typeSchema": null, - "notNull": true, - "dimensions": 0, - "default": "'{}'", - "generated": null, - "identity": null, - "name": "versioning", - "entityType": "columns", - "schema": "public", - "table": "storage" - }, { "type": "jsonb", "typeSchema": null, @@ -5746,4 +5720,4 @@ } ], "renames": [] -} \ No newline at end of file +} diff --git a/backend/db/schema.ts b/backend/db/schema.ts index 3e59f7bc6..00927a85f 100644 --- a/backend/db/schema.ts +++ b/backend/db/schema.ts @@ -99,9 +99,11 @@ export const assets = pgTable( meta: jsonb().notNull().default({}), createdAt: timestamp().notNull().defaultNow(), updatedAt: timestamp().notNull().defaultNow(), + // -> Set only while the database is one of the targets configured to store this kind of file. + // An asset is written to every target that claims it, and each derives where its own copy + // sits from the tree, so there is nothing to record here about where the bytes went. data: bytea(), preview: bytea(), - storageInfo: jsonb(), authorId: uuid() .notNull() .references(() => users.id), @@ -652,16 +654,19 @@ export const storage = pgTable( // -> Directory name under `modules/storage`, one row per module per site module: varchar({ length: 255 }).notNull(), isEnabled: boolean().notNull().default(false), - // -> `{ activeTypes: string[], largeThreshold: string }` + // -> `{ activeTypes: string[] }`, i.e. which kinds of content are written here. What counts as a + // large file is not among them: that is one answer per site, in the site's own config. contentTypes: jsonb().notNull().default({}), - // -> `{ streaming: boolean, directAccess: boolean }` + // -> `{ streaming: boolean, directAccess: boolean, servedTypes: string[] }`. `servedTypes` names + // the content types a reader's request is answered from this target, and is a subset of + // `contentTypes.activeTypes` — a target can only serve back what it was asked to store. assetDelivery: jsonb().notNull().default({}), - // -> `{ enabled: boolean }` - versioning: jsonb().notNull().default({}), // -> Values for the props the module declares in its `definition.yml` config: jsonb().notNull().default({}), - // -> Where the module stands, as opposed to how it is configured: `{ setup: 'notconfigured' | - // 'pendinginstall' | 'configured' }` for a module that has a setup process to go through. + // -> `{ status: 'healthy' | 'warning' | 'error', message: string, updatedAt: string | null }`: + // how the target is actually behaving, as opposed to how it is configured. Written by the + // storage model as it dispatches to the module — never by the admin area, which is why it is + // absent from the storage PUT — and reported by the Status card on the target's page. state: jsonb().notNull().default({}), siteId: uuid() .notNull() diff --git a/backend/locales/en.json b/backend/locales/en.json index ce9084ced..0f2825795 100644 --- a/backend/locales/en.json +++ b/backend/locales/en.json @@ -345,7 +345,7 @@ "admin.general.logoUploadFailed": "Failed to upload the site logo.", "admin.general.logoUploadSuccess": "Site logo uploaded successfully.", "admin.general.pageExtensions": "Page Extensions", - "admin.general.pageExtensionsHint": "A comma-separated list of URL extensions that address a page. For example, adding md redirects /foobar.md to /foobar.", + "admin.general.pageExtensionsHint": "A comma-separated list of URL extensions that address a page. For example, adding md redirects /foobar.md to /foobar. These extensions are reserved for pages: a file using one cannot be uploaded as an asset.", "admin.general.ratingsOff": "Off", "admin.general.ratingsStars": "Stars", "admin.general.ratingsThumbs": "Thumbs", @@ -832,124 +832,54 @@ "admin.ssl.writableConfigFileWarning": "Note that your config file must be writable in order to persist ports configuration.", "admin.stats.title": "Statistics", "admin.storage.actionFailed": "Failed to run {action}.", - "admin.storage.actionRun": "Run", "admin.storage.actionSuccess": "{action} completed successfully.", "admin.storage.actions": "Actions", - "admin.storage.actionsInactiveWarn": "You must enable this storage target and apply changes before you can run actions.", - "admin.storage.addTarget": "Add Storage Target", - "admin.storage.assetDelivery": "Asset Delivery", - "admin.storage.assetDeliveryHint": "Select how uploaded assets should be delivered to the user. Note that not all storage origins support asset delivery and some can only be used for backup purposes.", - "admin.storage.assetDirectAccess": "Direct Access", - "admin.storage.assetDirectAccessHint": "Assets are accessed directly by the user using a secure / signed link. When enabled, takes priority over file streaming.", - "admin.storage.assetDirectAccessNotSupported": "Not supported by this storage target.", - "admin.storage.assetStreaming": "File Streaming", - "admin.storage.assetStreamingHint": "Assets will be streamed from the storage target, through the server, to the user.", - "admin.storage.assetStreamingNotSupported": "Not supported by this storage target.", + "admin.storage.actionsInactiveWarn": "You must enable this storage target before you can run actions.", "admin.storage.assetsOnly": "Assets Only", - "admin.storage.cancelSetup": "Cancel", "admin.storage.config": "Configuration", + "admin.storage.configHint": "Settings that apply to every storage target of this site.", + "admin.storage.confirmDisable": "Save and disable this target?", + "admin.storage.confirmEnable": "Save and enable this target?", + "admin.storage.confirmToggleHint": "Any unsaved changes to this site's storage configuration are applied as well.", "admin.storage.contentTypeDocuments": "Documents", "admin.storage.contentTypeDocumentsHint": "Text or presentation documents in PDF, TXT, Word, Excel and Powerpoint formats.", "admin.storage.contentTypeImages": "Images", "admin.storage.contentTypeImagesHint": "Image Assets in JPG, PNG, GIF, WebP and SVG formats.", "admin.storage.contentTypeLargeFiles": "Large Files", - "admin.storage.contentTypeLargeFilesDBWarn": "For performance reasons, large files should not be stored in the database. Consider using another storage type for these files.", - "admin.storage.contentTypeLargeFilesHint": "Large files such as videos, zip archives and binaries. Pages never fall into this category, irrespective of their size.", - "admin.storage.contentTypeLargeFilesThreshold": "Size Threshold", + "admin.storage.contentTypeLargeFilesDBWarn": "For performance reasons, large files should not be stored in the database. Consider using another storage target for these files.", + "admin.storage.contentTypeLargeFilesHint": "Large files such as videos, zip archives and binaries. Pages never fall into this category, irrespective of their size. Set the threshold in the Configuration tab.", "admin.storage.contentTypeOthers": "Others", "admin.storage.contentTypeOthersHint": "Any other file types that don't match the other categories.", "admin.storage.contentTypePages": "Pages", "admin.storage.contentTypePagesHint": "Page content source, in Markdown, HTML or JSON format depending on the editor.", + "admin.storage.contentTypePagesSource": "Pages are always served from the database, so this cannot be turned off.", "admin.storage.contentTypes": "Content Types", - "admin.storage.contentTypesHint": "Select the type of content that should be stored to this storage target:", - "admin.storage.currentState": "Current State", - "admin.storage.deliveryPaths": "Delivery Paths", - "admin.storage.deliveryPathsLegend": "Legend:", - "admin.storage.deliveryPathsPushToOrigin": "Push to Origin", - "admin.storage.deliveryPathsUser": "User", - "admin.storage.deliveryPathsUserRequest": "User Request", - "admin.storage.destroyConfirm": "Confirm Setup Reset", - "admin.storage.destroyConfirmInfo": "Are you sure you want to reset the storage target setup configuration? Note that this action cannot be undone and you will need to start the setup process over.", - "admin.storage.destroyingSetup": "Resetting storage target setup configuration...", - "admin.storage.enabled": "Enabled", - "admin.storage.enabledForced": "Cannot be disabled on the database target.", - "admin.storage.enabledHint": "Should this storage target be used for storing and accessing assets.", - "admin.storage.errorMsg": "Error Message", - "admin.storage.finishSetup": "Finish Setup", - "admin.storage.githubAccTypeOrg": "Organization", - "admin.storage.githubAccTypePersonal": "Personal", - "admin.storage.githubFinish": "Once you have installed the application on the GitHub repository of your choice, click the Finish Setup button below to validate the installation and start using it. Otherwise, click Destroy to delete any pending configuration and start over.", - "admin.storage.githubInstallApp": "Setup GitHub Connection - Step 2", - "admin.storage.githubInstallAppHint": "On the next screen, you will be prompted to install the app you just created onto one or more repositories. You may select a single one or all repositories.", - "admin.storage.githubOrg": "GitHub Organization", - "admin.storage.githubOrgHint": "Enter the GitHub organization account to be used.", - "admin.storage.githubPreparingManifest": "Preparing manifest...", - "admin.storage.githubPublicUrl": "Wiki Public URL", - "admin.storage.githubPublicUrlHint": "Enter the full URL to your wiki (e.g. https://wiki.example.com). Note that your wiki MUST be accessible from the internet!", - "admin.storage.githubRedirecting": "Redirecting to GitHub...", - "admin.storage.githubRepo": "GitHub Repository", - "admin.storage.githubRepoCreating": "Creating GitHub Repository...", - "admin.storage.githubRepoHint": "Enter the name of the repository to create on GitHub and use for this wiki:", - "admin.storage.githubSetupContinue": "Continue Setup", - "admin.storage.githubSetupDestroyFailed": "Failed to destroy setup configuration.", - "admin.storage.githubSetupDestroySuccess": "Setup configuration has been reset successfully.", - "admin.storage.githubSetupFailed": "GitHub Setup failed.", - "admin.storage.githubSetupInstallApp": "GitHub Connection Setup - Step 2", - "admin.storage.githubSetupInstallAppInfo": "On the next screen, you'll be prompted to install the app you just created onto one or more GitHub repositories.", - "admin.storage.githubSetupInstallAppReturn": "Once the installation on GitHub is completed, you will need to manually return to this page to finish the setup.", - "admin.storage.githubSetupInstallAppSelect": "IMPORTANT: Select only the repository that will be used to sync with this wiki.", - "admin.storage.githubSetupSuccess": "Success! Wiki.js is now connected to GitHub.", - "admin.storage.githubVerifying": "Verifying GitHub configuration...", + "admin.storage.contentTypesHint": "Select the type of content that is written to this storage target. The same type can be stored on several targets at once.", + "admin.storage.delivery": "Content Delivery", + "admin.storage.deliveryHint": "Choose which storage target each kind of content is served from when a reader requests a file.", + "admin.storage.deliveryNoTarget": "No enabled storage target is configured to store this content type.", + "admin.storage.deliveryPagesHint": "Pages are always served from the database.", + "admin.storage.deliveryRelationHint": "A target can only be chosen here for a content type it is also configured to store, under Targets.", "admin.storage.inactiveTarget": "Inactive", - "admin.storage.lastSync": "Last synchronization {time}", - "admin.storage.lastSyncAttempt": "Last attempt was {time}", + "admin.storage.largeThreshold": "Large File Size Threshold", + "admin.storage.largeThresholdHint": "Assets of this size or larger count as large files, whatever their type, such as videos, zip archives and binaries. Pages never fall into this category, irrespective of their size.", "admin.storage.loadFailed": "Failed to load storage configuration.", - "admin.storage.missingOrigin": "Missing Origin", "admin.storage.noActions": "This storage target has no actions that you can execute.", "admin.storage.noConfigOption": "This storage target has no configuration options you can modify.", - "admin.storage.noSyncModes": "This storage target has no synchronization options you can modify.", - "admin.storage.noTarget": "You don't have any active storage target.", "admin.storage.notConfigured": "Not Configured", "admin.storage.pagesAndAssets": "Pages and Assets", "admin.storage.pagesOnly": "Pages Only", "admin.storage.saveFailed": "Failed to save storage configuration.", "admin.storage.saveSuccess": "Storage configuration saved successfully.", - "admin.storage.setup": "Setup", - "admin.storage.setupConfiguredHint": "This module is already configured. You can uninstall this module to start over.", - "admin.storage.setupHint": "This module requires a setup process to be completed in order to use it. Follow the instructions below to get started.", - "admin.storage.setupRequired": "Setup required", - "admin.storage.startSetup": "Start Setup", + "admin.storage.stateActive": "Healthy", + "admin.storage.stateError": "Error", + "admin.storage.stateInactive": "Not in use", + "admin.storage.stateNoContentTypes": "No content type", + "admin.storage.stateWarning": "Degraded", "admin.storage.status": "Status", - "admin.storage.subtitle": "Set backup and sync targets for your content", - "admin.storage.sync": "Synchronization", - "admin.storage.syncDirBi": "Bi-directional", - "admin.storage.syncDirBiHint": "In bi-directional mode, content is first pulled from the storage target. Any newer content overwrites local content. New content since last sync is then pushed to the storage target, overwriting any content on target if present.", - "admin.storage.syncDirPull": "Pull from target", - "admin.storage.syncDirPullHint": "Content is always pulled from the storage target, overwriting any local content which already exists. This choice is usually reserved for single-use content import. Caution with this option as any local content will always be overwritten!", - "admin.storage.syncDirPush": "Push to target", - "admin.storage.syncDirPushHint": "Content is always pushed to the storage target, overwriting any existing content. This is safest choice for backup scenarios.", - "admin.storage.syncDirection": "Sync Direction", - "admin.storage.syncDirectionSubtitle": "Choose how content synchronization is handled for this storage target.", - "admin.storage.syncSchedule": "Sync Schedule", - "admin.storage.syncScheduleCurrent": "Currently set to every {schedule}.", - "admin.storage.syncScheduleDefault": "The default is every {schedule}.", - "admin.storage.syncScheduleHint": "For performance reasons, this storage target synchronize changes on an interval-based schedule, instead of on every change. Define at which interval should the synchronization occur.", - "admin.storage.targetConfig": "Target Configuration", - "admin.storage.targetState": "This storage target is {state}", - "admin.storage.targetStateActive": "active", - "admin.storage.targetStateInactive": "inactive", + "admin.storage.subtitle": "Choose where the content of your wiki is stored and served from", "admin.storage.targets": "Targets", "admin.storage.title": "Storage", - "admin.storage.uninstall": "Uninstall", - "admin.storage.unsupported": "Unsupported", - "admin.storage.useVersioning": "Use Versioning", - "admin.storage.useVersioningHint": "Should previous versions of assets be retained on the storage target.", - "admin.storage.vendor": "Vendor", - "admin.storage.vendorWebsite": "Website", - "admin.storage.versioning": "Asset Versioning", - "admin.storage.versioningForceEnabled": "Cannot be disabled on this storage target.", - "admin.storage.versioningHint": "Asset versioning allows for storage of all previous versions of the same file. Unless you have a requirement to store all versions of uploaded assets, it's recommended to leave this oftion off as it can consume significant storage space over time.", - "admin.storage.versioningNotSupported": "Not supported by this storage target.", "admin.system.browser": "Browser", "admin.system.browserHint": "The browser name and version.", "admin.system.checkForUpdates": "Check", @@ -1440,11 +1370,13 @@ "common.actions.create": "Create", "common.actions.deactivate": "Deactivate", "common.actions.delete": "Delete", + "common.actions.disable": "Disable", "common.actions.discard": "Discard", "common.actions.discardChanges": "Discard Changes", "common.actions.download": "Download", "common.actions.duplicate": "Duplicate", "common.actions.edit": "Edit", + "common.actions.enable": "Enable", "common.actions.exit": "Exit", "common.actions.exitEdit": "Exit Edit", "common.actions.fetch": "Fetch", diff --git a/backend/models/assets.ts b/backend/models/assets.ts index 3ba2b2a63..abed50261 100644 --- a/backend/models/assets.ts +++ b/backend/models/assets.ts @@ -1,12 +1,13 @@ import fs from 'node:fs/promises' import path from 'node:path' import mime from 'mime' -import { and, desc, eq, inArray, sql } from 'drizzle-orm' +import { and, desc, eq, inArray, isNotNull, sql } from 'drizzle-orm' import { assets as assetsTable, tree as treeTable } from '../db/schema.ts' import { CustomError, decodeTreePath, encodeTreePath } from '../helpers/common.ts' import { makeImageThumbnail } from '../helpers/images.ts' import type { Readable } from 'node:stream' import type { DeletedEntry } from './tree.ts' +import type { StorageAssetRef } from './storage.ts' /** How large the file manager renders a preview. Generated once, at upload time. */ const THUMBNAIL_SIZE = { width: 320, height: 200 } @@ -161,20 +162,28 @@ function kindOf(mimeType: string, fileExt: string): AssetKind { /** * Assets model * - * An asset is a file a user uploaded: its bytes live in the `assets` table, while its name and place - * in the site live in the matching `tree` row, which shares its ID. Both are written together — an - * asset with no tree row would be unreachable, and a tree row with no asset would be a broken link. + * An asset is a file a user uploaded: its name and its place in the site live in the `tree` row, its + * metadata in the matching `assets` row, which shares its ID. Both are written together — an asset + * with no tree row would be unreachable, and a tree row with no asset would be a broken link. * - * Storage targets are not implemented yet, so the database is the only copy — but not the one that - * answers a request for a file. Serving goes through two caches, because `/_files/` is hit by every - * image on every page view and neither half of that lookup needs the database twice: + * Where the *bytes* live is a third thing, and not necessarily the database: they go to whichever + * storage target the site has configured for a file of that kind and size, and the row records which + * one took them in `storageInfo`. Nothing here knows what that means for a given target — writing, + * reading, moving and deleting all go through `WIKI.models.storage`, and the database is simply the + * target every site starts with. What never leaves this model is the metadata: renaming a file is a + * database write plus a request to the target to follow it. + * + * Serving goes through two caches, because `/_files/` is hit by every image on every page view and + * neither half of that lookup needs to reach the target twice: * * 1. **memory**, holding path → metadata for `PATH_CACHE_TTL_MS`, which is what decides the ETag and * answers the conditional requests a browser sends once its own copy goes stale * 2. **disk**, under `/cache/files`, holding the bytes, streamed straight to the response * - * Only the database is permanent; both caches are derived and can be deleted at any point, which is - * also what makes a cold instance correct rather than empty-handed. + * Neither cache is storage: both are derived and can be deleted at any point, which is also what + * makes a cold instance correct rather than empty-handed. The one under `/cache/files` is + * not to be confused with the local file system storage target, which is a place content actually + * lives and is never swept. */ class Assets { /** Path resolutions, keyed `siteId:path`. Insertion-ordered, so the oldest entry is evictable. */ @@ -197,6 +206,75 @@ class Assets { return UPLOAD_CONFLICT_BEHAVIORS.has(configured) ? configured : 'overwrite' } + /** + * Refuse an upload that is really a page, or that would land on one. + * + * A page and an asset occupy the same folder and are stored as the same kind of file, so the two + * name spaces are one. Nothing in the tree sees that — a page is `readme` and the file is + * `readme.md`, two different names — so it is enforced here, on the way in, and by + * `guardAgainstAssetCollision` coming the other way. + * + * Two rules, in the order an administrator would expect them: + * + * 1. **The site's `pageExtensions` are reserved.** They are the extensions that address a page by + * URL, so a file with one of them is a page, and uploading it as an attachment is a mistake + * rather than a collision — refused whether or not a page happens to be there today. This is + * what keeps `.md` out of the file manager on a default site. + * 2. **Otherwise, no landing on a page that is there.** For an extension a site has taken off that + * list, the two can legitimately coexist right up until one would overwrite the other's file. + * + * @throws `assetIsPageExtension` or `assetNameTakenByPage` + */ + private async guardAgainstPageCollision({ + siteId, + locale, + folderId, + folderPath, + fileName, + fileExt + }: { + siteId: string + locale: string + folderId?: string | null + folderPath?: string | null + fileName: string + fileExt: string + }): Promise { + const reserved: string[] = WIKI.sites[siteId]?.config?.pageExtensions ?? [] + if (fileExt && reserved.includes(fileExt)) { + throw new CustomError( + 'assetIsPageExtension', + `.${fileExt} is a page extension on this site, so a file with it cannot be uploaded as an attachment. Create a page instead, or remove ${fileExt} from the site's page extensions.`, + 409 + ) + } + + // -> The page this would be the file of, if there is one: same folder, same name without the + // extension. Its own extension has to match too — `readme.pdf` is not the file of the + // markdown page `readme`, and sits happily beside it. + const stem = fileName.slice(0, fileName.length - (fileExt ? fileExt.length + 1 : 0)) + if (!stem) { + return + } + const occupant = await WIKI.models.tree.getEntryAt({ + siteId, + locale, + parentId: folderId, + parentPath: folderPath, + fileName: stem + }) + if ( + occupant?.type === 'page' && + (await WIKI.models.pages.storageFileNameOf(occupant.id)) === fileName + ) { + throw new CustomError( + 'assetNameTakenByPage', + `The page "${stem}" is stored as ${fileName} here, so a file cannot be uploaded under that name.`, + 409 + ) + } + } + /** * Store an uploaded file. * @@ -230,6 +308,7 @@ class Assets { throw new CustomError('assetInvalidFileName', 'This file name cannot be used.') } const fileExt = extensionOf(safeName) + await this.guardAgainstPageCollision({ siteId, locale, folderId, fileName: safeName, fileExt }) // -> The extension decides the type, not the request: the declared one is whatever the client felt // like sending, and this value is what gets served back to a browser later const resolvedMime = mime.getType(safeName) ?? mimeType ?? 'application/octet-stream' @@ -272,6 +351,7 @@ class Assets { return this.replace({ id: occupant.id, siteId, + locale, folderPath: decodeTreePath(occupant.folderPath ?? '') ?? '', fileName: occupant.fileName, title: occupant.title, @@ -300,8 +380,10 @@ class Assets { } }) const storedName = entry.fileName + const folderPath = decodeTreePath(entry.folderPath ?? '') ?? '' try { + // -> The metadata row goes in before the bytes, since the database target writes them into it await WIKI.db.insert(assetsTable).values({ id: entry.id, fileName: storedName, @@ -309,13 +391,25 @@ class Assets { kind, mimeType: resolvedMime, fileSize: data.length, - data, preview, authorId, siteId }) + await WIKI.models.storage.putAsset( + { + id: entry.id, + siteId, + locale, + folderPath, + fileName: storedName, + kind, + fileSize: data.length + }, + data + ) } catch (err) { - // -> Nothing points at the tree row now, and leaving it would show a file the site cannot serve + // -> Nothing points at these now, and leaving them would show a file the site cannot serve + await WIKI.db.delete(assetsTable).where(eq(assetsTable.id, entry.id)) await WIKI.db.delete(treeTable).where(eq(treeTable.id, entry.id)) throw err } @@ -323,7 +417,7 @@ class Assets { WIKI.models.hooks.emit('asset:upload', { id: entry.id, fileName: storedName, - folderPath: decodeTreePath(entry.folderPath ?? '') ?? '', + folderPath, siteId, authorId, metadata: { fileSize: data.length, mimeType: resolvedMime, kind } @@ -336,7 +430,7 @@ class Assets { kind, mimeType: resolvedMime, fileSize: data.length, - folderPath: decodeTreePath(entry.folderPath ?? '') ?? '', + folderPath, title: entry.title, hasPreview: Boolean(preview), createdAt: entry.createdAt, @@ -355,10 +449,15 @@ class Assets { * The name it keeps is the stored one, which is why the extension and type are the incoming file's: * the two only differ when a browser sent `Photo.PNG` for what is stored as `photo.png`, and the * sanitized name is what both agree on. + * + * Which targets hold it are worked out again from the incoming file rather than inherited, since + * the two may not be the same size — replacing a thumbnail with a 40 MB one is how a file crosses + * the large-file threshold and starts being stored somewhere else entirely. */ private async replace({ id, siteId, + locale, folderPath, fileName, title, @@ -371,6 +470,7 @@ class Assets { }: { id: string siteId: string + locale: string folderPath: string fileName: string title: string @@ -381,6 +481,10 @@ class Assets { preview: Buffer | null authorId: string }): Promise { + await WIKI.models.storage.putAsset( + { id, siteId, locale, folderPath, fileName, kind, fileSize: data.length }, + data + ) await WIKI.db .update(assetsTable) .set({ @@ -388,7 +492,6 @@ class Assets { kind, mimeType, fileSize: data.length, - data, preview, authorId, updatedAt: sql`now()` @@ -536,27 +639,297 @@ class Assets { * An asset's bytes, along with what to serve them as. Null if there is no such asset. * * Not scoped to a site, unlike the rest: the ID is a UUID nobody can guess, and the routes that use - * this are the public ones, which have no site of their own to check against. + * this are the public ones, which have no site of their own to check against. Where the file sits + * is read off the tree, since that — not a record of where it was put — is how every target + * addresses its copy. */ async getContent( id: string ): Promise<{ data: Buffer; mimeType: string; fileName: string } | null> { const results = await WIKI.db .select({ - data: assetsTable.data, + id: assetsTable.id, + siteId: assetsTable.siteId, + kind: assetsTable.kind, + fileSize: assetsTable.fileSize, mimeType: assetsTable.mimeType, - fileName: assetsTable.fileName + fileName: assetsTable.fileName, + locale: treeTable.locale, + folderPath: treeTable.folderPath }) .from(assetsTable) + .innerJoin(treeTable, eq(treeTable.id, assetsTable.id)) .where(eq(assetsTable.id, id)) .limit(1) const row = results[0] - return row?.data ? { data: row.data, mimeType: row.mimeType, fileName: row.fileName } : null + if (!row) { + return null + } + const data = await WIKI.models.storage.getAsset({ + id: row.id, + siteId: row.siteId, + locale: row.locale, + folderPath: decodeTreePath(row.folderPath ?? '') ?? '', + fileName: row.fileName, + kind: row.kind, + fileSize: row.fileSize ?? 0 + }) + return data ? { data, mimeType: row.mimeType, fileName: row.fileName } : null + } + + // == STORAGE ======================== + + /** + * Where each of these assets sits, as a storage target addresses one. + */ + async getStorageRefs(siteId: string, ids: string[]): Promise { + if (ids.length < 1) { + return [] + } + const rows = await WIKI.db + .select({ + id: assetsTable.id, + kind: assetsTable.kind, + fileSize: assetsTable.fileSize, + locale: treeTable.locale, + folderPath: treeTable.folderPath, + fileName: treeTable.fileName + }) + .from(assetsTable) + .innerJoin(treeTable, eq(treeTable.id, assetsTable.id)) + .where(and(eq(assetsTable.siteId, siteId), inArray(assetsTable.id, ids))) + + return rows.map((row) => ({ + id: row.id, + siteId, + locale: row.locale, + folderPath: decodeTreePath(row.folderPath ?? '') ?? '', + fileName: row.fileName, + kind: row.kind, + fileSize: row.fileSize ?? 0 + })) + } + + /** + * Every asset of a site, addressed the way a storage target addresses one. + * + * What a target's export action walks in order to find the files it should be holding and is not. + * Metadata only — the bytes of each are fetched one at a time, since the point of moving them off + * the database is that they do not all fit in memory at once. + * + * @param withDatabaseCopy Only the assets whose bytes are in their own row, which is what the + * database target holds. For the offload action, which has nothing to move for the rest. + */ + async listStoredAssets( + siteId: string, + { withDatabaseCopy }: { withDatabaseCopy?: boolean } = {} + ): Promise { + const rows = await WIKI.db + .select({ + id: assetsTable.id, + kind: assetsTable.kind, + fileSize: assetsTable.fileSize, + locale: treeTable.locale, + folderPath: treeTable.folderPath, + fileName: treeTable.fileName + }) + .from(assetsTable) + .innerJoin(treeTable, eq(treeTable.id, assetsTable.id)) + .where( + withDatabaseCopy + ? and(eq(assetsTable.siteId, siteId), isNotNull(assetsTable.data)) + : eq(assetsTable.siteId, siteId) + ) + + return rows.map((row) => ({ + id: row.id, + siteId, + locale: row.locale, + folderPath: decodeTreePath(row.folderPath ?? '') ?? '', + fileName: row.fileName, + kind: row.kind, + fileSize: row.fileSize ?? 0 + })) + } + + /** + * Ask the target holding each of these assets to follow where the tree has since put them. + * + * Called after a rename, the tree rows already being correct: `getStorageRefs` reads the + * destination off them, and the caller says where each one came from. Every target holding the + * asset moves its own copy. + */ + async relocateAssets( + siteId: string, + moves: { id: string; previous: { locale: string; folderPath: string; fileName: string } }[] + ): Promise { + const refs = await this.getStorageRefs( + siteId, + moves.map((move) => move.id) + ) + for (const ref of refs) { + const previous = moves.find((move) => move.id === ref.id)?.previous + if (previous) { + await WIKI.models.storage.relocateAsset(ref, previous) + } + } + } + + /** + * Take a file a storage target already holds into the wiki, without writing it anywhere. + * + * The other direction from an upload: the bytes are already in place — restored from a backup, + * dropped into the folder by another tool — and what is missing is the wiki's record of them. A file + * the wiki has no entry for is adopted where it lies rather than written out again, which is why + * nothing is dispatched to the storage layer for it. Any *other* target configured to hold that kind + * will not have a copy until its own export action runs. + * + * `overwrite` turns the case the wiki DOES have an entry for from a skip into a replacement, for a + * restore where the folder is meant to be the authority. That one is dispatched, and has to be: the + * wiki's copy of those bytes may be what a reader is served — from the database target, typically — + * so leaving the other targets on the old file would make the import appear to have done nothing. + * There is no history behind an asset, so unlike an overwritten page the bytes it replaces are gone. + * + * Whatever `overwrite` says, only an *asset* is ever replaced. A page or a folder owning the name is + * left alone: a page's own file belongs to the other half of the import, and neither is something a + * loose file in a folder may take over. + * + * @param overwrite Replace an asset already at this path instead of leaving it alone + * @returns The asset, or null for a file this passed over + */ + async adoptStoredFile({ + siteId, + locale, + folderPath, + fileName, + data, + authorId, + overwrite + }: { + siteId: string + locale: string + folderPath: string + fileName: string + data: Buffer + authorId: string + overwrite?: boolean + }): Promise { + const safeName = sanitizeFileName(fileName) + if (!safeName) { + return null + } + + // -> Read in full rather than as an existence check, since replacing one needs its ID and the + // name and title it is already filed under + const occupant = await WIKI.models.tree.getEntryAt({ + siteId, + locale, + parentPath: folderPath, + fileName: safeName + }) + if (occupant && (!overwrite || occupant.type !== 'asset')) { + return null + } + + const fileExt = extensionOf(safeName) + try { + await this.guardAgainstPageCollision({ + siteId, + locale, + folderPath, + fileName: safeName, + fileExt + }) + } catch { + // -> Skipped rather than reported, as everything else this passes over is: the caller is + // walking a folder, and a file that is really a page belongs to the other half of the import + return null + } + const mimeType = mime.getType(safeName) ?? 'application/octet-stream' + const kind = kindOf(mimeType, fileExt) + const preview = + kind === 'image' + ? await makeImageThumbnail(data, THUMBNAIL_SIZE.width, THUMBNAIL_SIZE.height) + : null + + if (occupant) { + return this.replace({ + id: occupant.id, + siteId, + locale, + folderPath: decodeTreePath(occupant.folderPath ?? '') ?? '', + // -> The names it is already filed under, not the ones off the file: they only differ by + // sanitization, and the entry is the authority on what it is actually called + fileName: occupant.fileName, + title: occupant.title, + fileExt, + kind, + mimeType, + data, + preview, + authorId + }) + } + + const entry = await WIKI.models.tree.addAsset({ + parentPath: folderPath, + fileName: safeName, + title: safeName, + locale, + siteId, + meta: { fileSize: data.length, fileExt, mimeType } + }) + + try { + await WIKI.db.insert(assetsTable).values({ + id: entry.id, + fileName: entry.fileName, + fileExt, + kind, + mimeType, + fileSize: data.length, + preview, + authorId, + siteId + }) + } catch (err) { + await WIKI.db.delete(treeTable).where(eq(treeTable.id, entry.id)) + throw err + } + + const importedFolderPath = decodeTreePath(entry.folderPath ?? '') ?? '' + WIKI.models.hooks.emit('asset:upload', { + id: entry.id, + fileName: entry.fileName, + folderPath: importedFolderPath, + siteId, + authorId, + metadata: { fileSize: data.length, mimeType, kind } + }) + + return { + id: entry.id, + fileName: entry.fileName, + fileExt, + kind, + mimeType, + fileSize: data.length, + folderPath: importedFolderPath, + title: entry.title, + hasPreview: Boolean(preview), + createdAt: entry.createdAt, + updatedAt: entry.updatedAt + } } /** * An asset's thumbnail, or null when it has none — which is the normal state for anything that is * not an image, and for images uploaded while Sharp was unavailable. + * + * Always read from the database, whichever target holds the file itself. A preview is a few + * kilobytes generated by this model rather than anything a user uploaded, and the file manager asks + * for a screenful of them at a time — there is nothing to gain by sending that around a storage + * module, and a target being slow or misconfigured would cost a wiki its whole file browser. */ async getThumbnail(id: string): Promise { const results = await WIKI.db @@ -837,6 +1210,18 @@ class Assets { if (!fileExt) { throw new CustomError('assetInvalidFileName', 'The file name must keep a file extension.') } + // -> The same two rules an upload is held to: renaming is another way of arriving at a name, and + // `readme.pdf` renamed to `readme.md` would land on the page of that name just as squarely + const entry = await WIKI.models.tree.getById(id) + if (entry) { + await this.guardAgainstPageCollision({ + siteId, + locale: entry.locale, + folderPath: decodeTreePath(entry.folderPath ?? '') ?? '', + fileName: safeName, + fileExt + }) + } const resolvedMime = mime.getType(safeName) ?? asset.mimeType await WIKI.models.tree.renameEntry({ id, fileName: safeName, title: safeName }) @@ -856,6 +1241,21 @@ class Assets { .set({ meta: { fileSize: asset.fileSize, fileExt, mimeType: resolvedMime } }) .where(eq(treeTable.id, id)) + // -> Every target holding this asset lays its copy out by path, so each of them has a file to + // move now that the tree rows have been rewritten + if (entry) { + await this.relocateAssets(siteId, [ + { + id, + previous: { + locale: entry.locale, + folderPath: asset.folderPath, + fileName: asset.fileName + } + } + ]) + } + // -> Both ends of the move: the name it left, and the name it took, which something else may have // been resolved at before it was freed up this.forgetPath(siteId, asset.folderPath, asset.fileName) @@ -883,8 +1283,14 @@ class Assets { if (!asset) { return false } + // -> Read before the rows go: where an asset sits is the tree's to say, and the tree row is + // about to be deleted along with it + const [ref] = await this.getStorageRefs(siteId, [id]) await WIKI.db.delete(assetsTable).where(eq(assetsTable.id, id)) await WIKI.models.tree.deleteEntry(id) + if (ref) { + await WIKI.models.storage.removeAsset(ref) + } this.forgetPath(siteId, asset.folderPath, asset.fileName) await this.dropCachedContent([id]) @@ -907,8 +1313,37 @@ class Assets { return } const ids = entries.map((entry) => entry.id) + /* + Where each asset sat is rebuilt from what the folder deletion reported, not looked up: the tree + rows went with the folder, and they were the only thing that placed these assets. What is still + here is the `assets` row, which is where the kind and size come from — a target needs both to + work out whether it was holding the file at all. + */ + const stored = new Map( + ( + await WIKI.db + .select({ id: assetsTable.id, kind: assetsTable.kind, fileSize: assetsTable.fileSize }) + .from(assetsTable) + .where(inArray(assetsTable.id, ids)) + ).map((row) => [row.id, row]) + ) await WIKI.db.delete(assetsTable).where(inArray(assetsTable.id, ids)) + for (const entry of entries) { + const row = stored.get(entry.id) + if (row) { + await WIKI.models.storage.removeAsset({ + id: entry.id, + siteId, + locale: entry.locale, + folderPath: entry.folderPath, + fileName: entry.fileName, + kind: row.kind, + fileSize: row.fileSize ?? 0 + }) + } + } + // -> Which paths they sat at is no longer knowable from the tree: those rows went with the folder this.forgetAllPaths() await this.dropCachedContent(ids) diff --git a/backend/models/pages.ts b/backend/models/pages.ts index d03def58b..b20cee19f 100644 --- a/backend/models/pages.ts +++ b/backend/models/pages.ts @@ -8,6 +8,7 @@ import { } from '../helpers/common.ts' import type { RenderPermissions, TocNode } from './rendering.ts' import type { DeletedEntry } from './tree.ts' +import type { StoragePageContent, StoragePageRef } from './storage.ts' /** What each editor produces, which is what the content column holds. */ const EDITOR_CONTENT_TYPES: Record = { @@ -17,6 +18,45 @@ const EDITOR_CONTENT_TYPES: Record = { redirect: 'redirect' } +/** + * The extension a page's source is written under, by the content type its editor produces. + * + * A page is addressed without one — its path is a URL — so this only ever appears where a page has to + * be a file: a storage target laying content out by path. It lives here rather than in that module + * because it is a fact about the page, and because it is what decides whether an uploaded file would + * land on top of one. See `storageFileName`. + */ +export const PAGE_FILE_EXTENSIONS: Record = { + markdown: 'md', + html: 'html', + asciidoc: 'adoc', + redirect: 'json' +} + +/** For a content type added since this was written. */ +const DEFAULT_PAGE_FILE_EXTENSION = 'txt' + +export function pageFileExtension(contentType: string): string { + return PAGE_FILE_EXTENSIONS[contentType] ?? DEFAULT_PAGE_FILE_EXTENSION +} + +/** + * Which editor writes a given file extension. + * + * For a page being imported that did not say which editor it belongs to, which is allowed only where + * the site reserves the extension for pages — see `importAll` in the local disk module. + * + * @returns Null when no editor produces that extension, which for a reserved one means the site + * reserved something this wiki has no editor for + */ +export function pageEditorForExtension(ext: string): string | null { + const contentType = Object.entries(PAGE_FILE_EXTENSIONS).find(([, e]) => e === ext)?.[0] + if (!contentType) { + return null + } + return Object.entries(EDITOR_CONTENT_TYPES).find(([, ct]) => ct === contentType)?.[0] ?? null +} + /** * The editor whose pages send their reader somewhere else. * @@ -486,6 +526,15 @@ class Pages { throw new CustomError('pageDuplicatePath', 'A page already exists at this path.', 409) } + const pathParts = path.split('/') + await this.guardAgainstAssetCollision({ + siteId, + locale, + parentPath: pathParts.slice(0, -1).join('/'), + fileName: pathParts.at(-1)!, + contentType: EDITOR_CONTENT_TYPES[editor] ?? 'text' + }) + const alias = await this.validateAlias(siteId, input.alias) const { render, toc, text } = await WIKI.models.rendering.postProcess( siteId, @@ -496,7 +545,6 @@ class Pages { } ) - const pathParts = path.split('/') const inserted = await WIKI.db .insert(pagesTable) .values({ @@ -560,6 +608,9 @@ class Pages { reason: input.reasonForChange }) + const stored = this.toStoragePage(siteId, page, page.content ?? '') + await WIKI.models.storage.mirrorPage(stored.ref, stored.content) + await WIKI.models.search.indexPage(page.id, locale) await WIKI.models.hooks.emit('page:create', { id: page.id, @@ -708,6 +759,11 @@ class Pages { .where(eq(treeTable.id, id)) } + // -> The source is whatever this save set it to, else whatever it already was: a save that only + // changed the title still rewrites the copy, since the title is in its front matter + const stored = this.toStoragePage(siteId, updated, values.content ?? existing.content ?? '') + await WIKI.models.storage.mirrorPage(stored.ref, stored.content) + await WIKI.models.search.indexPage(id, updated.locale) await WIKI.models.hooks.emit('page:edit', { id, @@ -730,10 +786,13 @@ class Pages { { path, title }: { path: string; title?: string }, actor: PageActor ): Promise { - const page = await this.getPage({ siteId, id }) + // -> With the source, which the move itself does not need: it is what the copy kept by a storage + // target is rewritten from once the page has landed at its new path + const page = await this.getPage({ siteId, id, withContent: true }) if (!page) { return null } + const existingContent = page.content const newPath = normalizePath(path) if (newPath === page.path && (title === undefined || title === page.title)) { return page @@ -755,6 +814,13 @@ class Pages { if (duplicate.length > 0) { throw new CustomError('pageDuplicatePath', 'A page already exists at this path.', 409) } + await this.guardAgainstAssetCollision({ + siteId, + locale: page.locale, + parentPath: newPath.split('/').slice(0, -1).join('/'), + fileName: newPath.split('/').at(-1)!, + contentType: page.contentType + }) } await WIKI.db @@ -798,6 +864,13 @@ class Pages { ] }) + // -> Moved and then rewritten, rather than deleted and written afresh: the move is what keeps a + // versioned target's history of the file attached to it, and the rewrite is because a move may + // carry a new title and always carries a new modification time, both of which are in the copy + const stored = this.toStoragePage(siteId, moved, existingContent ?? '') + await WIKI.models.storage.relocatePage(stored.ref, page.path) + await WIKI.models.storage.mirrorPage(stored.ref, stored.content) + await WIKI.models.hooks.emit('page:rename', { id, path: moved.path, @@ -832,6 +905,13 @@ class Pages { // -> A page that overrode the sidebar owns a menu keyed by its own id, which nothing could reach // once the page is gone await WIKI.models.navigation.deleteNavForEntries([id]) + await WIKI.models.storage.removePage({ + id, + siteId, + locale: page.locale, + path: page.path, + contentType: page.contentType + }) await WIKI.models.hooks.emit('page:delete', { id, @@ -867,6 +947,21 @@ class Pages { authorId: actor.id }) } + // -> Read before the rows go: what a target filed each page under is decided by its content type, + // and guessing would mean reaching for names that may belong to the assets beside them + const contentTypes = new Map( + ( + await WIKI.db + .select({ id: pagesTable.id, contentType: pagesTable.contentType }) + .from(pagesTable) + .where( + inArray( + pagesTable.id, + entries.map((entry) => entry.id) + ) + ) + ).map((row) => [row.id, row.contentType]) + ) await WIKI.db.delete(pagesTable).where( inArray( pagesTable.id, @@ -877,9 +972,20 @@ class Pages { // -> One per page, as deleting them one at a time would have sent: a subscriber mirroring the // wiki has to hear about each page, not about the folder it happened to sit in for (const entry of entries) { + const path = entry.folderPath ? `${entry.folderPath}/${entry.fileName}` : entry.fileName + const contentType = contentTypes.get(entry.id) + if (contentType) { + await WIKI.models.storage.removePage({ + id: entry.id, + siteId, + locale: entry.locale, + path, + contentType + }) + } await WIKI.models.hooks.emit('page:delete', { id: entry.id, - path: entry.folderPath ? `${entry.folderPath}/${entry.fileName}` : entry.fileName, + path, locale: entry.locale, siteId, authorId: actor.id @@ -888,6 +994,315 @@ class Pages { WIKI.logger.debug(`Deleted ${entries.length} page(s) that went with a deleted folder.`) } + // == STORAGE ======================== + + /** + * The file name a page occupies on a target that lays content out by path. + * + * The name a page and an asset can collide on, and so the thing both of them are checked against + * before either is written — see `guardAgainstAssetCollision` here and its opposite number in the + * assets model. + */ + storageFileName(fileName: string, contentType: string): string { + return `${fileName}.${pageFileExtension(contentType)}` + } + + /** + * The file name an existing page occupies, or null if there is no such page. + */ + async storageFileNameOf(id: string): Promise { + const results = await WIKI.db + .select({ path: pagesTable.path, contentType: pagesTable.contentType }) + .from(pagesTable) + .where(eq(pagesTable.id, id)) + .limit(1) + const row = results[0] + return row ? this.storageFileName(row.path.split('/').at(-1)!, row.contentType) : null + } + + /** + * Refuse a page whose stored file would land on an asset that is already there. + * + * The page and the asset have different names as far as the tree is concerned — a page is `readme` + * and the asset is `readme.md` — so nothing in the tree stops the two coexisting. They only meet + * once the page has to be a file, and by then one of them would be overwriting the other. + * + * Normally unreachable, because the site's `pageExtensions` keeps assets off these extensions in + * the first place. It is the backstop for a site that has removed one from that list, and for + * content that predates its being on it. + * + * @throws `pageNameTakenByAsset` when the name is not free + */ + private async guardAgainstAssetCollision({ + siteId, + locale, + parentPath, + fileName, + contentType + }: { + siteId: string + locale: string + parentPath: string + fileName: string + contentType: string + }): Promise { + const storedName = this.storageFileName(fileName, contentType) + const occupant = await WIKI.models.tree.getEntryAt({ + siteId, + locale, + parentPath, + fileName: storedName + }) + if (occupant?.type === 'asset') { + throw new CustomError( + 'pageNameTakenByAsset', + `A file named ${storedName} already exists here, which is where this page would be stored.`, + 409 + ) + } + } + + /* + A page always lives in its own row, and none of what follows changes that. What it does is keep a + copy of the page on every storage target configured to hold `pages` — the local disk, today — + which is a backup of content the database owns rather than a place it has moved to. Nothing here + is ever read back: `getPage` goes to the row, as it always has. + + That is why every one of these calls is fired after the database write has succeeded and none of + them is allowed to fail the operation; `WIKI.models.storage` swallows and logs a target that could + not keep up. A wiki whose backup disk filled up is a wiki with a stale backup, not one that has + stopped accepting edits. + */ + + /** + * A page in the shape a storage target takes it. + * + * @param content The source, which the caller has to hand: it is not on the `Page` a mutation + * returns unless the read asked for it, and re-reading the row to get it back would cost a query + * per save for something the caller just wrote. + */ + private toStoragePage( + siteId: string, + page: { + id: string + locale: string + path: string + title: string + description?: string | null + editor: string + contentType: string + tags?: string[] + publishState: string + createdAt: Date + updatedAt: Date + }, + content: string + ): { ref: StoragePageRef; content: StoragePageContent } { + return { + ref: { + id: page.id, + siteId, + locale: page.locale, + path: page.path, + contentType: page.contentType + }, + content: { + title: page.title, + description: page.description ?? '', + editor: page.editor, + tags: page.tags ?? [], + // -> A scheduled page is not published yet, whatever its dates say it will be + isPublished: page.publishState === 'published', + createdAt: page.createdAt, + updatedAt: page.updatedAt, + content + } + } + } + + /** + * Every page of a site, in the shape a storage target takes them. + * + * What a target's `dump` action walks in order to write copies of content that predates it being + * enabled. Reads the source of every page at once, which is what makes this a maintenance action + * rather than something to call on a request. + */ + async listForStorage( + siteId: string + ): Promise<{ ref: StoragePageRef; content: StoragePageContent }[]> { + const rows = await WIKI.db + .select({ + id: pagesTable.id, + locale: pagesTable.locale, + path: pagesTable.path, + title: pagesTable.title, + description: pagesTable.description, + editor: pagesTable.editor, + contentType: pagesTable.contentType, + tags: pagesTable.tags, + publishState: pagesTable.publishState, + createdAt: pagesTable.createdAt, + updatedAt: pagesTable.updatedAt, + content: pagesTable.content + }) + .from(pagesTable) + .where(eq(pagesTable.siteId, siteId)) + + return rows.map((row) => this.toStoragePage(siteId, row, row.content ?? '')) + } + + /** + * Take a page a storage target holds and the wiki does not into the database. + * + * The direction that makes a target a peer store rather than a write-only backup: pages arrive here + * from a folder restored onto the disk, and — once a module exists that can hear about them — from + * commits somebody else pushed. What comes back is an ordinary page, because that is the only kind + * there is: it lands in `pages`, gets a tree entry and is served from the row like every other. + * + * A path the wiki already has a page at is left alone rather than overwritten, unless `overwrite` + * says the file is to win. Off, this is the safe direction: the wiki's copy is the one an author has + * been editing, and reconciling a file that changed on both sides is a merge, which is a target's + * business and not this model's. On, the file is taken as the authority — for a restore, where what + * is in the folder is what the wiki is supposed to say. + * + * An overwrite is an ordinary save, not a special path: it records a version like any other, so the + * copy it replaced is in the page's history and an administrator who did not mean it can put it + * back. Two things it does not take from the file, both because a save anywhere else in this model + * does not either — the page's **editor**, so a `.md` file cannot turn a redirection into markdown + * by landing on it, and its **path**, since that is what identified it in the first place. + * + * @param overwrite Replace a page already at this path instead of leaving it alone + * @returns The imported page, or null when there is already one at that path and `overwrite` is not + * set + */ + async adoptStoredPage({ + siteId, + locale, + path, + title, + description, + editor, + tags, + isPublished, + content, + createdAt, + updatedAt, + authorId, + overwrite + }: { + siteId: string + locale: string + path: string + title: string + description?: string + editor: string + tags?: string[] + isPublished?: boolean + content: string + createdAt?: Date + updatedAt?: Date + authorId: string + overwrite?: boolean + }): Promise { + const normalized = normalizePath(path) + const existing = await WIKI.db + .select({ id: pagesTable.id }) + .from(pagesTable) + .where( + and( + eq(pagesTable.siteId, siteId), + eq(pagesTable.locale, locale), + eq(pagesTable.path, normalized) + ) + ) + .limit(1) + if (existing.length > 0 && !overwrite) { + return null + } + + /* + Imported content is rendered with NO script or style permission, whoever ran the import. + + What those two allow is a ` diff --git a/frontend/src/css/app.scss b/frontend/src/css/app.scss index 36a8a4f10..b466d698b 100644 --- a/frontend/src/css/app.scss +++ b/frontend/src/css/app.scss @@ -2,4 +2,3 @@ @use 'animation'; @use 'page-contents'; -@import 'v-network-graph/lib/style.css'; diff --git a/frontend/src/pages/AdminStorage.vue b/frontend/src/pages/AdminStorage.vue index 2d6763c8d..3cfc368e0 100644 --- a/frontend/src/pages/AdminStorage.vue +++ b/frontend/src/pages/AdminStorage.vue @@ -23,7 +23,8 @@ :color="dark.isActive ? `dark-1` : `white`" :options="[ { label: t('admin.storage.targets'), value: 'targets' }, - { label: t('admin.storage.deliveryPaths'), value: 'delivery' } + { label: t('admin.storage.delivery'), value: 'delivery' }, + { label: t('admin.storage.config'), value: 'config' } ]" />
- - - - - - {{ t('admin.storage.setup') }} - - - - - - - - {{ t('admin.storage.setup') }} - - - - - - Uninstall - Delete the active configuration and start over the setup process. - - This action cannot be undone! - - - - - - - @@ -243,7 +94,7 @@ {{ t('admin.storage.contentTypes') }} - + {{ t(`admin.storage.contentTypePages`) }} {{ t(`admin.storage.contentTypePagesHint`) }} - - - - - - - - {{ t(`admin.storage.contentTypeImages`) }} - {{ - t(`admin.storage.contentTypeImagesHint`) - }} - - - - - - - - {{ t(`admin.storage.contentTypeDocuments`) }} - {{ - t(`admin.storage.contentTypeDocumentsHint`) - }} - - - - - - - - {{ t(`admin.storage.contentTypeOthers`) }} - {{ - t(`admin.storage.contentTypeOthersHint`) - }} - - - - - - - - {{ t(`admin.storage.contentTypeLargeFiles`) }} - {{ - t(`admin.storage.contentTypeLargeFilesHint`) - }} + {{ t(`admin.storage.contentTypeLargeFilesDBWarn`) }} - - - - - - - - - - - - {{ t('admin.storage.assetDelivery') }} - - - - - - - - {{ t(`admin.storage.assetStreaming`) }} - {{ t(`admin.storage.assetStreamingHint`) }} - {{ t(`admin.storage.assetStreamingNotSupported`) }} - - - - - - - - {{ t(`admin.storage.assetDirectAccess`) }} - {{ - t(`admin.storage.assetDirectAccessHint`) - }} - {{ t(`admin.storage.assetDirectAccessNotSupported`) }}{{ t(`admin.storage.contentTypePagesSource`) }} + {{ t('admin.storage.config') }} - - {{ t('admin.storage.noConfigOption') }} + + +
{{ t('admin.storage.noConfigOption') }}