Skip to content

Commit d0efb05

Browse files
JPPhotoJPPhoto
authored
Merge branch 'main' into dype-paper
2 parents 48553a9 + 1b50c1a commit d0efb05

78 files changed

Lines changed: 1746 additions & 191 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎docs/contributing/CANVAS_PROJECTS/CANVAS_PROJECTS.md‎

Lines changed: 205 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,205 @@
1+
# Canvas Projects — Technical Documentation
2+
3+
## Overview
4+
5+
Canvas Projects provide a save/load mechanism for the entire canvas state. The feature serializes all canvas entities, generation parameters, reference images, and their associated image files into a ZIP-based `.invk` file. On load, it restores the full state, handling image deduplication and re-uploading as needed.
6+
7+
## File Format
8+
9+
The `.invk` file is a standard ZIP archive with the following structure:
10+
11+
```
12+
project.invk
13+
├── manifest.json
14+
├── canvas_state.json
15+
├── params.json
16+
├── ref_images.json
17+
├── loras.json
18+
└── images/
19+
├── {image_name_1}.png
20+
├── {image_name_2}.png
21+
└── ...
22+
```
23+
24+
### manifest.json
25+
26+
Schema version and metadata. Validated on load with Zod.
27+
28+
```json
29+
{
30+
"version": 1,
31+
"appVersion": "5.12.0",
32+
"createdAt": "2026-02-26T12:00:00.000Z",
33+
"name": "My Canvas Project"
34+
}
35+
```
36+
37+
| Field | Type | Description |
38+
|---|---|---|
39+
| `version` | `number` | Schema version, currently `1`. Used for migration logic on load. |
40+
| `appVersion` | `string` | InvokeAI version that created the file. Informational only. |
41+
| `createdAt` | `string` | ISO 8601 timestamp. |
42+
| `name` | `string` | User-provided project name. Also used as the download filename. |
43+
44+
### canvas_state.json
45+
46+
The serialized canvas entity tree. Type: `CanvasProjectState`.
47+
48+
```typescript
49+
type CanvasProjectState = {
50+
rasterLayers: CanvasRasterLayerState[];
51+
controlLayers: CanvasControlLayerState[];
52+
inpaintMasks: CanvasInpaintMaskState[];
53+
regionalGuidance: CanvasRegionalGuidanceState[];
54+
bbox: CanvasState['bbox'];
55+
selectedEntityIdentifier: CanvasState['selectedEntityIdentifier'];
56+
bookmarkedEntityIdentifier: CanvasState['bookmarkedEntityIdentifier'];
57+
};
58+
```
59+
60+
Each entity contains its full state including all canvas objects (brush lines, eraser lines, rect shapes, images). Image objects reference files by `image_name` which correspond to files in the `images/` folder.
61+
62+
### params.json
63+
64+
The complete generation parameters state (`ParamsState`). Optional on load (older files may not have it). This includes all fields from the params Redux slice:
65+
66+
- Prompts (positive, negative, prompt history)
67+
- Core generation settings (seed, steps, CFG scale, guidance, scheduler, iterations)
68+
- Model selections (main model, VAE, FLUX VAE, T5 encoder, CLIP embed models, refiner, Z-Image models, Klein models)
69+
- Dimensions (width, height, aspect ratio)
70+
- Img2img strength
71+
- Infill settings (method, tile size, patchmatch downscale, color)
72+
- Canvas coherence settings (mode, edge size, min denoise)
73+
- Refiner parameters (steps, CFG scale, scheduler, aesthetic scores, start)
74+
- FLUX-specific settings (scheduler, DyPE preset/scale/exponent)
75+
- Z-Image-specific settings (scheduler, seed variance)
76+
- Upscale settings (scheduler, CFG scale)
77+
- Seamless tiling, mask blur, CLIP skip, VAE precision, CPU noise, color compensation
78+
79+
### ref_images.json
80+
81+
Global reference image entities (`RefImageState[]`). These are IP-Adapter / FLUX Redux configs with `CroppableImageWithDims` containing both original and cropped image references. Optional on load.
82+
83+
### loras.json
84+
85+
Array of LoRA configurations (`LoRA[]`). Each entry contains:
86+
87+
```typescript
88+
type LoRA = {
89+
id: string;
90+
isEnabled: boolean;
91+
model: ModelIdentifierField;
92+
weight: number;
93+
};
94+
```
95+
96+
Optional on load. Like models, LoRA identifiers are stored as-is — if a LoRA is not installed when loading, the entry is restored but may not be usable.
97+
98+
### images/
99+
100+
All image files referenced anywhere in the state. Keyed by their original `image_name`. On save, each image is fetched from the backend via `GET /api/v1/images/i/{name}/full` and stored as-is.
101+
102+
## Key Source Files
103+
104+
| File | Purpose |
105+
|---|---|
106+
| `features/controlLayers/util/canvasProjectFile.ts` | Types, constants, image name collection, remapping, existence checking |
107+
| `features/controlLayers/hooks/useCanvasProjectSave.ts` | Save hook — collects Redux state, fetches images, builds ZIP |
108+
| `features/controlLayers/hooks/useCanvasProjectLoad.ts` | Load hook — parses ZIP, deduplicates images, dispatches state |
109+
| `features/controlLayers/components/SaveCanvasProjectDialog.tsx` | Save name dialog + `useSaveCanvasProjectWithDialog` hook |
110+
| `features/controlLayers/components/LoadCanvasProjectConfirmationAlertDialog.tsx` | Load confirmation dialog + `useLoadCanvasProjectWithDialog` hook |
111+
| `features/controlLayers/components/Toolbar/CanvasToolbarProjectMenuButton.tsx` | Toolbar dropdown UI |
112+
| `features/controlLayers/store/canvasSlice.ts` | `canvasProjectRecalled` Redux action |
113+
114+
## Save Flow
115+
116+
1. User clicks "Save Canvas Project" → `SaveCanvasProjectDialog` opens asking for a project name
117+
2. On confirm, `saveCanvasProject(name)` is called
118+
3. Read Redux state via selectors: `selectCanvasSlice()`, `selectParamsSlice()`, `selectRefImagesSlice()`, `selectLoRAsSlice()`
119+
4. Build `CanvasProjectState` from the canvas slice; use `paramsState` directly for params
120+
5. Walk all entities to collect every `image_name` reference via `collectImageNames()`:
121+
- `CanvasImageState.image.image_name` in layer/mask objects
122+
- `CroppableImageWithDims.original.image.image_name` in global ref images
123+
- `CroppableImageWithDims.crop.image.image_name` in cropped ref images
124+
- `ImageWithDims.image_name` in regional guidance ref images
125+
6. Fetch each image from the backend API
126+
7. Build ZIP with JSZip: add `manifest.json` (including `name`), `canvas_state.json`, `params.json`, `ref_images.json`, and all images into `images/`
127+
8. Sanitize the name for filesystem use and generate blob, trigger download as `{name}.invk`
128+
129+
## Load Flow
130+
131+
1. User selects `.invk` file → confirmation dialog opens
132+
2. On confirm, parse ZIP with JSZip
133+
3. Validate manifest version via Zod schema
134+
4. Read `canvas_state.json`, `params.json` (optional), `ref_images.json` (optional)
135+
5. Collect all `image_name` references from the loaded state
136+
6. **Deduplicate images**: for each referenced image, check if it exists on the server via `getImageDTOSafe(image_name)`
137+
- Already exists → skip (no upload)
138+
- Missing → upload from ZIP via `uploadImage()`, record `oldName → newName` mapping
139+
7. Remap all `image_name` values in the loaded state using the mapping (only for re-uploaded images whose names changed)
140+
8. Dispatch Redux actions:
141+
- `canvasProjectRecalled()` — restores all canvas entities, bbox, selected/bookmarked entity
142+
- `refImagesRecalled()` — restores global reference images
143+
- `paramsRecalled()` — replaces the entire params state in one action
144+
- `loraAllDeleted()` + `loraRecalled()` — restores LoRAs
145+
9. Show success/error toast
146+
147+
## Image Name Collection & Remapping
148+
149+
The `canvasProjectFile.ts` utility provides two parallel sets of functions:
150+
151+
**Collection** (`collectImageNames`): Walks the entire state tree and returns a `Set<string>` of all referenced `image_name` values. This is used by both save (to know which images to fetch) and load (to know which images to check/upload).
152+
153+
**Remapping** (`remapCanvasState`, `remapRefImages`): Deep-clones state objects and replaces `image_name` values using a `Map<string, string>` mapping. Only images that were re-uploaded with a different name are remapped. Images that already existed on the server are left unchanged.
154+
155+
Both walk the same paths through the state tree:
156+
- Layer/mask objects → `CanvasImageState.image.image_name`
157+
- Regional guidance ref images → `ImageWithDims.image_name`
158+
- Global ref images → `CroppableImageWithDims.original.image.image_name` and `.crop.image.image_name`
159+
160+
## Extending the Format
161+
162+
### Adding new optional data (non-breaking)
163+
164+
Add a new JSON file to the ZIP. No version bump needed.
165+
166+
1. **Save**: Add `zip.file('new_data.json', JSON.stringify(data))` in `useCanvasProjectSave.ts`
167+
2. **Load**: Read with `zip.file('new_data.json')` in `useCanvasProjectLoad.ts` — check for `null` so older project files without it still load
168+
3. **Dispatch**: Add the appropriate Redux action to restore the data
169+
170+
### Adding new entity types with images
171+
172+
1. Extend `CanvasProjectState` type in `canvasProjectFile.ts`
173+
2. Add collection logic in `collectImageNames()` to walk the new entity's objects
174+
3. Add remapping logic in `remapCanvasState()` to update image names
175+
4. Include the new entity array in both save and load hooks
176+
5. Handle it in the `canvasProjectRecalled` reducer in `canvasSlice.ts`
177+
178+
### Breaking schema changes
179+
180+
1. Bump `CANVAS_PROJECT_VERSION` in `canvasProjectFile.ts`
181+
2. Update the Zod manifest schema: `version: z.union([z.literal(1), z.literal(2)])`
182+
3. Add migration logic in the load hook: check version, transform v1 → v2 before dispatching
183+
184+
## UI Architecture
185+
186+
### Save dialog
187+
188+
The save flow uses a **nanostore atom** (`$isOpen`) to control the `SaveCanvasProjectDialog`:
189+
190+
1. `useSaveCanvasProjectWithDialog()` — returns a callback that sets `$isOpen` to `true`
191+
2. `SaveCanvasProjectDialog` (singleton in `GlobalModalIsolator`) — renders an `AlertDialog` with a name input
192+
3. On save → calls `saveCanvasProject(name)` and closes the dialog
193+
4. On cancel → closes the dialog
194+
195+
### Load dialog
196+
197+
The load flow uses a **nanostore atom** (`$pendingFile`) to decouple the file dialog from the confirmation dialog:
198+
199+
1. `useLoadCanvasProjectWithDialog()` — opens a programmatic file input (`document.createElement('input')`)
200+
2. On file selection → sets `$pendingFile` atom
201+
3. `LoadCanvasProjectConfirmationAlertDialog` (singleton in `GlobalModalIsolator`) — subscribes to `$pendingFile` via `useStore()`
202+
4. On accept → calls `loadCanvasProject(file)` and clears the atom
203+
5. On cancel → clears the atom
204+
205+
The programmatic file input approach was chosen because the context menu component uses `isLazy: true`, which unmounts the DOM tree when the menu closes — a hidden `<input>` element inside the menu would be destroyed before the file dialog returns.

‎docs/features/canvas_projects.md‎

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
---
2+
title: Canvas Projects
3+
---
4+
5+
# :material-folder-zip: Canvas Projects
6+
7+
## Save and Restore Your Canvas Work
8+
9+
Canvas Projects let you save your entire canvas setup to a file and load it back later. This is useful when you want to:
10+
11+
- **Switch between tasks** without losing your current canvas arrangement
12+
- **Back up complex setups** with multiple layers, masks, and reference images
13+
- **Share canvas layouts** with others or transfer them between machines
14+
- **Recover from deleted images** — all images are embedded in the project file
15+
16+
## What Gets Saved
17+
18+
A canvas project file (`.invk`) captures everything about your current canvas session:
19+
20+
- **All layers** — raster layers, control layers, inpaint masks, regional guidance
21+
- **All drawn content** — brush strokes, pasted images, eraser marks
22+
- **Reference images** — global IP-Adapter / FLUX Redux images with crop settings
23+
- **Regional guidance** — per-region prompts and reference images
24+
- **Bounding box** — position, size, aspect ratio, and scale settings
25+
- **All generation parameters** — prompts, seed, steps, CFG scale, guidance, scheduler, model, VAE, dimensions, img2img strength, infill settings, canvas coherence, refiner settings, FLUX/Z-Image specific parameters, and more
26+
- **LoRAs** — all added LoRA models with their weights and enabled/disabled state
27+
28+
## How to Save a Project
29+
30+
You can save from two places:
31+
32+
1. **Toolbar** — Click the **Archive icon** in the canvas toolbar, then select **Save Canvas Project**
33+
2. **Context menu** — Right-click the canvas, open the **Project** submenu, then select **Save Canvas Project**
34+
35+
A dialog will ask you to enter a **project name**. This name is used as the filename (e.g., entering "My Portrait" saves as `My Portrait.invk`) and is stored inside the project file.
36+
37+
## How to Load a Project
38+
39+
1. **Toolbar** — Click the **Archive icon**, then select **Load Canvas Project**
40+
2. **Context menu** — Right-click the canvas, open the **Project** submenu, then select **Load Canvas Project**
41+
42+
A file dialog will open. Select your `.invk` file. You will see a confirmation dialog warning that loading will replace your current canvas. Click **Load** to proceed.
43+
44+
### What Happens on Load
45+
46+
- Your current canvas is **completely replaced** — all existing layers, masks, reference images, and parameters are overwritten
47+
- Images that are already present on your InvokeAI server are reused automatically (no duplicate uploads)
48+
- Images that were deleted from the server are re-uploaded from the project file
49+
- If the saved model is not installed on your system, the model identifier is still restored — you will need to select an available model manually
50+
51+
## Good to Know
52+
53+
- **No undo** — Loading a project replaces your canvas entirely. There is no way to undo this action, so save your current project first if you want to keep it.
54+
- **Image deduplication** — When loading, images already on your server are not re-uploaded. Only missing images are uploaded from the project file.
55+
- **File size** — The `.invk` file size depends on the number and resolution of images in your canvas. A project with many high-resolution layers can be large.
56+
- **Model availability** — The project saves which model was selected, but does not include the model itself. If the model is not installed when you load the project, you will need to select a different one.

‎invokeai/app/invocations/batch.py‎

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -56,7 +56,7 @@ def __init__(self):
5656
"image_batch",
5757
title="Image Batch",
5858
tags=["primitives", "image", "batch", "special"],
59-
category="primitives",
59+
category="batch",
6060
version="1.0.0",
6161
classification=Classification.Special,
6262
)
@@ -87,7 +87,7 @@ class ImageGeneratorField(BaseModel):
8787
"image_generator",
8888
title="Image Generator",
8989
tags=["primitives", "board", "image", "batch", "special"],
90-
category="primitives",
90+
category="batch",
9191
version="1.0.0",
9292
classification=Classification.Special,
9393
)
@@ -111,7 +111,7 @@ def invoke(self, context: InvocationContext) -> ImageGeneratorOutput:
111111
"string_batch",
112112
title="String Batch",
113113
tags=["primitives", "string", "batch", "special"],
114-
category="primitives",
114+
category="batch",
115115
version="1.0.0",
116116
classification=Classification.Special,
117117
)
@@ -142,7 +142,7 @@ class StringGeneratorField(BaseModel):
142142
"string_generator",
143143
title="String Generator",
144144
tags=["primitives", "string", "number", "batch", "special"],
145-
category="primitives",
145+
category="batch",
146146
version="1.0.0",
147147
classification=Classification.Special,
148148
)
@@ -166,7 +166,7 @@ def invoke(self, context: InvocationContext) -> StringGeneratorOutput:
166166
"integer_batch",
167167
title="Integer Batch",
168168
tags=["primitives", "integer", "number", "batch", "special"],
169-
category="primitives",
169+
category="batch",
170170
version="1.0.0",
171171
classification=Classification.Special,
172172
)
@@ -195,7 +195,7 @@ class IntegerGeneratorField(BaseModel):
195195
"integer_generator",
196196
title="Integer Generator",
197197
tags=["primitives", "int", "number", "batch", "special"],
198-
category="primitives",
198+
category="batch",
199199
version="1.0.0",
200200
classification=Classification.Special,
201201
)
@@ -219,7 +219,7 @@ def invoke(self, context: InvocationContext) -> IntegerGeneratorOutput:
219219
"float_batch",
220220
title="Float Batch",
221221
tags=["primitives", "float", "number", "batch", "special"],
222-
category="primitives",
222+
category="batch",
223223
version="1.0.0",
224224
classification=Classification.Special,
225225
)
@@ -250,7 +250,7 @@ class FloatGeneratorField(BaseModel):
250250
"float_generator",
251251
title="Float Generator",
252252
tags=["primitives", "float", "number", "batch", "special"],
253-
category="primitives",
253+
category="batch",
254254
version="1.0.0",
255255
classification=Classification.Special,
256256
)

‎invokeai/app/invocations/canny.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@
1111
"canny_edge_detection",
1212
title="Canny Edge Detection",
1313
tags=["controlnet", "canny"],
14-
category="controlnet",
14+
category="controlnet_preprocessors",
1515
version="1.0.0",
1616
)
1717
class CannyEdgeDetectionInvocation(BaseInvocation, WithMetadata, WithBoard):

‎invokeai/app/invocations/cogview4_denoise.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -33,7 +33,7 @@
3333
"cogview4_denoise",
3434
title="Denoise - CogView4",
3535
tags=["image", "cogview4"],
36-
category="image",
36+
category="latents",
3737
version="1.0.0",
3838
classification=Classification.Prototype,
3939
)

‎invokeai/app/invocations/cogview4_image_to_latents.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,7 @@
2727
"cogview4_i2l",
2828
title="Image to Latents - CogView4",
2929
tags=["image", "latents", "vae", "i2l", "cogview4"],
30-
category="image",
30+
category="latents",
3131
version="1.0.0",
3232
classification=Classification.Prototype,
3333
)

‎invokeai/app/invocations/cogview4_text_encoder.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@
2020
"cogview4_text_encoder",
2121
title="Prompt - CogView4",
2222
tags=["prompt", "conditioning", "cogview4"],
23-
category="conditioning",
23+
category="prompt",
2424
version="1.0.0",
2525
classification=Classification.Prototype,
2626
)

‎invokeai/app/invocations/collections.py‎

Lines changed: 3 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -11,9 +11,7 @@
1111
from invokeai.app.util.misc import SEED_MAX
1212

1313

14-
@invocation(
15-
"range", title="Integer Range", tags=["collection", "integer", "range"], category="collections", version="1.0.0"
16-
)
14+
@invocation("range", title="Integer Range", tags=["collection", "integer", "range"], category="batch", version="1.0.0")
1715
class RangeInvocation(BaseInvocation):
1816
"""Creates a range of numbers from start to stop with step"""
1917

@@ -35,7 +33,7 @@ def invoke(self, context: InvocationContext) -> IntegerCollectionOutput:
3533
"range_of_size",
3634
title="Integer Range of Size",
3735
tags=["collection", "integer", "size", "range"],
38-
category="collections",
36+
category="batch",
3937
version="1.0.0",
4038
)
4139
class RangeOfSizeInvocation(BaseInvocation):
@@ -55,7 +53,7 @@ def invoke(self, context: InvocationContext) -> IntegerCollectionOutput:
5553
"random_range",
5654
title="Random Range",
5755
tags=["range", "integer", "random", "collection"],
58-
category="collections",
56+
category="batch",
5957
version="1.0.1",
6058
use_cache=False,
6159
)

0 commit comments

Comments
 (0)