root/moteva
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(canvas): add visual annotation edit action

Browse Source 
Add a VisualAnnotationPanel letting users mark regions on an image (brush,
circle, rectangle, cross) with per-mark edit instructions, composited into
an annotation guide image. The new "visual-annotation" node action sends
the unmarked source plus the annotated guide to the image model, editing
only numbered regions. Server validates the annotation image/instructions
and threads them through the generator task.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
liangxu
2026-07-11 10:03:59 +08:00
parent 114198f1d9
commit bb8946b79b
17 changed files with 1271 additions and 46 deletions
Download Patch File Download Diff File Expand all files Collapse all files
server/API.md
+19
View File
@@ -434,6 +434,25 @@ Returns the updated project, the assistant message, and generated nodes.
The generation path uses the configured design agent runner. The default is the `sky-valley/pi` Go agent adapter, with local desktop/MCP agent behavior disabled.
## Image Node Actions
`POST /api/projects/:id/nodes/:nodeId/actions/async`
Visual annotation edits keep the selected clean image as the base and provide a second uploaded guide image containing numbered brush, circle, rectangle, or cross marks. Each number maps to an editable instruction in `prompt` and the normalized geometry in `selection`. The model must use the guide only for location and must not render marks or numbers into the result.
```json
{
"action": "visual-annotation",
"prompt": "1. Cross: remove the chest logo\n2. Rectangle: change the trousers to navy",
"imageModel": "gpt-image-2",
"imageSize": "1024x1024",
"annotationImage": "https://cdn.example.com/annotation-guide.png",
"selection": "{\"annotations\":[{\"index\":1,\"type\":\"cross\",\"instruction\":\"remove the chest logo\",\"points\":[{\"x\":0.42,\"y\":0.28},{\"x\":0.58,\"y\":0.5}]}]}"
}
```
`annotationImage` and a non-empty `prompt` are required for `visual-annotation`. The original node remains unchanged. Submission creates a generating copy 32 canvas units to its right, and the completed image replaces only that copy through the standard asynchronous event lifecycle.
## Generator Tasks
`POST /api/generator/tasks`