WorkRally AI generation guide

Core rules

• Run workrally generate image-models -o json before image generation.
• Run workrally generate video-models -o json before video generation.
• Do not hard-code model_id or provider_id; availability varies by environment.
• Upload files and use the returned managed asset URL rather than passing local paths.

Image generation

Start with a single prompt and a small output. Add reference images only after the basic generation path works.

workrally generate image-models -o json
workrally generate image   --prompt "an orange cat under a cherry tree"   --model <model_id> --aspect-ratio 16:9 --poll

# Reference-image generation
workrally generate image   --prompt "place the first image on the road in the second image"   --model <model_id> --input-images "https://image1,https://image2" --poll

Video generation

WorkRally exposes text, FirstLastFrame and SubjectToVideo drivers. Choose the mode according to the available anchor assets.

workrally generate video-models -o json
workrally generate video --prompt "waves on a beach at sunset" --model <provider_id> --poll
workrally generate video --mode FirstLastFrame --prompt "the character walks left to right"   --model <provider_id> --first-frame-url "https://start.png" --last-frame-url "https://end.png" --poll
workrally generate video --mode SubjectToVideo --prompt "the character walks through the scene"   --model <provider_id> --reference-assets '[{"type":"image","url":"https://character.png"}]' --poll

• Text: the model creates the scene from the prompt.
• FirstLastFrame: the motion is constrained by start and end frames.
• SubjectToVideo: a reference subject is placed into a generated motion scene.

Canvas placeholders

Generation can create a placeholder node in the canvas so the output remains connected to the project layout. Treat the canvas update as a separate state change from the generation task.

• Keep the project and canvas IDs explicit.
• Do not assume a failed generation created a valid node.
• Use incremental build or delete/overwrite behavior according to the operation.

Task polling

# Automatic polling
workrally generate image --prompt "..." --model <id> --poll

# Manual lookup
workrally generate task <task_id> -o json
workrally generate task <task_id> --poll

Keep the task ID, request parameters and raw response. Poll with a limit, classify queued, running, succeeded and failed states, and do not create duplicate work after a client timeout until the original task is checked.

For uploads, canvas composition and troubleshooting, continue with the WorkRally overview.