Files
ai-video-editor/skills/edit-timeline-studio/docs/command-reference.md
T
Martin Delophyandhaixin.yang 11ddf231e0 Publish Timeline Studio agent editing skill (#42)
* publish timeline agent editing skill

* install ffmpeg in CI

---------

Co-authored-by: haixin.yang <haixin.yang@weimob.com>
2026-07-22 18:47:40 +08:00

6.9 KiB
Raw Blame History

How does an Agent call the Timeline Studio command runner?

Inspect a project

npm run agent -- project.inspect /absolute/path/project.timeline
npm run agent -- track.inspect /absolute/path/project.timeline visuals
npm run agent -- clip.inspect /absolute/path/project.timeline visual-123
npm run agent -- transcript.inspect /absolute/path/project.timeline voice-123

The project result includes revision, duration, ratio, track counts, archived media inventory, applied operation IDs, and warnings. Track inspection returns ordered timing summaries; clip inspection returns source mapping, links, transforms, effects, and other serialized properties. Transcript inspection returns ordered caption segments, speaker labels, linked audio IDs, and word timing/confidence when available; omit the audio ID to inspect all captions.

Apply a plan

node skills/edit-timeline-studio/scripts/validate_edit_plan.mjs /absolute/path/plan.json
npm run agent -- project.diff /absolute/path/plan.json
npm run agent -- project.run /absolute/path/plan.json

The standalone validator checks JSON transport shape, finite time fields, and duplicate operation IDs. It does not prove that an operation is implemented or valid for the selected project. Treat project.diff as the authoritative semantic dry run because it executes the same registry, revision checks, and reducers as project.run without writing an archive.

{
  "schemaVersion": 1,
  "project": "/projects/input.timeline",
  "baseRevision": 0,
  "dryRun": false,
  "operations": [
    { "id": "move-voice-001", "type": "timed.move", "track": "audio", "clipId": "voice-1", "start": 3 },
    { "id": "caption-001", "type": "caption.update", "clipId": "caption-1", "text": "Available today" }
  ],
  "output": { "project": "/projects/output.timeline" }
}

Supported write operations:

Type Required fields Optional fields Effect
asset.import file, track, clipId name, duration, width, height, start, volume, muted, replace Probes, hashes, and embeds an absolute local visual/audio path. Supports Visuals, Music, and an empty Voiceover media slot.
timed.move track: "audio", clipId, start Moves a voiceover and any still-linked captions by the same delta.
timed.resize track, clipId, duration start Resizes audio, sticker, or overlay clips; linked captions follow audio movement and clamp to its end.
visual.trim clipId, sourceIn, sourceOut Trims a video within its serialized source range and remaps keyframes.
visual.split clipId, at, rightClipId Splits at clip-local seconds using an explicit stable ID for the right clip.
visual.reorder clipId, toIndex Reorders the contiguous main Visuals sequence.
visual.append sourceClipId, clipId duration Appends a new stable clip reusing media already stored in the archive.
visual.insert sourceClipId, clipId, atIndex duration Inserts archived media at a main-sequence boundary.
overlay.add sourceClipId, clipId, start duration, layer, muted, transform Adds a timed Overlay that reuses archived visual media.
transition.set clipId, transitionId duration Sets or clears a validated transition on an outgoing Visuals junction.
caption.add clipId, text, start, end audioClipId Adds a timed caption, optionally linked to voiceover audio.
caption.update clipId text, start, end Updates caption content or its finite, non-negative range.
caption.unlink_audio clipId Preserves the remembered audio ID but stops synchronization.
caption.link_audio clipId audioClipId, align Relinks remembered or explicit audio; align: true copies its range.
clip.delete track, clipId Deletes a caption or voiceover; deleting audio leaves caption relink metadata intact.
clip.set_property clipId, property, value Sets an allowlisted numeric transform/audio/layer property with range validation.
clip.set_speed clipId, speed Sets 0.254× video/audio speed while preserving source duration and remapping keyframes.
clip.set_muted clipId, muted Mutes or unmutes video, overlay-video, or audio clips.
track.set_visibility track, visible Changes serialized track visibility using the editor's canonical base-track key.
track.set_locked track, locked Changes serialized track locking using the editor's canonical base-track key.
project.set_ratio ratio Sets 16:9, 9:16, 1:1, or 4:5.

Set dryRun: true to return the predicted before/after summary without writing output. project.diff also includes project-field changes plus per-track added, removed, modified, and reordered clip IDs; modified clips identify their exact changed fields and before/after values. A successful non-empty batch increments revision once. Reusing an applied operation ID is a no-op; a stale revision with new operations returns REVISION_CONFLICT. Failures return a stable code and operation ID and write no partial archive.

asset.import supports JPG, PNG, WebP, GIF, MP4, WebM, MOV, MP3, WAV, M4A, AAC, OGG, and FLAC from an explicit absolute path. Missing duration and visual dimensions are probed with ffprobe; images still default to four seconds when no media duration exists. It computes SHA-256, embeds bytes under media/visuals/ or media/audio/, adds the correct manifest entry, and writes integrity metadata. Music preserves its stable timed segment when reopened in the browser. Voiceover currently has one portable binary slot: importing into a populated Voiceover track fails unless replace: true is explicit. Diff and dry-run probe and validate but never change the input archive.

Render a portable project

npm run agent -- project.render /absolute/path/render.json
{
  "schemaVersion": 1,
  "project": "/projects/edited.timeline",
  "output": { "video": "/renders/edited.mp4" },
  "render": { "width": 1280, "height": 720, "frameRate": 30, "crf": 18, "preset": "medium" }
}

The initial renderer uses local ffmpeg, writes a temporary output, atomically renames it only after encoding succeeds, and verifies the final dimensions, duration, and audio-track presence with ffprobe. It renders contiguous image/video Visuals with contain fitting plus portable Voiceover and Music clips into H.264/AAC MP4. Width and height are normalized to even values; defaults come from the project ratio.

The command fails with UNSUPPORTED_RENDER_FEATURE instead of silently dropping visible captions, stickers, overlays, transitions, visual effects, or separated source audio. Use the browser renderer for those projects. project.render does not run AI generation and is intentionally separate from project.run; the editable .timeline remains the source of truth.