Building ChatCut: Conversation... Note

Building ChatCut: Conversational Video Editing in the Browser with Cloud Models

ChatCut is a conversational editing workspace integrated into Timeline Studio's Anna edition, allowing users to edit videos through natural language commands without desktop software. It divides tasks between the browser for editing state and playback, and the cloud for model interpretation and generation. A bounded tool loop around the editor ensures that only registered operations are dispatched, with the application enforcing the model's proposed actions to protect user edits. The system captures project state tokens at the start of each turn, validating any generated edit previews against this initial state before committing. Visual evidence inspection allows the model to view selected source frames, but with strict limits on the number of frames to prevent overclaims of understanding. Video generation is an asynchronous process where requests are validated, submitted, and then polled for results, with downloaded media being stored separately from the timeline. Generation and placement decisions are intentionally separated, allowing for user review of generated content before it is added to the project. Music generation begins by discovering model capabilities, preparing prompts, and opening official generation forms for user confirmation. Recovering music results is facilitated by persisting exact requests, preventing redundant generation and billing. Music-aware cutting involves analyzing waveform and timing data to create a cut plan that respects the track's accents. Exports reuse the editor's composition pipeline, including captions and effects, with performance depending on local device capabilities. The open-source repository provides access to the code and documentation for further exploration.