rompecabezas

Every clip on this page starts as video generated by AI.
LTX 2.5 (Lightricks), first-and-last-frame image-to-video: two stills and a text prompt go in, the model generates the motion between them. 1280×704, 24 fps, 145 frames (6 s). The same shot is generated many times with the same frames and prompt; only the seed changes. Later sections also redraw these takes with Stable Diffusion through Deforum.

first frame → … → last frame

S02C · nine takes of one shot, unedited. Same first frame, same last frame, same prompt, nine seeds.

These experiments started with a small edit: replacing a frame with the same frame from another version of the shot. I wanted to see how much I could change inside an image before it became another image. The tools grew through testing, watching, and giving names to what I was finding.

how the frames are reordered

I start with several versions of one shot. They share the first frame, last frame, prompt, and duration; the seed changes. I choose one as the base, then replace selected frames with frames from the other takes at the same position. Frame 34 replaces frame 34.

The pattern decides where I make the substitutions. The run mode decides how long they last. In this 48-frame example, a dot means the base take; a digit identifies the take I put in its place.

positions        (48 frames)

fib              .xxx.x..x....x.......x............x.............
phase:3:8:0.12   x.................x.x...............x...x.......
euclid:3:8       x.................x.................x...........

run mode

single   1.................2.3...............4...1.......
grow     1.................22333.............444411111...

Fibonacci places the substitutions close together at the beginning, then spreads them out. Euclidean and phase patterns repeat. Single replaces one frame at a time. Grow makes each substitution longer as the clip progresses, until what began as a flash can become a short shot.

the vocabulary

I call the cuts between takes aspero. Velvet averages them. Breathe moves between an average and a single take on a rhythm. Quantum uses the differences between takes to control that movement. Melt redraws frames with feedback from the previous result. Chunkforum builds movement across clips and uses cuts to reset it. The names help me work with the operations and remember how they feel.

I measure divergence: how much the takes differ at each frame. It helps locate places where a substitution may show or an average may soften the image. Quantum also uses it as a control signal. The number tells me something about the material; I still have to watch it.

Some rhythms come from Bjorklund’s Euclidean algorithm, including E(3,8), the tresillo, and E(5,8), the cinquillo. There are named clave patterns too. Phase runs two tresillos at slightly different rates so their relationship keeps shifting.

the pipeline

I use GPT to prepare shot briefs and pairs of images. The pipeline reads the original script, finds the latest image versions, and adapts a working ComfyUI generation. It changes the images, prompt, duration, seed, and output name while keeping the settings from that graph. It then waits for the renders, selects a base take using audio dynamics, makes the micro-edits, and builds a comparison grid.

  python pipeline.py --shots S02D --n 8

velvet 5

Here I average the takes. Where they line up, the image stays relatively clear. Where they differ, it softens or splits. Ramp begins with one take and gradually brings in the others. Breathe returns to one take in three frames, then opens back out over fourteen. I am testing how that change between a definite image and several overlapping images can carry a rhythm.

melt 9

I made melt to work on the result of a micro-edit. It redraws each frame at partial strength and feeds the previous output into the next frame. I was looking for the continuous transformation I had worked with in my earlier Deforum films. These tests brought me closer to it. I also lost a lot of detail by rendering below the source resolution and using the tiny VAE. Returning to the source resolution, using the full VAE, and adding back high-frequency detail improved the result. The measurements help compare versions, but they do not establish that the images feel the same.

quantum 7

In quantum, the differences between takes control the edit. Where the takes agree, I keep their average; where they separate, the image moves toward one take. The inverted version does the reverse, keeping several images visible through the most unstable moments. These tests produced fewer measured discontinuities than breathe. I am interested in how the timing changes with each shot, because the material is part of what drives it. Quantum is a name for the experiment, not a claim about physics.

agua 16

Water behaves differently. Eight takes of a winter swell can overlap into a denser sea. Cutting between them can also work because there is no single performance to preserve. But the measurements become less useful: two takes of the same wave can have very little correlation in their frame-to-frame changes. I have to judge these by watching the water.

chunkforum 12

Chunkforum builds a warp across a segment, then cuts to another clip when the movement needs to reset. I wanted to test how far I could get toward continuous travel using existing footage and no new generation. Some movement and color measurements approach those of my earlier films.

deforum 1

Here I put LTX inside a feedback loop. The current image becomes the first frame, a warped version becomes the last, and the model generates the movement between them. I use the result to start the next link. New objects appear as the chain develops, while others disappear.

lora 8

These are tests with 2.3 LoRAs loaded into the 2.5 workflow. They act during generation, so I need a new family of takes before I can apply the micro-edits. In these examples, transition produced a cut with a magenta wipe rather than the morph I was looking for. Restyle increased saturation without the same measured discontinuities. These are results from the tested workflows, not a guarantee that every LoRA will transfer.

faces 25

For Miranda’s sequence, I assembled moving portraits and faces cropped from group stills. I kept hard cuts and reshuffled the order with each seed. Crescendo shortens the holds from fourteen frames to one. Live gives more time to the moving portraits and uses the stills for the shorter flashes. I wanted the sequence to move through the faces without settling into a fixed order.

S02 9

The Coney Island scene, assembled using the timings in my sequence plan.

aspero 128

Aspero is the operation of the original idea: substituting frames between takes of the same shot on a rhythm. The takes share a setup (prompts. number of frames, first and last frame) but differ in their details. I use that small difference to make the image unstable from within.

V3Y seq 22

Earlier versions of the subway scene. Some use frame substitutions to move between shots. Another version is built around events and ends with a freeze, a cut to black and silence, then a single held frame.

V3Y 50

S01 51

Early Fibonacci substitutions across takes of the subway shot, followed by assemblies with the ElevenLabs voiceover.

archaeological software excavation

I made my 2023–24 films with AUTOMATIC1111, Deforum, and Parseq. Parseq let me score changes frame by frame using keyframes, formulas, and audio. When I moved to the RTX 5080, the old PyTorch installation could no longer run on the card. Returning to those films meant rebuilding access to the tools that made them.

Working with GPT, I copied the engine and Deforum into a separate environment with a newer PyTorch build. The first animation failed before rendering a frame. After a patch for an empty ControlNet namespace, the retry produced 48 frames. A handoff note still described the failure after the render had succeeded. I had to check the logs and the actual output to know where the work stood.

excavation 3

The first renders from the restored Deforum engine. A 48-frame Realistic Vision test brought back the kind of continuous change I had been looking for. Its measurements fell within the range recorded from my earlier films. ComfyUI can now launch the engine, send a job, retrieve the frames, and stop it before another model uses the GPU.

jam 5

I went back through eighty settings files containing Parseq data. They showed how I had been working with drifting seeds, audio-driven strength, camera movements scored against the bar, and prompts that blend over time. I tested those mechanisms separately on one still. Abrupt prompt changes lost the face early. Blended prompts held it longer while gold and leaves appeared. Seed drift reduced some changes but also led to a dotted texture. In the old scores, strength kept moving; holding it constant changed the behavior.

hyper 5

I tested my julietest settings with an SDXL Hyper model and the leonsol22 score. The image quickly became spots and radial streaks. Changing the step count, camera, CFG, and noise did not resolve that behavior in these tests. The same score held its form better with Realistic Vision at CFG 6. I am keeping the comparison because a score that worked in one setup did not carry over cleanly to another.

feed 15

I use an LTX clip to guide Deforum through depth and, in some versions, edges. This gives the redraw a moving structure while the prompts change the surfaces. In the subway tests, coats become gold and foliage appears in the windows. Stronger control preserves more of the source and gives the prompts less room. Looser or pulsing control lets more transformation through. I also used the bass from Subwayseq to drive strength on every frame, then tested it on the ControlNet weights as well. The surfaces respond to the music while the figures largely keep their outlines. One apparently steady result was almost frozen, which made the limit of the measurements very clear: less frame change is not automatically better movement.

mondongo 11

Mondongo combines several controls. The LTX clip supplies structure through ControlNet, motion through optical flow, and a portion of its own image through Hybrid Video. A constant 30% source blend held the wider group shots so firmly that little of the prompt came through. On the scar close-up, gold seams and stone textures could develop while the eye and gesture remained recognizable. Changing the blend let the transformation go further, but one version lost the woman’s identity. I also play the results backward. On the scar, reversing the sequence changes the reading: the marks draw together and the surface returns toward skin. That change in meaning interests me as much as the settings.

ripple 13

With Ripple, I give the model a reference clip and an edited first frame. I am testing how much of the movement it can keep while changing the image. In the professor replacement, the measured motion correspondence was much higher than between two independently generated takes. That makes another kind of micro-edit possible: cutting between versions that share more of the same movement. Repeated passes retained the measured motion in this test while gradually losing visual fidelity. It is useful for this operation; the continuous melt remains a different problem.

monstruo 9

These are the tests where the image went somewhere I did not want. A close-up of skin and a brow did not give Ripple enough information to preserve the person. The movement survived, but the model invented the face revealed at the end. Enlarging a face from a group image introduced other problems: compression and sharpening became marks that looked like wounds. Those changes brought a feeling the scene did not need. I kept the failures to show where the operation stopped serving the piece.

cangrejo 22

Cangrejo begins with one clip. I reorder its frames: three steps forward, one back. The shot advances while repeatedly pulling backward, and the result runs twice as long. Four forward and three back slows that progress much further. The effect depends on the subject. A wave can reverse without announcing it; a hand returning a bag makes the undoing visible. Salad adds other takes of the same shot, placing some backward steps in another version. I test the substitution on the back-step, once per cycle, on every frame, or at random.