Skip to content

Animation

Jump to bottom
Ben Rugg edited this page Nov 29, 2022 · 7 revisions

Animation Rendering Instructions

With AI Render, you can render animations with all the traditional Blender animation tools, in addition to being able to animate Stable Diffusion settings and even prompt text!

You can also use animation for batch processing - for example, to try many different settings or prompts.

Getting started

It's important to note that AI Render has its own Render Animation button, which must be used. Rendering an animation with Blender's built in Render Animation function will not trigger Stable Diffusion.

  1. Enable AI Render in your scene like normal

  2. In the AI Render panel, twirl down Animation, and then set a path for your image frames

  3. Set your start frame, end frame and step in Blender like normal (under Output Properties > Frame Range)

  4. Optionally: Animate your scene in Blender like you normally would (animating objects, poses, camera moves, etc)

  5. Optionally: Animate AI Render settings (all settings are animatable, including prompt text; see below for more info)

  6. Click Render Animation in the AI Render panel (Important: You must use AI Render's button, not Blender's standard animation rendering)

Your frames will be rendered one by one in Blender and then sent to Stable Diffusion. Rendered images will be created in the path you set.

Converting image sequence to video file

When animation rendering is done, you will have a sequence of still frames. To create a video file, use Blender's Video Sequencer like you normally would for converting an image sequence to a movie.

If you haven't done video editing in Blender before, see Blender's manual, starting on Step 8, or follow a YouTube tutorial.

Animated Prompts

In AI Render, twirl open the Animation panel and check Use Animated Prompts. Then click Edit Animated Prompts and you will see a text editor in the AI Render workspace.

Use this text field to enter frame numbers and corresponding prompts. The frame number is the starting frame for that prompt. So for example, in this text...

1: A cactus in a blue sand desert on Mars
20: A man on the horizon of a pale blue salt dune
53: Surreal spaceship melting into an hourglass

...it would use the prompt "A cactus in a blue sand desert on Mars" for frames 1-19, and then the prompt "A man on the horizon of a pale blue salt dune" for frames 20-52, and so on.

Negative Prompts

When using animated prompts, you can also animate negative prompts. You can use the same start frames as the positive prompts, use a single negative prompt on frame 1 (which will be applied to all frames), or mix and match start frames to batch test different combinations.

As an example:

1: Elven forest
3: Elven warrior

Negative:
1: 
2: moss, fog
3: ugly, bad anatomy
4:

This text would produce four different combinations:

  1. "Elven forest" (no negative prompt)
  2. "Elven forest" (negative: "moss, fog")
  3. "Elven warrior" (negative: "ugly, bad anatomy")
  4. "Elven warrior" (no negative prompt)

See the other wiki page for more info about negative prompts.

Animation Tips

  • Animations will be most stable when setting "image similarity" to 0.7 - 0.8. However, this won't modify your image much. So it's a balancing act between how much you want Stable Diffusion to do for you, and how smooth you want the animation to be. Even in the best cases, there isn't currently a way to get fully smooth animation.

  • Unchecking the "random seed" setting may also help create smoother animations. Especially when the scene varies just slightly between frames, it may produce more similar results from frame to frame with the same seed.

  • Definitely do use a "random seed" if you are using animation to batch test lots of different images, and you aren't concerned about similarity from frame to frame.

  • Try setting your "frame step" (normal Blender setting in "output properties") to a higher value than 1 to skip frames. For example, a value of 5 would render every 5th frame (ie. 1, 6, 11, etc). This can give you more of an "animatic" feel if you then keep the frame rate slower when you turn the image sequence into a video.

  • Try experimenting with "prompt strength" and the "ddim" sampler. Some have suggested that this could give you smoother animations.

  • If you have suggestions or tips, please share them with me on Twitter @AI_Render or email me at airenderblender@gmail.com.

Caveats

  • If you are using DreamStudio, you will eat up your credits quickly when rendering lots of frames. Try setting your "frame step" (normal Blender setting in "output properties") to a higher value than 1 to skip frames. Or consider running stable diffusion locally.

  • Gettin a timeout from DreamStudio will stop the whole render process. Make sure to check a long running render every once in a while. You can resume rendering by changing your start frame to wherever it left off. Also consider running stable diffusion locally as an alternative.

Clone this wiki locally