Happy to attend the week long ViZARTS AI & Virtual Production for Filmmakers Workshop at Aalborg University’s VP Training Facility Studio at the Medialogy education in Copenhagen.

The challenge was for each of our four crews to explore how generative AI could assist in producing exterior and interior shots (Approx. 1.5 hours shooting time for each location). We experimented with how any location – real or imaginary – could be re-created or generated by using an AI prompt.

All crews first used text prompts to create 2D location images generated with the help from Midjourney. These 2D images were then adjusted using ComfyAI and transformed into a 3D Depth map using Stable Diffusion, which could then be used as a 3D Set Extension behind the actors, while achieving a parallax/depth effect when the camera was moving.

We also worked to achieve Final Pixels In-Camera by creating physical- and in-camera effects ( ICVFX ) with no greenscreen, using a simple rear-projection screen, Unreal Engine and Vive camera tracking.