LAB31

To quote Dr Karl Kruszelnicki, “It’s not an experiment unless you write down your results.” Today we’re experimenting with Luma’s Dream Machine, and we’re writing down our results.

What is Dream Machine?
Luma’s website says “Build workflows that take static images and instantly create magical high quality animations. Instruct Dream Machine in natural language to create narratives.”

But for real… what is it?
Basically you upload two images – the starting frame and the end frame, and Luma will create a 5 second video with their AI magically imagining the content in between. In addition to the images, you also write a text prompt to tell the camera what to do. E.g. camera pan left, camera crane down, camera push in etc.

It got us wondering what a practical use case for this might be. Since it’s static imagery that you upload, it seems to lend its self nicely to “exploring” static spaces, like, a house. So we thought it might be interesting to hop on Zillow and find a random property listing, grab some the photos from it, and see if we can make a video fly through.

Here’s the listing we used. It had 100 photos to choose from, which turned out to be a pretty crucial decision.

Here’s the video we made from 7 photos:

It turned out to be a fairly involved process. Lots of trial and error between image selection and prompt adjustments. It’s also an expensive exercise – burning through our $9/m subscription credits in 5 minutes and having to dish out another $11 for science.

Here were the lessons learned from this particular use case:

• Each image needs to have as many elements of the next image within it as possible.
• Images all need to be the same aspect ratio.
• Moving through doorways is tough.
• Moving up or down stairs is impossible.
• The second you get too ambitious, things get weird. Swinging around from the kitchen island to the TV took way longer to land than you’d expect, and it still looks a bit wacky.
  

So then it got us wondering, what if the images were taken to be optimized for Luma? What if each photo was taken with the following photo in mind?

We gave it a shot. Here’s a fly through made from 8 photos of one of Lab31’s satellite offices…

Okay so we got a little bit ambitious at the end… but did you notice the new french doors beyond the ping pong table?

This technology is in its infancy, but we suspect use cases like this are going to become more and more commonplace in the next few months.

Remember, this is the worst this technology will ever be.