Culver Cup Artist Spotlight: Joey from MetaPuppet

Joey has a background in traditional documentary filmmaking - what he calls "the exact opposite of AI." His path shifted when he began interviewing people working in AI technology, sparking his curiosity about merging tech with art. After spending time documenting others in the AI space, he took the leap into creating his own AI-powered art when the Culver Cup competition caught his eye. Despite having no background in animation or VFX, he found he could use AI tools to bring his creative vision to life, producing a short film that would have been impossible for him to make using traditional methods alone.

How have you used Playbook to help your creative processes?

“I first heard about Playbook back in April at the AI on the lot event. I think their use case was a bit different; it was more of a cloud based unreal 3D platform. And then I saw them again at the Augmented World Expo and that was when they kind of had more of a clear vision of a 3D tool where you can create your frame but completely style it with AI. I think this is probably the biggest and coolest thing they are doing because we know AI is gonna make really good looking images, it's just a crap shoot of what you get and having control of this image. So Playbook is an unlock in being able to use a 3D camera, you can block your frame, you can get it looking the way you want. And then you can use AI to make it look a lot better. We used it for the competition. For the diner scene it worked out pretty well because in our short film the diner kind of changes realities and looks completely different. So we had transition shots of the dinner changing into different locations. So having that control where we can put the camera in a spot and completely change the diners look in a couple of ways and make that transition happen, would not have been doable if we were just tied down and text prompting “generate a diner that looks like its in space” because we would just get a random diner everytime, so with Playbook we were able to get that consistency of like keeping the frame constant and changing the way everything looks. So it was mainly used for competition but I definitely want to go down the rabbit hole of what else Playbook is capable of. I saw either towards the end or right after the competition that they added some video keyframing features and key framing in general. So that's the next thing I want to mess around with. And also doing 360 panoramic exports which is not currently supported in the app but talking to JD and Skylar they said they can do it and I think that would be a big unlock for creators as well. If you are in a location and you can stylize a 360 scene, you can load that scene into your phone using different apps and kind of create a low budget virtual production set using Playbook.”

“Do you use AI in your film-making process, other than Playbook” 

Yeah. That is such a broad topic but yes in a lot of ways and there are different groupings to it. Because in AI there are very practical uses of AI like going through footage and tagging stuff and helping search for clips faster. We use AI for mastering the audio when we are done with the film. There is a tool e-master that will mix all the levels appropriately, so it sounds better and that's an easy app practical use of AI. Definitely a lot of previous stuff, I mean that kind of like I think a lot of people are using it for that of mid-journey or ideogram images at least to kind of get an idea of what you visually want a project to look like. 12 labs voiceover, that one we have been using in some videos where it makes sense where we don't need to hire a voice over artist we just use AI. But there are some use cases where we would need to hire a professional voiceover artist to get that delivery, you just can't get that out of AI right now. So it's been a mix of both, across the whole gamut i'm trying to think, we use resolve and they have definitely  taken different approach to AI than adobe has where adobe is very much like ‘we are gonna do generative AI, and we are going to build it into the timeline’ Davinci Resolve they have AI, I mean they don't call it that, they call it, a neural… I can't remember but they have different names for it but it's basically AI. Their app is much more practical application and it was actually useful on AI output since stuff like this with the short film because mostly AI tools they wont give you a full HD video output, it will be like a little less than HD, so you have to do some stuff to uprise it and Davinci Resolve has built in uprising, that's really good, so you can just put it in a timeline and uprise it right there. They have some new audio cleanup tools that are also really good so that's also much more practical for documentary work when you are recording out in the field and  when there is background noise, it's windy, it's noisy. I have definitely been a lot less worried about crappy audio because I know the tools to repair it and improve it have gotten so much better. Obviously you want to record the best stuff possible but I am less worried if there is a noisy AC in the background because AI tools have gotten so good at cleaning up audio. 

“What features would you like to see integrated into Playbook” 

Yeah, I mean, I know the stuff they have on the radar, like basic stuff like saving and different project files. They were def a number of times that I did not realize that if you leave a session and then come back it resets. You start from zero everytime. So, but I know that's like an obvious one. The 360 panoramas, and they said they could do that, that would be interesting and open different opportunities. More usability stuff too, in this project I also tried to figure out, at least get a base level using Unreal and it obviously has more, there are more keyboard and mouse ways to navigate Unreal. Stuff that is probably more familiar for people who work with 3D tools. Bringing that into Playbook would be useful. Moving the camera around in Playbook was a bit of a challenge sometimes because you have to like use the mouse and switch modes of how you want to rotate it so if you can bring in the more traditional mouse and WASD keyboard controls would be handy.The one thing that's really cool is that you can segment the layers and have different prompts for different layers or different objects in the scene. It's a bit tricky to navigate because you can't really see all your layers in a panel. So also just having a way to see all your layers and group them and name them in a more sidebar interface would be useful.

“Do you think AI will replace filmmakers?”

No, I don't think they will be replaced. I mean there will be changes but there are always changes when new technology comes along. I think there will be different flavors of it too, there could be a possibility where someone wants to super personalize film and they give a prompt where they get a film that is, I don't know, decent , mediocre. But maybe they are super into that topic, I think that may be a genre that exists but I think film as we know it will still be around. It still just comes down to creative people and filmmakers using tools to their best ability, to their advantage, and those tools are just changing and making things more accessible. I think there is a question: is AI art? And i think the question really comes down to, ‘how do you use it’. Because, yes, if you go and type in the prompt ‘make me a cinematic image’ and you take that and you just publish and you are like ‘look i made this’ then no I would not really say that you are the artist. Or you had it write an essay for you and you take it and publish it and you are like
‘I wrote this essay!’ and it's like no I would not call that art. But if your using it to like help generate a bunch of ideas and a bunch of elements but you are ultimately the filter who is picking and choosing how you want to use those things, I still think that still part of the creative process ultimately, you the person are still making the thing and are just using different tools to actually make it.