The AI-dubbed version premieres as Watch the Skies in the U.S in May. NFTVF talked to producer Albin Pettersson about AI controversies and the cutting-edge technique “vubbing”.

The Swedish adventure film UFO Sweden (2022), produced by the film collective Crazy Pictures, supported by Nordisk Film & TV Fond, and sold internationally by REinvent Studios, is now set to meet audiences outside the Nordics in an English-dubbed version titled Watch the Skies.

But this won't be the typical dub where the audio and visuals risk being awkwardly out of sync. Thanks to an AI dubbing technique called visual dubbing, or “vubbing”, Watch the Skies will seamlessly both sound and look like the original language is English. This groundbreaking method uses artificial intelligence to alter the actors' mouth and facial movements to match a new recorded dialogue by the actors.

Watch the Skies will be the first feature film dubbed in this way, and together with the American AI company Flawless, Crazy Pictures has spent two years perfecting the method. Crazy Picture co-founder and producer Albin Pettersson says that the company has always been quick to embrace new digital filmmaking techniques.

“We started working with digital cameras early on, and began making a lot of films for YouTube, where we also did a lot of special effects and visual effects. Somewhere along the way, we realised that suddenly we could create really cool stuff with fairly small resources. And this is also a new technology that suddenly allows you to open up a local film to a much wider audience.”

The partnership between Crazy Pictures, Flawless, and XYZ Films, which handles international distribution, began during the Cannes Film Festival in 2023.

“A significant reason why our film was discovered is likely that this is a genre film with a strong visual appeal, which should, in theory, have considerable potential. However, the biggest obstacle was the language,” Pettersson says.

Although Crazy Pictures is progressive when it comes to digital developments, they were initially sceptical about using AI for dubbing the film. When they received the project proposal, the Hollywood strike was at its peak, with writers and actors pushing back against AI’s role in the industry.

“AI is a controversial topic, so at first, we were a bit uncertain about what this actually means for the film and the industry, and whether this was the right or wrong path to take.”

The use of AI in filmmaking is still a heated topic, which became especially evident ahead of this year’s Oscars when Brady Corbet’s The Brutalist faced significant backlash for using generative AI to enhance the Hungarian dialogue. In the Nordics, AI technology is also making its way into filmmaking. For example, in SVT’s series Faithless (Trolösa), which premiered earlier this spring, AI was used to make Danish actor Jesper Christensen’s lines sound more Swedish.

However, the AI dubbing of Watch the Skies didn’t involve AI-generated voices – instead, the original actors re-recorded their lines in English. To make the English dialogue match the existing visual footage, a computer program analysed the mouth movements and jaw structures in the film. Based on this analysis, the program generated animations that, with the help of AI, adapted to the new audio, creating the illusion that the actors were speaking English.

“What we’re doing is essentially refining that process to enhance the experience for viewers. No one is losing their job in this process; if anything, we’ve actually created more opportunities. The actors are still performing their roles, but the visual aspect is adjusted to make the experience much better for the audience.”

Although using AI-generated voices would have made the process faster and cheaper, it was never an option, Pettersson says. Crazy Pictures didn’t want to compromise the film's quality or the actors' performances.

“There was basically a finished film a year ago, but we kept feeling like we wanted to refine it, make things better, and improve the software along the way. The goal is for this process to become much faster in the future, and it's already significantly quicker now.”

Pettersson explains that, in this case, the dubbing has not entailed any additional costs for Crazy Pictures, as Flawless has purchased the film rights and covered all expenses for the dubbing and visual changes. He adds that these costs must remain relatively low for the method to remain an attractive option.

On the technical side, scenes where someone is smoking or making large mouth movements have been tricky for the AI to handle, but overall, the result turned out better than expected, says Pettersson.

“We had a lot of uncertainty about how well we would actually make it look, but today the results look fantastic. It’s truly beyond our expectations, so we’re really pleased.”

How has your approach towards AI in filmmaking changed along the way?

“Above all, we’ve gained a deeper understanding of how everything works. As long as we can maintain creative control and involve the actors in bringing our vision to life, we’re creating something that ultimately becomes better for the viewer.”

“We find it more exciting to embrace AI and explore what we can achieve with it, rather than dismissing it and saying that it doesn’t seem like the right path,” Pettersson adds.

Will you consider doing this again?

“Yes, it’s something we’re thinking of for future projects. This is a technology that can definitely be implemented in other types of projects as well, but I think it will be most useful for projects with potential for the international market.”

Watch the Skies will hit the theatres in the U.S. on May 9th, with later premieres in both the UK and Australia.

Official trailer