The world’s first movie fully translated and dubbed by AI is about to be released
Flawless, a company founded by filmmakers and scientists, may have finally crossed the uncanny valley of dubbing content.
Currently, watching a foreign language content dubbed into English is an uncomfortable experience.
With streaming platforms bringing a glut of local-language movies and TV shows, the experience of watching a Danish crime drama or a Spanish boarding-school soap dubbed into English is a nightmarish journey into the uncanny valley, a term widely used to describe technology that fails to convince us, effectively throwing our human likeness into a dystopian nightmare.
Enter Flawless , an AI-powered filmmaking studio that wants you to be able to sleep soundly while watching hit shows without mismatched lip syncs and brutal scene cuts. Founded in 2018 by multi-talented directors Scott Mann and Nick Lynes, Flawless’ proprietary technology, TrueSync, overlays actors’ faces and delivers some of the most impressive translations we’ve ever seen in the AI space. The company showed off a trailer for its work at this year’s Cannes Film Festival, which Variety shared exclusively with us.
This AI company wants to revolutionize the way multilingual dialogue is re-dubbed
Flawless aims to protect artistic copyrights and abide by Hollywood labor union rules, which raises big questions about the value of international content and its potential to break through in the US market. Variety also reports that Flawless has officially entered the film distribution world. The company has entered into a joint venture with XYZ Films and Tea Shop Productions to acquire foreign film rights. The partnership will convert these projects into English, promising “perfect lip-synced visuals”, and distribute them in relevant territories. The first film to be released is Victor Dannell’s award-winning sci-fi film UFO Sweden, which will be released domestically under the title Watch the Skies. A slew of projects are expected to be launched in the coming months*.
In a lengthy conversation with Variety, Mann discussed Hollywood’s fear and loathing of AI, establishing ethical norms, and the brain mechanisms of the uncanny valley.
How did you get into this field?
It all started in 2018. My background is as a director, screenwriter, and producer. I stumbled upon the first paper on Generative AI. It was called “ Deep Video Portrait ” and it was published at SIGGRAPH, which is basically the Oscars of science. It blew my mind.
I had been working in the field of visual effects (VFX) and it was clear what impact this technology could have on our industry. I contacted the authors of the paper, which led to me co-founding the company with these people and my partner Nick Lynes. Christian Theobalt, Pablo Garrido and Hyeongwoo Kim founded our science department and developed it into a world-leading computer-driven generation science department. This was our secret sauce.
There’s a lot of fear and anxiety in Hollywood right now about AI. What’s interesting about your company is that you’re full of optimism.
We started this company from a filmmaking perspective. I love art, and our mission is to look at this technology as the ability to make tools. There’s a view that this technology is designed to eliminate creatives and steal art, just like Napster did. But we believe in the expression of human stories, and film is the most advanced form of expression. Technology is amplifying that and adding more creative layers. There are benefits to that. Also, the economics of this whole business rely on certain protections and an understanding of how it works.
There is a severe lack of education about AI in this community. How has this impacted your business?
This is a global problem, not just Hollywood. Education and understanding are key. Some companies — let’s call them flawed companies — would rather not educate because it reveals a shady practice. You need to have honest conversations about how destructive it is and how proactive it is. We’ve prioritized understanding with the studios and labor unions — including SAG-AFTRA — over the last few years and said, “Look, this is how it should be used.” The next [AMPTP contract] negotiation is something everyone should start discussing now. The industry is transforming, and we should be ready.
Well said. Is the industry taking this seriously enough?
I can confidently say that attitudes are changing. Before last year’s strike, the approval route for projects using AI was cautious. People were having conversations in the early stages and seemed to understand them, and were trying to improve the process. But there is a fear of crossing that threshold. The industry is in such a bad shape in some places right now that many don’t want to rush into the wrong solution.
How did Flawless establish its code of ethics?
This was triggered by a personal experience I had as a filmmaker. I always wanted to fix the dubbing of movies. I made a movie called Heist, with Robert De Niro. I made the movie with great care, and then when I watched it dubbed in a foreign language, the script had been completely rewritten, and the lip movements were completely different. I realized, “Oh my god, no wonder this stuff doesn’t go over well.”
I felt the urge to correct this and tried a lot of visual effects (head scans etc.), but these were impractical and failed to cross the uncanny valley. In short, you can’t do it without removing and destroying the entire contribution to the film. We learned that ethics and legal rights are inseparable.
Speaking of the “uncanny valley”, it’s really cruel to watch the current form of dubbing. Especially streaming makes it even more serious.
As humans, we study each other in great detail. If we see something out of the ordinary, our brains go into high alert. Our brains are constantly being fooled and retrained as filmmaking evolves. Your subconscious is working to check reality. If something doesn’t mesh well — especially sound and visuals — you immediately put up a huge barrier when you’re watching, hindering any sense of immersion, empathy, and trust. Think about how expressive an actor’s facial expressions and gestures can be, without even hearing the sound. All of this makes [current dubbing] a problem. But it’s good to have clear problems to solve.
There’s also a pretty game-changing factor here for international films. It could open up a new market for stars and eliminate the need for American-language remakes of many acclaimed films.
As we’ve shown our material to filmmakers over the past year, they’ve realized the potential to go from the local stage to the global stage. It’s a huge opportunity, and it’s exciting to get your work shown. They’re very excited about getting their work out to a wider audience, especially in the U.S.
Have any studios or companies tried to acquire you outright?
Yes, when we first came to Hollywood in 2021. We had a very short pitch, and some people tried to make a direct offer. Nick and I agreed that we wanted to serve everyone, not just the interests of one studio. We wanted to be neutral because otherwise it would be very limited. But it helped us raise money and expand the company. It’s a good problem.
You also want to get into distribution.
In a way, we’ve always seen ourselves as a studio. We want to be future-proof. You need to get every stage right from pre-production, production, to distribution. All of these stages are impacted by technology. My view is that we should be able to make films to 10 times the audience. That, coupled with the fact that it’s cheaper to make amazing films with these tools. If you look at our first film, UFO Sweden, it’s obviously been translated into English, but it also gave the filmmakers the freedom to add and change things that they couldn’t afford the first time around. You can choose to try to do a job like an algorithmic component, but I think the real power is in making the film and distributing it. The whole process. And in post-production, we can do so much more. Look at Pixar, they can iterate on their films right up until the moment before release.
https://kcgod.com/first-movie-fully-translated-by-ai
More about AI: https://kcgod.com
Peace of Mind, Guaranteed. Your data is our priority
