Text2Movie

End To End Movie Generation

Rensselaer Polytechnic Institute

Overview

With the rise of multi-modal implementation, more artistic interpretation of ideas have been realized through text- to-image synthesis. Especially during the first lecture, where we see a story of a goat going from rags to riches selling goat cheese. Then throughout most of our class, we have explored storytelling from a single block of text. However, by utilizing storytelling, we can develop a more concrete story through a playwright's script. This allows us to expand storytelling to a movie. We refer to a movie as a visual and auditory experience.
We introduce an end to end framework, where by providing a description for a story, we are able to generate an entire script and movie that corresponds with that script.

How It Works

More Details Coming Soon

Results

Radke in an ancient mysterious introduction about sheep

a commercial where dwayne 'the rock' johnson sells orange juice

Italian mobsters start an advertisement for a pizza company

gordon ramsay going to the hospital after eating too many burgers

a Burger King commercial with Shaq endorsing 50 pound Whoppers

sherlock holmes in london investigating a murder at a ice cream store

a mysterious ghost that has been haunting a university

a commercial where Michael Jackson is selling an anti-aging pill