Bytedance Hyper Realistic AI Video Animator, Dreamina, Nears Microsoft VASA-1 Level

by

One year ago Microsoft Researchers created a hyper releastic AI video animator, VASA-1, but did not release it. Now, Bytedance owner of TikTok, is getting closer to matching the VASA-1 ability to hyper-realistically animate the facial muscles of faces and lip-sync based upon a single photograph. The Bytedance product is called Dreamina.

The lipsync is solid in Dreamina. However, VASA-1 has superior eye tracking and whole face animation.

VASA-1 is able to extrapolate the whole voice from a single audio file recording of the person in the picture. VASA-1 is still superior. VASA-1 has not been released.

VASA is a framework for generating lifelike talking faces of virtual characters with appealing visual affective skills (VAS), given a single static image and a speech audio clip. The premiere model, VASA-1, is capable of not only producing lip movements that are exquisitely synchronized with the audio, but also capturing a large spectrum of facial nuances and natural head motions that contribute to the perception of authenticity and liveliness. The core innovations include a holistic facial dynamics and head movement generation model that works in a face latent space, and the development of such an expressive and disentangled face latent space using video.

AI will become available that will enable the precise mimicking of humans in realtime video and audio. On a video call, you will not be able to tell what is real and what is generated.

7 thoughts on “Bytedance Hyper Realistic AI Video Animator, Dreamina, Nears Microsoft VASA-1 Level”

  1. [ calling it hybrid AI, is maybe a step in-between towards AGI, like real world is providing a step in-between towards available capacities for battery long haul trucking? Edison
    ‘https://youtu.be/dBMguDfirgA?t=405’
    but, Do ‘we’ like that, with no ‘gears shifting’? (thx) ]

  3. This is *okay* for levels of realism, but it won’t fool anyone who’s spent a good chunk of time watching A.I. stuff and following its progression. With that said, I think all schools and companies should begin mandatory classes/training for how to recognize deepfakes and other A.I. videos and pictures. I say start at first grade and progress every year from there so that recognizing fakes will become second nature. A lot of.people won’t like to hear that because it means the public would less likely be swayed to act on false information.

    • I agree that current AI output is easy to recognize if you know what you’re looking for, but you should not expect that to persist; It’s the same general principle: The computers are getting better all the time, and we’re not.

      • Yeah, I’m not getting any “uncanny valley” vibes from these renderings. The VASA-1 video does a good job of the ‘meh’ female whereas “Dreamina” is prettier than a fully airbrushed Scarlet Johannson on her best day. I so see some blurring in her facial muscles while she forms words, similar to what I’ve seen with hands on sewing machines per some recent Chinese anti-MAGA anti-tariff memes (what a hoot these were!).

  4. If everything is real then nothing is real.

    Remember that when you are surfing instagram, reels, tiktok, snap, facebook, and X.

Comments are closed.