Metaverse and A.I.

OmniHuman: ByteDance’s New AI Creates Realistic Videos From a Single Photo

ByteDance researchers have developed an AI system that transforms single photographs into realistic videos of people speaking, singing and moving naturally — a breakthrough that could reshape digital entertainment and communications. The new system, called OmniHuman, generates full-body videos that show people gesturing and moving in ways that match their speech, surpassing previous AI models that could only animate faces or upper bodies.

“End-to-end human animation has undergone notable advancements in recent years,” the ByteDance researchers wrote in a paper published on arXiv. “However, existing methods still struggle to scale up as large general video generation models, limiting their potential in real applications,”  The team trained OmniHuman on more than 18,700 hours of human video data using a novel approach that combines multiple types of inputs — text, audio and body movements. This “omni-conditions” training strategy allows the AI to learn from much larger and more diverse datasets than previous methods.

“Our key insight is that incorporating multiple conditioning signals, such as text, audio and pose, during training can significantly reduce data wastage,” the research team explained. The technology marks a significant advance in AI-generated media, demonstrating capabilities that range from creating videos of people delivering speeches to depicting subjects playing musical instruments. In testing, OmniHuman outperformed existing systems across multiple quality benchmarks.

The development emerges amid intensifying competition in AI video generation, with companies like Google, Meta and Microsoft pursuing similar technologies. ByteDance’s breakthrough could give its TikTok parent company an advantage in this rapidly evolving field. Industry experts say such technology could transform entertainment production, educational content creation and digital communications. However, it also raises concerns about potential misuse in creating synthetic media for deceptive purposes. The researchers will present their findings at an upcoming computer vision conference, although they have not yet specified when or which one.

Terron Gold

Recent Posts

Bitcoin Crashes Below $72,000 as BlackRock Leads $1.44 Billion ETF Selloff and Crypto Liquidations Surge

Bitcoin plunged below $72,000 on Thursday as institutional investors continued withdrawing money from U.S. spot…

4 days ago

Nvidia CEO Jensen Huang Resigns as Third Woman Emerges in Cheating Scandal Involving Alleged Chinese Spy

Nvidia co-founder and CEO Jensen Huang has resigned after 33 years leading the artificial intelligence…

4 days ago

BNY Partners With Figment to Bring Ethereum and Solana Staking to Institutional Investors

BNY (NYSE: BNY), one of the world's largest financial services and custody institutions, has partnered…

4 days ago

TRON, Tether and Chainalysis Launch New Crypto Crime Task Force After $450 Million Anti-Fraud Alliance Collapses

TRON, Tether and blockchain analytics company Chainalysis have launched the Digital Crime Task Force (DCTF),…

4 days ago

DOJ, Wall Street Banks and DeFi Giants Unite to Fight Illinois Crypto Transaction Tax

The U.S. Department of Justice (DOJ), major Wall Street banking organizations and leading decentralized finance…

4 days ago

OKX and NYSE Owner ICE Set November 23 Launch for Tokenized Stocks as Apple, Nvidia and Big Tech Opt Out

Cryptocy exchange OKX and Intercontinental Exchange (ICE), the parent company of the New York Stock…

5 days ago