Professor and media artist James She urges businesses to invest in Artificial Intelligence (AI) infrastructure and services for the media and film industry. He explains how emerging AI technologies for video generation can put Hong Kong on the map again for cutting-edge and immersive film production. This is not just due to OpenAI’s Sora video generation tool, because plenty others are coming out from China and elsewhere. These are revolutionizing film production, and Hong Kong has a chance to play a leading role again if it acts now. Read on…
The emergence of generative AI technology heralds a pivotal moment for video, media, and film industries. Its transformative capabilities have become increasingly significant, offering unparalleled opportunities for innovation and efficiency. And the landscape is evolving with unprecedented speed.
Leading the charge is Sora, a generative artificial intelligence model developed by OpenAI, that specializes in text-to-video generation and more. Sora employs advanced machine learning techniques to craft high-quality videos that seamlessly emulate real-life cinematography styles while adapting to diverse artistic preferences. Sora is not fully operational yet as of this writing, although the industry thinks the likely full launch is in this coming summer. As Hong Kong’s video, media, advertising, marketing, and film-related industries strive to reclaim their previous global prominence, embracing the transformative potential of generative AI is imperative.
Hong Kong’s vibrant film industry of the 1990s was left stranded in an increasingly technological era that requires serious capital investment and technical expertise. Many of Hong Kong’s studios have moved across the border, and the creative talent that produced the blockbusters of previous decades now lacks the resources to adapt to the capital- or space-demanding technologies. But there is a silver lining. Collaboration with the mainland is our strong suit, and mainland production houses are moving in the same direction as Hollywood. But it still does not answer the question, who could better use the exciting AI tools to tell Hong Kong’s, Chinese and Eastern stories?
The groundbreaking OpenAI’s Sora technology, and its counterparts, not only streamline many video production processes but also unlock new avenues for creative expression and audience engagement. These can be deeply rooted in the unique local culture and history of Hong Kong. Hong Kong can also serve as a dynamic connector between China and the world. By harnessing the power of generative AI, video- and film-related businesses in Hong Kong can position themselves at the forefront of digital content creation again, ensuring relevance and competitiveness in an ever-evolving landscape.
A primer on OpenAI Sora
Decoding the AI Behind OpenAI Sora OpenAI’s Sora stands as a testament to the cutting-edge capabilities of generative AI technology. Powered by a unified representation of visual data, Sora can train generative models across a spectrum of duration, aspect ratios, and resolutions. Unlike earlier AI models limited to shorter and less variable content, Sora achieves a significant milestone by generating videos lasting longer and in higher fidelity. This feat is made possible mainly by a synergy of diffusion model with the object coherence capability, transformer architecture with recaptioning and spacetime latent patching techniques, each contributing to Sora’s prowess in crafting intricate, dynamic and detailed visual narratives from user textual input and more.
Crafting Text-to-Video Transformation with OpenAI Sora
Input Processing The journey of text-to-video transformation commences with the user inputting a descriptive text prompt outlining the desired video content. Sora’s sophisticated models meticulously analyze these prompts, deciphering complex elements such as narrative structure, visual details, and emotional nuances.
Content Generation With a comprehensive understanding of the input text, Sora’s AI initiates the synthesis of video frames. This intricate process involves simulating realistic scenarios and visual elements aligned with the text, ensuring each frame’s coherence and contextual relevance to the narrative.
Refinement and Output Following the initial generation phase, Sora meticulously refines the output to enhance visual continuity, quality, and realism. The resulting video is rendered in high resolution, offering a visual experience that can be previewed and further edited as needed.
Through this step-by-step process, OpenAI Sora revolutionizes the landscape of video creation without much upfront capital- and space-demanding investment, empowering filmmakers and content creators with unparalleled tools to bring their imaginative ideas and visions to life quickly.
Sora is still in its infancy. OpenAI just announced Sora in February 2024, and there are already a multitude of AI tools similar to Sora in the market. At least several companies in mainland China and other countries have produced something comparable. ChatGPT in the beginning produced very short text content, but two years later it is writing much longer content, from term papers to movie scripts, and has empowered many ChatGPT-based chat businesses. Although the short duration of a Sora-generated video may seem like an obstacle, in the hands of skilled movie producers it is no the issue at all. The typical film or advertisement is made up of short episodes, and Sora is likely to be deployed in the very near future for these.
How generative AI can help the Hong Kong film industry get its mojo back
Hong Kong was once hailed as the Hollywood of the East. Its heyday was in the 1980s, when it produced iconic stars like Bruce Lee, Jackie Chan, and Chow Yun-fat, whose names continue to resonate globally. Despite its illustrious history, the Hong Kong film industry faces contemporary challenges that until the advent of generative AI seemed insuperable. It survived and remained relevant by embracing collaboration with mainland China.
Hollywood has dominated the 21st century with its major studios and streaming services. The technological prowess of the US film industry led it to global success, by leveraging advanced technologies backed by substantial technology investments and a robust talent pool in both technology and film sectors. Hollywood’s relentless drive for innovation culture and continual adoption of new technologies and techniques has made it possible to enhance film production efficiency and quality for both local and global markets.
Yet the US film industry also faces hurdles. Like Hong Kong it is under pressure to meet the escalating demand for high-quality content and continuous innovation.
Unlocking opportunities with generative AI
Integrating generative AI technology into workflows heralds a multitude of benefits and opportunities for stakeholders in the video, media, and film-making industries, especially in Hong Kong, where the film sector faces challenges in competing on a global scale.
At the forefront of these advantages lies enhanced cost-effectiveness and time efficiency. Generative AI can streamline many and soon most of the production processes by automating repetitive tasks and accelerating content creation, thereby reducing operational costs and allowing businesses to allocate resources more strategically. Moreover, generative AI unlocks unparalleled creative potential, empowering artists, filmmakers and even individuals to explore new creative styles, experiment with innovative storytelling techniques, and push the boundaries of visual expression. At the same time, the investment for producing high-quality and sophisticated video and film is becoming more affordable and flexible.
These opportunities are available to the global film industry but serve a particular purpose for Hong Kong. Generative AI video needs far less up-front capital- or space-demanding infrastructure than the old studio model. In the past, film studios required enormous space for their sets and large-scale equipment. In the future, generative video technologies will be able to create whole geographies and interiors on the Cloud or in virtual environments for content production. That eliminates a major disadvantage for Hong Kong, which is limited space and initial capitals.
Historically, Hong Kong’s film industry has lacked the scale and resources of its US counterpart, which has long been known for its robust infrastructure and technological prowess. Generative AI presents an opportunity for Hong Kong to bridge this gap by leveraging AI-driven technologies to enhance production efficiency and creative output, narrowing the gap between the two film industries. It creates the possibility that Hong Kong could become once again a formidable player in the global cinematic landscape.
Harnessing generative AI for film and media innovation in Hong Kong and the region
OpenAI’s Sora may not have been launched yet, but companies like New York-based Runway ML and Beijing-based Hidream.ai are already offering similar technology. Hong Kong’s universities boast high-quality education and research performance on a global scale, offering a strategic advantage in training cross-disciplinary expertise in computational media and art. For example, my university, Hong Kong University of Science and Technology (Guangzhou) in Nansha is the new home base for HKUST’s cross-disciplinary training and research, which includes computational media and arts.
Last year, the Hong Kong government proposed putting HK$4.3 billion into the Film Development Fund (FDF) and the CreateSmart Initiative, including two new initiatives under the FDF – the Filming Financing Scheme for the Mainland Market and the Hong Kong-Europe-Asia Film Collaboration Funding Scheme. This strategic approach aligns with the current timing of Greater Bay Area initiatives, positioning the region to capitalize on the opportunities presented by generative AI for future media and film industries as well as other video-related businesses.
Across Asia-Pacific, urbanization and digitalization are driving demand for innovative solutions in filmmaking. Generative AI presents a transformative opportunity, enabling filmmakers to explore new creative avenues, reach diverse audiences, and stay competitive in a rapidly evolving landscape. Businesses can leverage AI to navigate challenges and seize opportunities, shaping the future of content creation in the region as part of the digital transformation. With generative AI as a catalyst, Hong Kong’s film industry can thrive, forging new paths and redefining storytelling in the digital age.
For those eager to embrace AI-driven innovation for film and video making, adaptability and foresight are paramount. Staying abreast of emerging trends, investing in AI talent and infrastructure, and fostering a culture of experimentation and collaboration are indispensable to remain at the forefront of the AI revolution and seize its myriad opportunities. Moreover, there is an urgent need to invest, participate, and collaborate with leading universities, startups, and organizations already immersed in research, projects, and talent cultivation in generative AI for video and filmmaking. This includes leveraging expertise in computational media and art within the Greater Bay Area. By doing so, we can gain early momentum.
There is a big gap still in resources for training in the new skills and mindset for generative AI. Why were Kungfu Panda or Mulan, basically Chinese stories, produced in Hollywood? The short answer is that Hollywood has skills that Hong Kong lacks, not that Hong Kong and China lack stories. Indeed, some of the greatest novels in world literature are Chinese, Shui Hu Zhuan (Water Margin) and Hong Lou Meng (Dream of the Red Chamber), so why not tell these stories using generative AI?
As we navigate this evolving landscape, Hong Kong’s unique position as a bridge between the Greater Bay Area and the rest of the world presents unparalleled opportunities for leveraging generative AI. Let’s seize the moment to embrace the boundless potential of generative AI. Together, by harnessing AI-driven technologies, the film and video industry and creative talent in the Greater Bay Area have a chance to establish Hong Kong and the Greater Bay Area’s leadership in the regional and even global cinematic landscapes, usher in a new era of creativity and innovation, and serve as a catalyst to global collaboration.
Dr. James She is a media artist and an associate professor in computational media and art at Hong Kong University of Science and Technology (Guangzhou). His research interests focus on the uses of artificial intelligence and multimedia for art, media, culture and human creativity. He has been a visiting artist at Fire Station Museum, Qatar Museums in 2020, and his artworks have been exhibited in Qatar, UAE, South Korea, Australia, Thailand, China and various other countries. His art may be found here. Learn more about OpenAI’s Sora here.