A closer look at the rising use of virtual locations for film and TV in the last decade

In the early 19th century, directors began experimenting with ways to set their films in locations without traveling. These early pioneers used various methods to accomplish this, with varying levels of success. But none stuck until the development of chroma key technology in the 1930s.

Chroma key technology uses chroma screens (more commonly called green and blue screens) to layer two images together, meaning that as long as footage of the location existed, directors could combine it with footage of their actors in post-production. Audiences would be none the wiser about the fact that filmmakers shot the two pieces independently.

Around the same time, filmmakers also developed front- and rear-projection. These techniques are incredibly similar to chroma key technology and involve actors performing in front of physical screens onto which a background image is simultaneously being projected or reflected.

A lower-tech version of chroma key technology, these in-camera effects were used more commonly than chroma key until the 1950s. But as technology developed that made the post-production combinations easier, the industry essentially abandoned older tech in favor of the cleaner, more convincing chroma key practices.

Front-projection, rear-projection, and chroma key technologies restrict camera movement and prevent actors from interacting believably with their surroundings. And it's Hollywood's neverending crusade to improve visual effects that led to the development of virtual sets.

Virtual sets, in their current form, consist of several key elements and novel technologies. There are wrap-around LED screens that display backgrounds on three sides and above the heads of the actors. Outside the LED bubble, a series of computers run real-time rendering software that allows the virtual background to shift along with the camera angle, ensuring that audience immersion remains intact. A video mixer then combines the computer-generated images with the principal photography of the actor. And, of course, virtual/VFX artists are always on hand to ensure the computer-generated and real-life elements are blended as seamlessly as possible.

Naturally, virtual productions, and virtual sets, didn't just spring up overnight. A handful of innovative directors, including James Cameron and Peter Jackson, laterally pioneered virtual filmmaking in the early 2000s.

Award-winning production designer Norm Newberry described some of the hiccups the casts and crews faced in those early days to The Hollywood Reporter. "Part of my job was to help the actors understand where they were in this invisible world," he said. Continuing, "I was inventing the process along with 100 other people. 'Avatar' used live-action photography and motion capture, so the sets are digital, and it required a different mindset."

Despite the difficulties that virtual productions create, Newberry is optimistic the method will only continue to grow in popularity. He told the outlet: "The future of cinema in the digital world is going to be about refining the ways we work. These tricks will become easier to do; there will be better computer and camera technology that allows us to work faster, simpler and, most important, less costly."

Following in the footsteps of James Cameron and Peter Jackson's innovations, the 2013 sci-fi thriller "Gravity" was the next movie to profoundly change how virtual productions work. The film was set almost entirely in outer space. Filmmakers had to help their actors convincingly play their roles despite all sets and props being CGI and had to procure a big enough lighting source to convince audiences the characters were actually in outer space.

The solution was to create a box out of LED screens that Sandra Bullock and George Clooney could work inside. The panels of the box would show what the actors would see if they were actually floating in space and dodging space debris. This strategy gave Bullock and Clooney something tangible to work with. The images on the screens also provided a more natural and effective lighting source than they would have been able to achieve otherwise, even with any of the previous virtual production technologies.

While the film marked a considerable step forward in virtual production techniques, it still wasn't perfect—the screens were low-resolution, at least compared to today's quality. The higher-quality CGI effects had to be rotoscoped back into the frame in post-production.

One of the perks of modern virtual production is that it allows film and television productions to shoot in a single space. Instead of traveling from destination to destination, crews can settle in one studio corner and shoot a globe-trotting story.

This is handy in terms of budgets; while investing in the necessary equipment can be costly upfront, a lot can be saved in the long run when you no longer have to pay for travel for hundreds of folks.

Following the COVID-19 shutdown in 2020, projects filmed in virtual studios could return to work much faster than those that relied on physical sets and locations. Shows like "The Mandalorian" didn't need to travel around the country to reach their various locales, which meant there was less exposure risk. Additionally, virtual productions require fewer people to be present on the set as much of the work can be remote, significantly lowering exposure risk.

Virtual productions may be the future of filmmaking and TV production, but the process isn't flawless just yet. As with any other filmmaking method, there are definite downsides, including shooting restrictions caused by the limitations of the technology and the overall believability of the finished product.

Virtual productions force filmmakers to work in an entirely new way. With virtual productions, much of the work must be completed ahead of time, requiring more planning and forethought for every second of the project.

Additionally, the skillsets necessary to run the technologies virtual productions require are advanced, meaning there could be worker shortages unless the workforce accommodates the gap. Ultimately, technologies simply aren't advanced enough just yet to ensure that every virtual set will look as believable as its real-life location.