No matter the size and breadth of a traditional video production, one of the most important findings is that a startling number of the extras you'd traditionally find on a movie or television set aren't actually required to be there. Director, DP, Producer, Camera Operator, Actors, Gaffer, Production Designer, etc. are just a few of the crucial positions that are frequently required. In the world of virtual production, the list is mostly the same, but there are a few new roles on the roster. The majority of personnel, which are referred to as the Brain Bar collectively, are technicians needed to set up and manage this novel technology.

Rapid alterations and adjustments are to be expected, especially since this new model has had such a significant impact on manufacturing at every level, given that virtual video production with LED quantities is still very much in its infancy. New workflows will include the knowledge gained from experiences in the real world. The use of tools will be improved, updated, or even changed. There will be a strengthening of abilities and a finding of new avenues.

 

What is meant by virtual production?

Virtual production is a technique for making movies that merges the virtual and real worlds.

The green screen is certainly something you've seen before. It's a tool that editors frequently employ in the backdrop of a shot to make it simple to add visual effects later on.

The green screen is swapped out by a huge LED screen in high-end virtual production. These screens' high resolution renders any films or images that are displayed on them to appear extremely lifelike.

Virtual production techniques include using LED live walls as one example. A couple more are performance recording and visualization.

 

Where virtual production first began

Front and back projection techniques served as the foundation for virtual production.

The 1930 motion picture Liliom was one of the first to employ back projection. 2001: A Space Odessy, directed by Stanley Kubrik, was the first film to use front projection. The 2013 video game Oblivion made excellent use of this technology as well. On a big white muslin cloth, a steady video of the sun and moving clouds was displayed while the movie was being filmed.

Performance capture and visualization methods including previs, postvis, and techvis are additional types of virtual production technology.

A computer-generated 3D character is created via performance capture, which records an actor's performance. In the 2002 Lord of the Rings film The Two Towers, Andy Serkis' actions as Gollum were initially captured using the technology.

Let's now discuss visualization approaches in more detail and explain what each of the three techniques is.

1. Previs is a step in the production process. It involves storyboards, storyboard animation, concept art, and anything else that aids in the development of the movie's plot.

2. Previs is followed by techvis. The technical elements of previs, such as camera settings, shot locations, and essential green screen dimensions for CGI scenes, are handled by techvis artists.

3. In post-production, a process known as postvis, live-action scenes are combined with short-term visual effects to act as stand-ins for the final cut.

Tim Burton's Batman Returns, which was released in 1992, featured an early iteration of this previsualization. The movie Gravity from 2013 was the one to advance this technology and visualization methods:

Gravity - With the aid of extremely finely detailed computer graphics, the sequences in the film Gravity were shot by shot previsualized. The Light Box, another remarkable advancement in VFX (visual effects), was also included in the movie. This container stood 20 feet tall and contained 4096 LED lights.

These lights were programmed by the crew with moving images of Earth and space. The performers could see what they were responding to in the film when they glanced at the visuals that were displayed by the LED lamps. Additionally, the lights gave the right lighting for various images.

Gravity pushed the limits of visual effects; the performers' faces were the only really real objects in the majority of the big shots.

Later films like Rogue One and Solo also employed the same Gravity-inspired technology for dynamic lighting effects to illuminate the actors' faces. Additionally, these movies made use of huge laser projectors and projection screens.

The 2019 The Lion King movie's development saw one of the biggest advancements in virtual production. Cinematographers used real cranes, Steadicam rigs, and drones to operate virtual cameras throughout the whole movie, which was shot entirely in a digital environment.

The Mandalorian, a contemporary virtual production, uses LED walls or volumes. Hyperrealistic visuals are displayed on these walls. This technology was also employed in shows like Star Trek: Discovery and the upcoming Netflix German series 1899.

 

Virtual Production Recordings' advantages

1. Lower Expenses

The custom of everyone arriving in the morning, leaving at night, and occasionally hanging out on set until they are needed is also broken by remote working. (Although being compensated.) Now that they have more flexibility, remote workers may choose to return part of that flexibility to you.

Virtual production eliminates the need for real sets by generating digital backdrops. Because much of the labor is done while filming in real-time, it also lessens the need for post-production and reshoots. All of this significantly lowers production costs.

2. Decreases the Requirement for a Physical Location

Cast and crew do not need to travel to a physical place for a virtual production. The crew also doesn't have to worry about external illumination and weather fluctuations that could interfere with the production schedule because everything is digital. 

It's safe to say that some of us would prefer not to commute, deal with rush hour traffic, or have to drive home after a long day at work from a human standpoint. It makes sense that those of us who have successfully recovered our lost travel time during the pandemic era and set up productive remote working environments at home will be hesitant to give up these advantages.

3. Working from Home

Since not every participant must be present on set while shooting, virtual production permits remote work.

It's possible that they would have previously added a charge for transit time and expenses, but they no longer have to do so as a line item. Perhaps they might provide hourly fees rather than daily or weekly rates. Without a doubt, you won't need to account for trip expenses and lodging expenses. Going remote could therefore quickly reduce your manufacturing costs and increase the agility of your initiatives.

4. Efficiency of post-production

Pre-production, production, and post-production are only a few of the many processes in the filmmaking process, which is sometimes compared to factory-line production.

Sadly, this system is ineffective. Any production error lengthens the post-production process, and vice versa. For instance, the director can film the scene improperly if he or she has trouble visualizing the VFX creature that is supposed to be in the shot.

Without LED walls, the video production process may be too uncertain. Additionally, fixing errors in post-production can be expensive and delay projects.

Virtual productions, on the other hand, let you make changes in real time. The director can request a real-time adjustment from the graphics team if he doesn't like the virtual creature on the LED wall.

5. Decreased Set Time

Actors may find it challenging to see the final shot when working on a green screen. LED walls, on the other hand, give the actors a real-time view of their surroundings, reducing the need for retakes and the length of time spent on set.

6. Creativity

Virtual productions provide countless opportunities for creativity.

For example, VFX specialists can quickly alter the glow of the sunset while filming. Or the sun could be entirely turned off. They can make the mountain appear larger if the one on the horizon doesn't appear to be intimidating enough.

7. Overall Business Benefits

The advantages for video production companies are just as strong. Being able to cast your net across a much wider talent pool is more advantageous than only hiring locally, regardless of whether your goal is to identify the greatest talent possible, the least priced candidates, or simply those who are available during your project window.

 

Which Positions Allow Remote Work?

The size of the project and the technology used will determine whether a crew member can work remotely or not. It goes without saying that you will need a pair of hands to move anything that needs to be physically moved, whether it's replacing a damaged panel, altering the volume's shape, or moving a camera. Even so, some physical apparatus can be built first and then managed remotely.

1. Directors

The majority of the time that directors spend on site is already spent gazing carefully at monitors before giving performers performance criticism based on what they have just seen. Therefore, it is debatable how much of this can be done using a video conferencing application and a livestream from the camera. (Some of the latest episodes of the Disney+ series Obi-Wan Kenobi were done in exactly this manner.)

Of course, it might not be possible to mimic the director's manner if they like to approach the actors and demonstrate the physical parts of their roles. However, not every production necessitates intense actor-director communication. It may be sufficient for the director to be able to watch and provide feedback from a place off-site for low-key projects, talking heads, presentations, education, and corporate work.

2. Operators of the Environment, Engines, and Animation

These Brain Bar members have among the most evident off-site jobs because there isn't much physical activity they need to engage in. They'll need to see both the camera's video feed and an expanded view of the full scene that they can control from anywhere.

Consequently, if someone points at the top corner and requests that something be changed to a different color or that an entirely new environment or lighting style be applied, someone can easily step in, and effect the necessary changes in a matter of seconds—whereas, before now it probably would have required several stagehands under the direction of the set designer, with cherry pickers, safety harnesses, and health and safety supervisors in place.

The old lengthy physical switchover between sets, which may last days or even weeks, is replaced by a complete volume background manipulation and change in just a few short seconds.

3. Lighting crew and LED techs

Although these roles differ in actuality, they are essentially somewhat similar in concept. After installation, the lighting and LED systems can both be controlled remotely. A well-designed lighting rig can be managed remotely, and depending on the size of your production, you might feel more comfortable having someone on set who can move things around when necessary. However, with a full-scale volume, there is nothing to move.

4. Digital Imaging Technician

For some time, the focus of the DIT position has shifted away from managing physical cards, shipping drives, and daily workflows toward the flow of data.

Making ensuring that the original camera files and any other captured material reach their destination has always been the goal. But the tools needed to accomplish that aim have undoubtedly changed; RED's in-camera Frame.io integration is only one example of how the media pipeline has rapidly transformed.

Therefore, your DIT would be better off managing your selected cloud-based pipeline, organizing the OCFs, system access, and footage curation throughout the stakeholder hierarchy, rather than juggling media on set. As soon as your cameras are connected, you can do it from anywhere.

5. Actors

Although it may seem like I'm pushing the meaning of "remote" here, there are times when virtual production can turn the performance into a distance-based activity. Simply not in the manner you may anticipate. You don't trust me? What if I told you that none of the actors who portrayed members of the Illuminati in Doctor Strange in the Multiverse of Madness shared a single space?

Imagine for a moment that one or more members of your cast are unable to travel to your virtual production site due to their location abroad or an issue with their schedule.

Imagine being able to communicate the set design and lighting configurations to a similar LED volume later or closer to the location of your talent. Even though you recorded your actors' performances in separate nations at various times, you may conceivably create a sequence that sets your actors inside a scene convincingly if you're careful with your framing decisions and there's little physical touch between characters.

 

What does a virtual production setup require?

Although the equipment needed to make high-end virtual productions function is a vast improvement above what you'd find on a typical set, the equipment required to make the remote portions of virtual production function is simpler.

The following are the essential items to buy if you want to set up a virtual video production environment.

1. Technology for tracking cameras

To track camera movement in relation to the LED wall, a virtual reality headset and tracker like the HTC Vive Pro and Vive Tracker are required.

2. iPad and iPhone

Thanks to their LiDAR sensors, Apple's most recent smartphones are perfect for recording in-engine camera movement. The iPhone camera's movement in space is picked up by these sensors. If you want to avoid buying a camera tracker, this is really practical.

You can move your Apple camera around and have it change camera angles in the virtual scene with the aid of the Unreal iOS app and these sensors. For this to function, both your PC and Apple device must be connected to the same wireless network.

3. Actual Cameras and Lenses

You can use any camera to capture the image, but if you have the money, we highly recommend the Blackmagic Design URSA Mini 4.6K Pro G2.

4. Camera Rigs

You will want a camera rig so that you may attach extras to the camera, such as the VR tracker. The Neewer DSLR Shoulder Rig is what we advise.

5. Capture Card

Live camera feeds can be streamed to a computer using a capture card. This is required so that the real-time footage can feed easily to the game engine.

A high-end production card would be the Black Magic Decklink 8K Pro, while an entry-level card would be the Elgato Cam Link 4K.

6. LED Walls and Volume

Numerous banks of LED panel are used to construct LED walls and volumes. An RTX Quadro-style graphics card powers each panel.

For those on a tight budget, we advise the Nvidia Quadro 6000 RTX, while the Nvidia Quadro 8000 RTX is recommended for high-end productions.

Small production houses only need an LED wall; larger production houses need a more expensive LED volume, which is recommended.

7. Green Screen

If money is an issue, consider using a green screen instead of an LED wall. Just make sure the illumination is uniform throughout the screen and there aren't many creases.

8. Unreal Engine

You will require a game engine, such as the industry-standard Unreal Engine 5, to build the digital environment. Use the Unreal Engine marketplace if you require digital world assets.

Both Unreal Engine and Unity are essential parts of virtual production setups and, in the hands of a pro, are able to do amazing things. They are also developing quickly, so keeping up with everything could require some training. Because they were originally designed to be game development engines, they also offer multi-user manipulation of active projects and environments. As we all know, today's games are made to allow for seamless communication between numerous remote players. Therefore, this fits perfectly into the remote production workflow.

9. High-Speed Internet.

You'll need a strong and reliable connection with all of this data flying around. It's reasonable to expect that if you work in a reputable virtual production facility, they will be able to provide you with the bandwidth you require, frequently with fiber to the building. Bonded modems and a respectable mesh network will still get you where you need to go even if that's not the case or if you choose to keep your set and communications networks separate—especially if you're in a location with 5G coverage.

However, do yourself a favor and configure a wired connection to your router or switch. While wireless internet is practical, it also raises the possibility of packet loss, range limitations, and connectivity problems—neither of which you want on a multi-hour shoot day.

 

What steps go into virtual production?

Let's examine the process of virtual production. In a nutshell, the following will essentially occur in a virtual production:

1. The camera, the camera tracker, the LED wall, and the game engine must all be set up before filming can begin.

2. The filming has now begun.

The scene must first be made in the game engine by the visual effects team. Let's assume that the desert is the intended setting for the scene. As a result, the group will have to build an engine rendering of the desert.

When that is finished, the desert render will be shown on the LED screen.

3. The actor is then positioned in front of a screen that depicts a desert in a render. The actor appears to be in the desert when the camera captures him with the LED screen in the background.

4. The visual effects crew can assist in making any lighting or color corrections that are required in real time. Real-time editing in this way lessens the need for post-production editing.

This is essentially how virtual production is carried out.

 

In Conclusion

We learned a lot about adaptability while we were under lockdown, and the principles we learned are transferable to working virtually on video productions. The fact that a new way of working is only getting started may be the best thing. There has never been a better opportunity to specify our methods of operation and the positions we decide to take up in this fascinating new arena.

Being in the right place at the right time can still determine whether or not you get the job. Just as "the right time" no longer necessitates a two-hour trip, "the right place" can now be anywhere with a respectable internet connection.