Introduction

Separating actors and props from a green screen scene is a critical function for any virtual-set or mixed-reality project. The aim is to:

  • identify and isolate the foreground elements from the background screen color
  • preserve:
    • fine edge details such as hair
    • transparency of translucent objects such as glass bottles
  • remove any spill color from the actors
  • recover the shadows from the actors and props

The foreground objects need to be completely solid where there is no transparency.

The quality of the produced key is highly dependent on a good keying setup. 

Here are some basic rules.

The green screen

  • The screen should be evenly colored if painted. Ensure a proper chroma green paint is used, and that the surfaces are free from brush marks
  • The screen must be stretched tightly to avoid creases if it is made of fabric
  • Ensure the floor color is the same color as the walls, and that it is clean and without marks
  • The walls should be at least 3 m high to allow keying of standard-height actors - taller props or actors would require higher walls
  • The walls should meet the floor seamlessly without creases or shadows. Often, coving is used at the junctions to smooth the transition
  • The walls should be lit evenly with a wash of light and with no visible highlights or hot spots
  • The floor should be lit evenly
  • The subjects should be picked out from behind by a rim light to give them some separation from the screen
  • The front of the subjects should be illuminated with beauty or fill lights

The camera

  • Ensure that the camera has the correct lens and that it is positioned in the desired position in the set
  • It is normal to have the camera at the same height as the actors' heads so that it is easy to key the actors if they move some distance from the green screen towards the camera
  • Make sure proper white balance is set
  • Make sure the scene is correctly exposed
  • Ensure the camera is not set to a high gain, which might introduce noise
  • Check the camera's "clean" video signal on a monitor to ensure there is minimal noise in the image

Noise is an inherent part of any camera video image, and minimizing this is critical for good images as well as creating a good key.

The Talent

The main issue is to not pick up too much spill color from the environment onto the actors, which would make keying more difficult.

  • Do not position them too close to the walls, and do not let them get closer than 1 m 
  • Light them from behind to create good edge definition
  • Light them from the front to create attractive lighting, and to remove any under-illuminated hollows in skin and clothing that could show the screen color
  • Position them far enough from the camera to be fully framed with the choice of lens and zoom setting
  • Talent should not wear any clothing that includes the same color as the screen color

Video keyer

The video keyer is an easy-to-set-up real-time chroma keyer that you can use to key your video input. It is a component implemented inside the Pixotope Video Pipeline. The keyer extracts an alpha mask from the received frame, based on information provided to the algorithm, and removes specific color (despills) for the background of the scene and its reflections in the image. The user interface for the keyer is included in the Pixotope Director under Production → Adjust → Video Keyer. The UI and keyer component communicate over the network.

Initialize

The first step is to set the reference color(s) for the keyer. The colors are automatically extracted from the labels provided in the labeling tool. The labeling tool is located in Video Keyer → Initialize. 

Adjustments

Refine

The Refine category remaps the alpha values so the user will be able to suppress values with low transparency and increase values with high opaqueness. The functionality is driven by two threshold values and linearly interpolated in between. See the image below.

Key color intensities

By default, all colors are considered equal, but as for example yellow is quite close to green mathematically, it can be useful to key certain colors harder than others (and even make colors repel keying). With these sliders you can change how aggressive the key will be towards the three color channels individually. Negative values mean that this color repels keying.

Mask Adjustments

Erode

Removes pixels on the mask boundaries, making the mask smaller.

Blur

Smooths the mask and its edges.

Gain

Adjusts the transparency of the mask.

Gamma

Scales the alpha by an exponential function.

Example of gamma transformation with different exponents.

Cutoffs

The Cutoff controls help the user to exclude the area that is to be opaque from the keying process. The exclusion is based on the color deviation and the color saturation. Both parameters are computed independently and are added to the computed mapped alpha (in the diagrams below, Lt and Ut represent Remove Background and Adjust Foreground respectively).

Displayed functionality of Refine adjustments.

Shadows

The Shadow controls allow the user to add shadows that exist on the evenly lit backdrop. Shadows are added based on the luma of the pixels; any pixel darker than the original pick will be included in the shadow, the alpha strength depending on just how dark the pixel is. The "Add shadows" slider controls the alpha of the shadows, while "Adjust contrast" controls the color, trying to darken the fill in the area where shadows are identified.

Despill

The Despill controls allow the user to suppress the background color removal and adjust hue for the areas containing a significant amount of spill.