What you will learn

Here you will learn how to do a simple virtual studio production using:

  • 1 tracked camera
  • 1 computer
  • 1 Pixotope Live license

Or check out the following links to get an overview of Pixotope, its concepts and components.

Schematic

  • Camera system
    • Camera
      • The captured video comes in via an SDI input on the video card
      • Signal gets keyed or just passed through into the Pixotope Engine
    • Tracking system
      • Tracking data comes through the network card
  • Pixotope Tracking Server translates the data and sends it to the Pixotope Engine
  • Pixotope Engine translates the real camera into a virtual one
    • The rendered virtual scene gets merged with the real scene
    • The scene gets output through the video card
  • Pixotope Director is the main software you interact with
    • Can be used locally or remotely
    • It controls all aspects of Pixotope
    • The Director is connected via the Data hub to the other components
  • Pixotope Data hub
    • In a multi-camera multi-computer setup, the Data hub connects all Pixotope computers with each other. See multi-camera setup.

Example projects using Pixotope

Pixotope lets you merge the real and the virtual world, in real time and in perfect quality.

To see what Pixotope has made possible, visit https://www.futureuniverse.com.

Or try it out yourself and download our example projects.

Check beforehand

Start Pixotope Director

The Pixotope Director is the main interface for setting up, adjusting and controlling a Pixotope virtual production.

  1. Start Director
  2. Log in
  3. Choose a Live license

The interface

  • The main menu on the left is divided into 2 different views:
    • The SETUP view is used for setting up the machine together with camera and object tracking
    • The PRODUCTION view is used for adjusting running levels and also controlling them through your control panels
    • The "Launch" panel is used for browsing your projects and launching levels
      • This panel is present in both modes
  • The user menu in the top right-hand corner gives user- and license-related options
  • The status bar at the bottom shows all the servers and Pixotope computers connected to this Data hub

Learn more about the SETUP and PRODUCTION views.

Learn more about how to 1.5 | Starting Director.

Set up the machine

Set up the machine together with camera and object tracking. 

Create a project

All Configure and Calibrate steps are stored per project and on each Director machine individually.

You can easily import a previous setup from another project or another machine. Learn more about Importing a setup.

Add a camera system

Specify your camera setup:

  1. Choose the assigned tracking server
  2. Choose your camera's film back size
  3. Choose your camera's lens aperture

Configure camera tracking

  1. Choose the assigned tracking server
    1. On your camera tracking hardware, enter the Tracking Server's IP and port and start sending tracking data
    2. Check the status field for incoming tracking data
  2. Choose the camera tracking protocol and if necessary change its default configuration
  3. Choose the camera mount
  4. Choose a lens file if available

When doing Calibrate → Tracking, you might have to come back to change some of the configurations.

  • If the camera movement is mapped wrongly, change the mapping
  • If the camera movement moves in the right direction but is scaled, check the "unit" configuration

In case you need object tracking, check out 1.5 | Setting up object tracking

Configure video inputs and outputs

  1. Set the project's compositing color space
  2. Configure the input format of the camera system
  3. Add video output and configure its format
  4. Add additional media inputs (optional)

Configure routing

Recreate the physical routing of your machine.

Test video and tracking configuration

To test whether your configuration was successful, let's launch a level and see whether camera tracking and video input and output are configured correctly.

Launch level

  1. Go to the "Launch" page
  2. Expand the current project
  3. Launch in Editor to open the Virtual studio template level that was created for us

Learn more about the different ways to launch a level: 1.5 | Edit, Preview Live and LIVE mode.


When you launch a level for the first time, it can take a bit longer to launch as shaders and other elements have to be prepared.

Calibrate tracking and syncing

This step is about matching the real with the virtual world. The placement and movement of the real-world camera have to exactly match the placement and movement of the virtual camera.

This is a 2-step process:

  1. Match the location in Calibrate → Tracking
  2. Match the timing of the video and tracking signal in Calibrate → Syncing

While this should primarily be handled by the tracking systems and their operators, Pixotope allows real-time adjustments to correct any misalignment or synchronization issues.

Prepare a simple VS level

For a virtual studio, we need to place the talent in our 3D environment.

  1. Film the talent with a tracked camera in front of a green screen
  2. Key the video feed with Pixotope's video keyer
    • The keyed video is projected onto a compositing plane
  3. Position the compositing plane in the virtual studio


Learn more about AR and VS levels in 1.5 | Setting up a Pixotope project.

For artists using the Pixotope Editor:
Learn more about how to 1.5 | Preparing levels in Pixotope Editor.


To perfectly match video and 3D graphics, use Pixotope's adjustment panels.

Key video and adjust compositing

Getting keying and compositing right is another important part of creating a realistic virtual production.

Key video

The internal keyer is an easy tool to set up a real-time chroma keyer which is used to key our video input.

  1. Select the camera system you would like to key
    1. The camera system input type must be set to "Internal Keyer" in SETUP → Video I/O
  2. Initialize the key
    1. Click "Background"
    2. Label the background area by drawing a couple of strokes
    3. Click "Initialize Key"
  3. Use the "Show alpha on SDI output" checkbox to toggle the alpha mask on the SDI output
  4. Refine the key


Adjust video composite

The level has a compositing plane and the keyed video input is mapped onto the plane.

  1. Select the compositing plane in PRODUCTION → Adjust → Video Composite → Plane specific
  2. Position the compositing plane in-depth
  3. Scale it to cut away surrounding unkeyed garbage areas
  4. Tweak the Z-position so the keyed video intersects nicely with the floor and realistic shadows are cast on it
    • If no shadows appear, make sure you have a directional light set up and the compositing plane has "Cast virtual shadows" activated
  5. Go to the "General" tab in PRODUCTION → Adjust → Video Composite

  6. Fine-tune the compositing plane

This can also be done directly in the Editor.

Adjust effects

The camera effects and color grading possibilities for the 3D graphics, as well as for the video, are the final things you would adjust. Try it out yourself.

Adjust color grading

Grade your 3D graphics and video so the 2 different sources match.

Create a simple control panel

  1. In the Editor: Expose the properties you would like to control

    Learn more about how to Expose properties.

  2. Create a new control panel
  3. Create a trigger widget
  4. Add a target action
  5. Save your control panel

Learn more about how to 1.5 | Creating a custom control panel.

Run live

  1. Launch your level in LIVE mode
  2. Use the control panel to control your level

Learn more about how to 1.5 | Running live production.

Data integration and API

For data integration into and API control of Pixotope levels, see 1.5 | Advanced - Pixotope API.

More

Find additional information here: