For the 2011, 2012 & 2013 Tonys, we were tasked with digitally recreating and animating theater sets that can be manipulated live during broadcast, for each of the performing nominees.
We started the process by shooting thousands of photos (and several gigs of reference video) inside the various theaters, in order to capture lighting changes and set elements. The photos were then carefully mined for textures and scenic pieces that were needed. Finally, the finished video-based “sets” were painstaking constructed from this large bank of imagery and then animated in After Effects and Cinema 4D to the specifications of the set & lighting designers.
Lots of hard work and long hours were put into separating & layering the video content so that we could do things like dynamically shift hues and cue lighting/set changes during the performances.