Saturday, October 24, 2020

Is Colorization Part of Digitally Remastering a Film?

A friend recently asked:

When they say they are digitally re-mastered, does that include the colorization technique [I had previously commented] about?

I responded: 

The ability to manipulate and even infer missing information (replacing missing frames of film, correcting physical damage mold, fading of dyes and the image-recording structures themselves) in the digital realm of computers makes makes them invaluable for working with all recorded media forms: still images, sound and moving-pictures. In these very mature disciplines, original content is converted into the digital language of computers. This is often referred to as "digitization," and includes such things as transferring the sound from a 140 year-old Edison Phonograph cylinder to scanning your children's baby photos from the original negative, to all the motion pictures in history. When you watch a movie on television (and back when we used to go to movie theaters, almost all of which have converted to digital projection), the original film was digitized to be transported and exhibited using digital devices. 


So "digitally re-mastered" is a bit of marketing hype, since no one would consider the alternatives to working with old films in the old analog realm if they could avoid it. Before there was digital manipulation of media, each stage of manipulating the content likely imposed "generational loss" on the facsimile. If you make a copy of a copy of a copy of a photograph, or an audio tape recording, or of a motion picture film in the analog realm the way all things were done before the last 40 years, the integrity of the original content suffered - it didn't look or sound as good as the original. Every movie you ever saw until the 1990s was degraded by at least three or four generations of the film production process by the time you saw it in the theater. "Signal" - the part you want - is increasingly displaced by "noise" in the process. In the digital workflow, you only have to manage noise during that first step from the real world of light and sound to the digital realm. After that, there can theoretically be zero loss of the original - 100% integrity (in practice there is quite a bit of loss incurred to economically transport the content to the user with a deliberately-designed, minimal amount of noticeable degradation).


Ideally, "re-mastering" a movie would mean getting access to the original negative film that went through the camera, and has been in storage in a secure, climate-controlled vault for decades. As mentioned previously, this one-of-a-kind recording media contains quite a bit more information that ever got to the Big Screens on the hundreds or thousands of mass-produced "release prints" that were shipped to theaters, after being copied from yet another generation-old "Intermediate Prints" used to protect the precious Camera-Original Negative (which is worth however much it cost to make the movie - so potentially hundreds of millions of dollars) - dozens of these intermediate prints were distributed to mass-duplication labs to be copied to Release Prints. 


(Today's theaters can receive movies over broadband data connections, rather than having heavy cans of film shipped around the world. Furthermore, these "prints" can not be scratched and broken over time. Remember what it was like to see a movie that had been at the Janus Theatres for two months? They looked like someone had dragged them across a gravel parking lot.)


In addition to the camera-original neg and visual effects negatives, an ideal re-mastering effort might attempt to access original sound production elements. These would included dialog recorded on the set (independently of picture), music scoring sessions, and sound effects. While these would have been meticulously edited and mixed on potentially hundreds of channels of audio, they would have been performed on equipment from the last 90 years, and older equipment would have imposed more noise on the results. So rather than use the pre-edited, pre-mixed soundtrack, a re-mastering team might consider (if available) re-mixing from dozens or hundreds of already-edited tracks of sound-striped 35mm film, or they might simply use that as a reference, and if the source materials were comprehensively archived, replicate or even improve on the original work by starting from scratch while using the completed materials as a reference. In the case of sound, one might create a stereo or surround-sound version of a film which was originally released as monophonic (although many enterprises create stereo versions of mono soundtracks with nothing but the final media - I once interviewed with a company whose primary business was stereo conversion). 


Using ever-evolving computational algorithms, the tools with which moving images can be altered continue to be improved and invented. And with them, so does the ethical question of where one crosses the line from "remaster" to "remake." If someone cleaning a world famous painting decides to paint a repair over a chipped, missing section of the painting, is that OK? If a recording of a famous singer's performance is missing, is it OK to replace that section with a piece synthesized with a sample of their voice? Or by using another singer?


In re-mastering, it is possible not only to "restore" the visual intent of the original filmmakers - their expectation of what would be exhibited in theaters, generational loss and all - but to deliver far more of that camera original information to the viewer. 


It's also possible to deliver MORE than was originally recorded, expected or intended. From any given film source material, a modern project could: 

  • Deliver higher resolution than the original camera recording by predictively synthesizing additional image information never recorded, but inferred by surrounding recorded information
  • Be presented at a higher frame rate than the original (early silent films were hand-cranked at 12 to 18 frames per second, and play back with motion too fast in modern 24 fps systems) by synthesizing digitally interpolated intermediate frames
  • Have color information never recorded
  • Be stereoscopic (3D), having never been shot with two separate lenses (most of today's "3D" movie releases are no longer shot with complicated and bulky stereo camera rigs, but are created independently by production businesses which uses software tools and tedious labor to create adequately believable stereoscopic images)

As you might imagine, these choices are polarizing amongst film historians, nerds and fans. Back in 2006, I attended a CBS Paramount Domestic Television presentation of the first completed "digitally remastered" episode of the original 1966 Star Trek series. They were planning to remaster all 79 episodes, and their product is uniquely valuable, having effectively never been off the air - perpetually in syndication. The really exciting prospect that I had been speculating to a Trekkie friend about for decades was that we'd see this show in High Definition (that was the year we bought our first HDTV), with far more image information that was ever intended. I predicted being able to see nails and taped seams in the set walls. Those shows were shot in the same 35mm film as theatrical motion pictures, even though they were only going to be viewed on the crude television system which we all used until the last decade. 


I don't remember if we knew that this was part of the project until that night, but the shocking part of the Star Trek "remastering" project was not that they had re-scanned all of the 35mm film to make beautiful high-definition masters, but they had replaced nearly every visual effects shot with a modern, digital replacement. Did they look better? Yes, perhaps. Were die-hard Trekkies/Trekkors/Trekkers going to be upset? Yeah, a lot of them were. But Paramount/CBS was looking to inject some new life into the property, and propel it into their licensing future. So they did it anyway. 


You may be aware of revisionist alterations of movies that have been "remastered" - the single most famous revision is "Han Shot First" (ask a Star Wars nerd if you don't know what this is about). 


With the power of modern tools comes the responsibility of utilizing them. We have to make good choices and respect the work of artists and craftspeople who invented and created the works we are tasked to curate and protect.