Cryptomatte - OpenEXR's hot new de-facto feature

If you find it silly to be a fanboy of a file format, then count me guilty as charged: I love OpenEXR. It's elegant yet powerful, specifically tailored to the needs of the VFX industry, it's the unsung hero of the silver screen. If you read my book you already know about the numerous amazing extensions: Tiled EXR for working with huge images, stereoscopic SRX files, or unlimited embedded layers for extra flexibility in compositing. Truly remarkable. Cyptomatte is the latest - although somewhat unofficial - extension.

Cryptomattes solve the eternal compositing question: How can you select a particular material/object in a rendered image?

You see, up until now everybody was depending on Material ID or Object ID passes, more commonly known as "clown pass". By rendering a flat color for each material, the hope was to extract a selection mask for targeted adjustments. That strategy works great for texturing (i.e. in Substance Painter), because here that mask only needs to provide a starting point for further refinement. But for compositing, such ID passes never worked reliably.

Highslide JS
Example of a classic ID pass from Trollbridge


The trouble is, on the border of two surfaces you get antialiasing, and that naturally graduates through all the colors inbetween, which may happen to include colors of other masks. This cross-talk between colors makes it nearly impossible to extract precise masks for fine structures and details. Worse even, in real production scenes you may well have hundreds of materials and objects in a scene, and then the ID colors end up very close to each other - completely defeating the purpose of easy selection.

Highslide JS
Why traditional ID passes never really worked.


So the whole concept of rendering compact ID passes has always been questionable at best. In real life we either constrained ourselfes to individual alpha masks on-demand (only when the compositor asks for it), or resorted to manually bundling up 3 masks at a time in RGB layers. Tedious monkey work for the CG artist, that also inflates the file size, yet still leaves a non-zero chance to receive panicked phone calls with matte requests from the comp department.

Cryptomatte solves all that.


And best of all, it does it fully automatically. By using an auto-generated combination of ID and coverage masks, it can deal with subpixel-accuracy, motion blur, transparencies, and automatically packs all that data in the most efficient form. More importantly: it also includes a clever hashing mechanism to preserve the names of materials and objects. That means, instead of picking a color in compositing, you can literally pick materials from a list!


Originally developed by Jonah Friedman & Andrew C. Jones at Psyop, this idea was just too good to remain a secret. They published their tech at SIGGRAPH 2015, proclaimed it an Open Source project, and it has quickly turned into a buzzing grassroots-like movement to integrate it everywhere. By now Cryptomatte is the de-facto standard for ID mattes, it is readily supported in V-Ray, Houdini, Blender Cycles, RenderMan, Redshift, Clarisse, Arnold, Nuke, Fusion, Flame, After Effects. The latest addition is Lightwave 2018 support, via Micheal Wolf's EXRTrader plugin:


Big thanks to all involved! Even though my beloved Modo is not on that list yet, I'm very happy for all my fellow CG artists and compositors, whose lifes have just become a little bit easier. And it's also a shining example of a good idea sweeping through the industry, fueled by the power of Open Source, and establishing a de-facto standard solution for a previously painful problem. Go Crypromatte, Go!

Grab the plugins for your pipeline from https://github.com/Psyop/Cryptomatte.

And if you're in Vancouver for SIGGRAPH2018, don't miss your chance to meet the creators at the first ever Cryptomatte BOF meetup:
"Cryptomatte - Present and Future Uses" / Monday August 13, 3:30pm-5pm / Vancouver Convention Centre, East Building, Meeting Room 11

View Comments

Soooo close!

Troll Bridge is racing to the finish line, oh sweet victory, it's so close I can almost touch it! We called pencils down for compositors last week, and now it's just me and my trusty comp supervisor Addison polishing up the last straggler shots. From all these 350 shots we're down to 5, and those only need final finishing touches before they get handed off to the Grading Department (that is, to Tim).

As little teaser, here is a GIF of my favorite family scene.


If you're curious why we're doing all this, and what kind of person would volunteer for such a mad adventure - here is a rather extended Making-Of show. It was filmed at a Gnomon Event many many moons ago, back when we were still in the middle of animation.

View Comments

Breaking the silence, with a Trailer Release

Hello world, this is my first update in 4 years. Figured it's about time to awaken this blog from its slumber. There are two reasons for this long delay.

First one is silly stupid: a software update broke my website generator, rendering me technically incapable of publishing updates. Thanks, Rapidweaver! The problem is now partially resolved, but with the effect that I will have to rebuild the page from the ground up. Guess it was due anyways, the original design is from 2007 and isn't really holding up anymore...

Second reason is that all my time and attention was spent on crafting a short film: Troll Bridge. This is a cinematic adaptation of a Discworld story by Terry Prattchett. It was a passion project of mine for the last 15 years - in fact, work on it started long before I launched this website or even published my first HDR book. With a running length of 30 minutes, filled with 250 VFX shots, it's also my most ambitious project ever. The film has a whole family of animated trolls, a talking horse head replacement, massive digital set extension, plus a magical sword and hundreds of minor effects. Truly epic. All crowdsourced, which means hundreds of kind souls donated their work and time to get this film done. We basically formed an ad-hoc artist commune over the internet. I made many new friends, met exceptionally talented artists from all over the world, and had the privilige to lead this unique army of volunteers in my role as VFX supervisor. There are certainly enough Making-Of stories to fill several blog posts, but for now...

Please enjoy our brand new trailer!


Currently we're filing off the last rough edges on a couple of shots, but the film is already submitting to festivals. I can't wait to see it on the big screen.


Going forward, I will try to figure out what to do with this blog. HDR has infiltrated all walks of life while I was gone. Dolby Vision is everywhere (as I predicted in my book), panorama cameras had a huge revival, it's never been easier to author and edit panoramic material (including video foortage), and you can't even walk into a Best Buy without a salesman praising the latest 8k HDR flatscreen. We're already on the verge of VR light field technology, and so the amount of news item I didn't cover is staggering. Not sure if I should concentrate on catching up on pivotal developments from the last 4 years or stick to reporting on brand-new cutting edge news. What do you think? Tell me in the comments! Either way, good to be back!


Oh, and another concern is the future of Smart IBL, which is in desperate need of some maintenance to work with the latest flavors of Maya. If you know Python and MEL, and want to help out, please drop me a line!

View Comments

Tim's Vermeer and Bob's HDR pano tutorial

bob_timjenison
If you haven't seen Tim's Vermeer yet, go see it!
It's about using technology to make great art. It's about persistence and plowing through an insane project one day at a time. Things every photographer and 3D artist can relate to.

My friend Bob Groothuis, maker of the famous Dutch Skies 360 collections, actually played a vital part in this movie and was at the movie shoot in the Vermeer museum. He talks about this close encounter here, and he shot this panorama with Tim and all the cast and crew.

Bob's been very busy.

He just published Vol.5 of his Dutch Skies collection, with ... wonderful new HDRI containing all the golden light and fluffy clouds the dutch painters - including Vermeer - were famous for. All his Dutch Skies Vol 1,2,3,4, and 5 taken together are 232 skies. And the strange thing is, he doesn't seem to get tired of it. The more he shoots, the more accurate and better quality his HDRIs get.

We had a little chat about techniques and how he has streamlined his approach, and the result is that Bob sponsored this month's free sIBL-set and wrote this sweet tutorial for us. There's a lot of good stuff about HDR color management, HDR stitching, and just the general workflow of producing massive amounts of HDR panos.

I still recommend reading my HDRI Handbook 2.0 (and working through the tutorials in it), but if all you want is a quick online tutorial, then listen to the things Bob has to say.

[ Guest Article]


Bob Groothuis' big HDR panorama tutorial



Intro

In recent years I shot an enormous amount of skies here in Scheveningen (Netherlands).


Previously I made Dutch Skies 360° Volumes 1, 2, 3 & 4. Now I finished the series with the last one: Dutch Skies 360° Volume 5. It’s a bit later than planned (wanted to launch it 1,5 years ago) but I had other projects that needed attention first.

I have an external storage place and during a revision of Dutch Skies 360° Volumes Volume 2 found some panos that I completely forgot about.

You might wonder if this is the last one? Yes, because now I'm already shooting new stuff with the Nikon D800, so the resolution will be a bit bigger than the Vol 1,2,3,4 & 5. Instead of 11k I have now 19k skies. So I'm starting with a new product line simply called: Dutch Skies 360° XL (eXtra Large). The first XL skies will be available soon (probably together with brand new website & online shop). Not going to create volumes anymore, files sizes are getting way to big so the XL skies can be bought separately.

Enough of this shameless self promo let me show you some production notes/tips of the creation of the HDRLabs Free sIBL set of the month february 2014: DS360_119a. Please note that there could be better methods, I only show how I did it.


Postproduction workflow notes


The whole post production is not that difficult, but a crucial thing is being consequent. A helpful tool is a checklist where you can see exactly where you are in the workflow and be assured you keep full dynamic range and quality. A mistake is easily made so such a checklist helps a lot.

Here is a reproduction of a more extended checklist. Old school, on paper.



Highslide JS



Now let me explain some of the most important steps in more detail:

- Adobe Lightroom 5 cleaning / retouch / export
- HDRI generation with Photomatix
- Rendering the HDRI panorama with PTGUI Pro
- Bottom retouch trick
- Advanced HDRI color profile conversion method with LittleCMS Color Translator

Important note:


A suggestion that could help you: To prevent going nuts in the case your HD crashes, your office burns down or you get a visit by burglar's etc be sure you backup a lot. Here I have daily backup to an external HD. Also upload finished files to a FTP (here use Filefactory). And then I also backup to Blue Ray. Those Blue Ray backups are taken to a storage place outside the office. Almost sounds like paranoia but believe me, it’s not. You will feel very horrible when you have to redo your work from scratch. I worked for about 2 months almost full-time on Volume 5, and it would be a nightmare when I had to do it al over (or even partially).




Preparation in Adobe Lightroom 5


The source files from DS360_119a were shot in JPG and have a Nikon aRGB color profile (more about that later). To follow this tutorial you can download the original files here. Compare with the completed DutchSkies360_119a.

Lately I’ve been using Adobe Lightroom 5 for cleaning the source files, remove birds and spots on the lenses. It is inexpensive and cleaning goes pretty quick.

I’m not going to explain the whole process only mention the most important steps. If you want a more in-depth tutorial, you should check out Christian's HDRI-Handbook 2.0.

1) Use white balance as shot (with Nikon the safest WB setting when shooting is Cloudy specially when you are in a hurry).

2) Tone Curve MUST be disabled.

3) Turn on "remove Chromatic Aberration" - just default settings.

4) Used Camera Calibration - Process 2012 - Profile - Embedded.


Highslide JS Highslide JS Highslide JS


After that I cleaned out all the birds & spots with the Clone tool (this can be a bit time consuming when there is the need to clean 63 images ;) ).


Highslide JS


When that's done the images can be exported. Because the source files are Nikon aRGB we can safely export as 8 bit Adobe aRGB tiff files (Nikon aRGB and Adobe aRGB don’t differ that much).


Highslide JS


Be sure after cleaning all 63 images that you export the whole catalog (File / Export as Catalog…) that will assure you have all settings (like the cloning) saved so you can later open that Catalog again to do some refinements if needed.


HDR merging in Photomatix


We now move the Lightroom export 8 bit aRGB tiff files into separate folders, so we end with 7 folders, each containing all the exposures of every angle (the pano was shot in 6 horizontal shots and one top shot).

Use Batch Processing to create the HDRIs in Photomatix using the following settings:


Highslide JS


Try to shoot your HDR panorama source files with the lowest ISO as possible. That reduces the noise in the images. In this case I used a mild denoise in the lower exposures. Also I used a normal deghosting. Always make some test to look if all is ok.

The same time as Photomatix generates the HDRIs it tonemaps the images with the default Tonecompressor method. Those files can be used to make the stitch template in PTGUI pro (more on this in the next step).

Extra deghosting note: be careful with the High deghosting settings in Photomatix. The areas where there a fast moving objects there could be severe noise in the HDR after merging. Always check all your merged HDR's before stitching. You can retouch the bad spots in Photoshop by mixing it with a normal deghosted version.


HDR Panorama stitching in PTGUI (Pro)


In the previous step I mentioned to enable generating a tone map (compressor) together with the HDRI generation.

It goes much quicker when you do the stitch in PTGUI (Pro) with JPG files instead of HDR's. After you did the stitching with the JPG angles successfully just save that as a PTGUI template. Now create a new document (stitch) in PTGUI and load the HDR files created with Photomatix. Then apply the JPG template. Don’t forget to enable HDR Stitching in the Exposure/HDR tab.

Now save the stitch as DS360_119a_PTGUI_11k_aRGB.pts and render the HDR.

NOTE: Often you have to use PTGui’s masking feature on some parts of the sky or render the image in layers so you can do some HDR editing in Photoshop. Especially skies with lots of moving clouds can be a time consuming thing. Do not give up easy there. Be also careful with retouching the outer left or the outer right edge of the image. This is the fragile panorama seam, you can retouch/correct that part better in the next step.


Bottom retouch


For the bottom retouch I have a nice and simple trick to keep the maximum of quality. To prevent HDRI quality lost during converting the PTGUI output (Equirectangular) into cube faces .
Some converting applications can loose quality during the conversion.

Here is a trick that prevents loss:

1) Use PTGUI (Pro) to convert the Equirectangular into Cube faces (6 separate files - .hdr file format)

2) Make sure you are in aRGB color mode when you retouch the bottom part in your favorite image retouch application (here use Photoshop). After retouching save the bottom part by overwriting the original bottom part.

3) Now convert the cube faces back into a Equirectangular. Use here the tool Pano2VR.

4) Check if the file size are the same (so the original PTGUI Pro output and the new Pano2VR need to have the same size for this step - sometimes the Pano2VR output is a bit smaller)

5) Now open both panoramas in Photoshop (or any other image manipulation program) and paste them in one document. Use a layer mask to paint in the bottom part.



With this method if there would be some quality lost it’s only in the bottom part!

6) Save the comp as 1) DS360_119a_PTGUI_11k_aRGB.exr (or .hdr) and next as a 2) DS360_119a_PTGUI_11k_aRGB.tiff file - 32 bit float with Linear Adobe aRGB profile.




NOTE: .hdr cannot store profile information so you perhaps can better use the .exr format. In the case you use .hdr and want to open the .hdr in for example just assign the aRGB profile.


LittleCMS Color Translator


For years Marti Maria is known for his excellent work on the free color engine LittleCMS. Recently he made a nifty application called: Color Translator. It’s the definitive tool for converting, assigning and embedding ICC profiles in TIFF, JPEG and PNG images. A must-have for anyone working in any type of media production (print, website creation, 3D etc).

Lately I tested with my good friend Gerardo Estrada the Color Translator application successfully in an HDRI workflow. More info please see below.

So we now we have from the previous steps:


a) An 11k .hdr based on aRGB primaries that can be used by users who want to take advantage of a wider color range.

For creating the sIBL set its better to convert this aRGB based panorama into a sRGB version. So we need:

b) An 11k .hdr based on sRGB primaries that is used for generating the sIBL HDR set and for regular usage in sRGB range.



For converting the profiles we need to install the Color Profiles for Color Translator

The needed profiles for this work can be downloaded here.

You need to install these profiles in your Color profile folder. A very good manual how to do that on your platform just have a look here.

When opening Color Translator you see in the Input & Output tab on the right a black folder just click on it and select your Color Profiles folder as mentioned below.

Highslide JS Highslide JS


Now we are going to use the Translator to convert to Linear sRGB (D65):

Default input tab


Be sure you have the default RGB space set to Linear Adobe RGB (D65) – we just installed

Highslide JS


Output tab


Destination space must be set to the Linear sRGB IEC61966-2.1 (D65) – we just installed

Highslide JS


Tiff tab


In this specific case we keep the Pixel Depth set to default “Keep Original Depth”

Highslide JS


The output is now a linear sRGB (D65) FP 32 bit Tiff file that can be saved as a .hdr or .exr with Photoshop. Note that .hdr files cannot store profile but the image is sRGB (D65) based. So when opening in Photoshop you simply assign the sRGB profile (photoshop automatically assigns the correct linear profile).

The sRGB version panorama can be used for generating your own sIBL sets + tonemapping.

Extra note: This workflow was using Photomatix, Photoshop & Color Translator 1. PTGUIPro is also mentioned but for the color conversion specifically we just need Photoshop and Color Translator. Soon I will make a new workflow made with the latest Color Translator 2. We need to test it because version 2 has OpenEXR support, so most likely we can then skip the Photoshop part and do the whole conversion with Color Translator 2! The same process as described above is the same in version 2.


75 % Discount!


Proudly announce a collaboration with LittleCMS:
You can order LittleCMS Color Translator with 75% discount! Normaly LittleCMS Color Translator cost 19,95 euro (+/- 27,44 $) now you can get it for only 5 euro (+/- 6,90 $)!

More info and ordering here on my blog.

Colofon:


Many thanks to Marti Maria ( LittleCMS ) for his great discount offer & specially Gerardo Estrada who helped with the workflow research! Also thanks to Christian Bloch and HDRLabs for making this publication possible.

Legal notes:


All other trademarks and trade names mentioned here are the property of their respective holders.

Bob Groothuis

[ END of guest article ]


Post remarks


Thanks Bob, that was a really great article.
I'd like to remind all readers to download Bob's free sIBL-of-the-month. The original files (for this tutorial) can be downloaded here.
And if you want some more tutorials on shooting and stitching HDR panoramas you should get my HDRI Handbook 2.0. It has extra in-depth information and step-by-step instructions for you. Just check out the interactive table of content or read the stellar reviews on Amazon!

View Comments

Lots of free HDRIs

Frequent visitors might have noticed that there was no free HDR in November. Ever since I started the monthly giveaway in June 2007, this was the first month I forgot. Well, just to keep the chain of free HDRIs unbroken, let me make it up for you: December brings 2 new sIBLs-of-the-month!

The first one is an 16K panorama of a landmark church in my home town Halle, Germany. The church sits on a hill surrounded by a green park, which makes it a very popular hangout spot. Zoom in and pan around, and you will discover a champaign bottle, love graffiti, and bikes.


The second HDR is exactly 4 years old, but unreleased. It shows the Christmas Tree in the fancy lobby of Tokyo’s Park Hotel. That’s a super high-class hotel in the heart of the futuristic Shiodome district. The hotel starts at the 25th level, spans 9 levels upwards, and the lobby is an atrium that spans the entire hollowed-out tip of the skyscraper.



This hotel was also used as location in the movie "Lost in Translation". I could only afford two nights there, and of course I switched rooms half-way so I could shoot the cityscape in all directions. Just check out the vista:

3DWorld Advent Special: 4 Dutch Skies


And then there’s Bob Groothuis again. He gives away 4 exclusive Dutch Skies!




View Comments

Bublcam on Kickstarter: Panoramic HDR Video Cam

I have become addicted to Kickstarter. All the new VR gadgets show up there first and get mad funding. So far I backed the STEM tracker, the castAR glasses, and most recently the Bublcam.


Sure, you can get a similar result by using one of the many GoPro pano rigs. However, this won't be any cheaper because you need 5 GoPro cameras. And you'll get into all sorts of trouble synchronizing them, collecting the footage, and hiding the seams in the stitch. That's why an integrated solution like the Bublcam is much more reliable.

And while there have been several other ball-type all-in-one pano cameras proposed, this one seems to make the most sense:
  • Optimum orientation of the lenses.
  • Small form factor for minimizing perspective shift between sectors.
  • Rather 4 good quality wide-angle lenses than a multitude of cheaper ones.

Highslide JS

The bublcam captures a single 4-quadrant multiplex image. This is the raw .MP4 that comes off the camera and when placed in Bubl software you get a full spherical experience. Also, such source footage is extremely easy to stitch with regular tools like Fusion, Nuke, or After Effects.

But the deciding factor for this blog is that it will also have an HDR mode! This was made possible when the project reached a stretch goal last week.

Now, if everyone here would help giving it an extra push, we can have a time-lapse mode and maybe even double the resolution. There are only 5 days left. So, please, everyone, go ahead and reserve your very own Bublcam on Kickstarter!

View Comments
Next Page