Welcome, Guest. Please Login or Register
Home Help Search Login Register
Page Index Toggle Pages: 1
Send Topic Print
Interesting paper (Read 7344 times)
davidb
Ex Member


Interesting paper
04/07/11 at 12:05:43
 
I was reading a paper on "Artifact free high dynamic range imaging" by Orazio Gallo et al yesterday. 

It looks like an interesting approach to zero ghost merging, with a possible cost of reduced dynamic range of the output. 

I'm almost tempted to put tone mapping and image processing on hold for a bit and give it a go.

The basic approach seems to be to pick the best possible reference image, and go for absolute consistency with it. 

This makes ghosts impossible in areas where the reference image is well exposed, but I'm still not 100% clear what the best approach is for areas where it is saturated.

One possibility may be to build on the algorithm described and find another reference image for saturated and dark bits of reference image, and build some sort of evil data structure that allows me to force consistency at the boundaries between segments of different images.
Back to top
 
 
IP Logged
 
Blochi
Administrator
*****
Offline



Posts: 1726
Hollywood
Re: Interesting paper
Reply #1 - 04/07/11 at 16:34:06
 
Well, that's the general "hero image" strategy that most ghost-removal algorithms use. It fails miserably when you have a guy with a white shirt and black pants walking through frame, because there is no single well-exposed image. On the book DVD you will find such a sequence.

A much more interesting approach I found just recently - was gonna write a blog post about it....

http://www.mia.uni-saarland.de/Research/SR-HDR/index.shtml

It's using Optical Flow techniques to analyze the image, and attaches a motion vector to each pixel, with sub-pixel precision. So instead of suppressing the ghosty appearance in the secondary exposures, it nudges the pixels over to match the placement in the hero image.
Back to top
 
WWW  
IP Logged
 
davidb
Ex Member


Re: Interesting paper
Reply #2 - 04/07/11 at 16:57:25
 
I find the best solution to someone walking across your scene is a gun and the willingness to use it.  Clone out the bodies later Smiley

I read some optical flow based papers a while ago.  I like the way that they can handle alignment as well as ghosting in one pass, but I was never entirely convinced that it could be done reliably in complex scenes without generating  false positive matches.

This one does look interesting though, and I have printed out a copy to read later.

The results you linked look quite impressive but the images are very low res, so it's hard to see just how well it is really working.


My view is that the way to go is to try to build a hybrid of several approaches.  The holy grail would be to have a good, fast algorithm that works most of the time but be able to spot edge cases and fall back on more complex techniques automatically.

One of the reasons I quite like the look of the paper I linked is that there are a couple of obvious extensions that may be able to give me an oracle that can spot some of the cases when it isn't working.
Back to top
 
 
IP Logged
 
Blochi
Administrator
*****
Offline



Posts: 1726
Hollywood
Re: Interesting paper
Reply #3 - 04/07/11 at 18:29:48
 
Well, you didn't actually link the paper Wink

Ultimately, I think algorithmically you can only take it so far. The moment an algorithm has to guess, I would always prefer if it asks the user for confirmation. That's where user-guides approaches are very appropriate, and even Photoshop's "tag your hero image" helps a lot.

Where Photomatix sux, however, is the user interface of selecting the ghosted areas. Lasso selections are rather clunky. The new masking tool in PTGui, stroke-based is much better. DPHDR has a similar ghost selection...
Back to top
 
WWW  
IP Logged
 
davidb
Ex Member


Re: Interesting paper
Reply #4 - 04/07/11 at 19:22:53
 
Oops.  I certainly intended to.  It is here:  http://people.csail.mit.edu/kapu/papers/GalloICCP09.pdf

Maybe this has led to a more interesting question - given that all attempts for a mere machine to solve the problem completely are likely to be doomed, what would your ideal ghost removal UI look like?
Back to top
 
 
IP Logged
 
Blochi
Administrator
*****
Offline



Posts: 1726
Hollywood
Re: Interesting paper
Reply #5 - 04/07/11 at 20:00:16
 
I thing a good starting point would be the UI from PTgui, for masking out things that are not supposed to be considered in the stitch. Very simple: Red marks something, Green means "keep this for sure".

...

There just should be a simple way to tell each ghosted area which is the preferred hero image. Something better than the Right-click text menu. Ideally, it would be a mashup with Photoshop, showing Thumbnails of all input images, and a green frame as selection. And depending on the ghost you're removing right now, it would allow you to select a different hero image, and it would show the result right away.

PS: Picturenaut also allows "creator" plugins, hint hint....
Back to top
 
WWW  
IP Logged
 
davidb
Ex Member


Re: Interesting paper
Reply #6 - 04/08/11 at 10:01:00
 
That seems sensible.  One of the algorithm extensions I had in mind was a way of automatically selecting "hero" images for regions that are saturated or black in the main "hero" image. 

The output from that would essentially be an image with a channel specifying the image ID.

Assuming cheap and easy tone mapping (exposure, filmic etc.) you could probably paint in one window and have a live update in another.

Maybe the brush could show the image you currently have selected for "painting".
Back to top
 
 
IP Logged
 
davidb
Ex Member


Re: Interesting paper
Reply #7 - 04/14/11 at 13:43:47
 
I'm happy to implement the merge it if someone else wants to do the UI.  For alignment I'm happy enough with the align_image_stack from hugin.
Back to top
 
 
IP Logged
 
davidb
Ex Member


Re: Interesting paper
Reply #8 - 07/05/12 at 10:06:17
 
Seems a bit daft to resurrect such an ancient thread, but I have finally started working on some of this stuff.

I currently have camera response extraction, naive hdr merge (no ghost removal), a slightly buggy multigrid solver, about half of a tile analyser for ghost zapping and a pile of notes about the rest of my design.

Considering I have been at it for about 2 weeks I think that's not bad.

Short term plans as follows:

1. Single threaded image merge tool with ghost removal, but no alignment.  Use cases: HDR merge & astrophotography* stack noise reduction.

LDR case probably about 70% done.  HDR probably needs about 50% more work.

2. Extend with selective deghosting (gui required at this point).

3. Enable concurrency when it's all nicely debugged.  This should be easy enough as I can just plug in the STM threading framework.

The deghost algorithm has a couple of interesting features.  For starters it seems very sensitive to the accuracy of your camera response recovery.  You can forget about using it with a generic gamma curve unless you are OK with just getting an HDR version of your hero image.  This might not be so bad with the selective deghost approach I have my eye on, but it's worth considering.

Anyway, that's what's been going on in spectral3d land of late.

I'm also still really pleased with the d7k I picked up last year Smiley

*I reckon you can do 2 alignment jobs - one on sky, one on foreground, throw the two incompatible stacks together, pick a hero image and let the deghosting algorithm sort it out.
Back to top
 
 
IP Logged
 
alexandra
Ex Member


Re: Interesting paper
Reply #9 - 07/12/12 at 02:48:17
 
I almost always even after deghosting have problems with the eyes of people, any tip on how to correct that?
Back to top
 
 
IP Logged
 
davidb
Ex Member


Re: Interesting paper
Reply #10 - 07/16/12 at 17:03:33
 
Some tests on a lw dynamic range stack of 8 images.

http://www.spectral3d.co.uk/Vapourware/deghost/

HDR deghosting is mostly done, but I need to put some more work in.
Back to top
 
 
IP Logged
 
Page Index Toggle Pages: 1
Send Topic Print