So in my last journal, I got in a small side-discussion with someone over how many colors and things the human eye can perceive, and how it relates to displays and movies and games and the like. Given that "HDR" is a kind of current buzz word, and a lot of people say "I can't tell the difference between white and the faintest gray on my monitor, we have enough colors" I feel that I should clear things up from a perceptual perspective. There area number of factors in play here that you need to be aware of.

First, let's consider the dynamic range of the human eye. The brightest thing you can look at without damaging your eyes is on the order of 10^7 candelas per meter squared (10 times as bright as the filament of a 100-watt lightbulb) and the dimmest thing you can see is 10^-6 candelas per meter squared (as bright as a black piece of paper illuminated by only starlight, no moon). That means the brightest thing you can look at versus the darkest thing you can see vary by a factor of 10^13, or 10 trillion.

Of course, that's not to say that your eye or brain can resolve the difference between 1000000 photons and 1000001 photons. The amount of contrast you can resolve varies based on your age, personal factors, the level of illumination, and the distance over which it varies. For a 20-year-old, a 200:1 or 300:1 contrast ratio seems to be pretty good. My perception professor says that 1000:1 is about the limit, which disagrees with some of the graphs I'm seeing, but who knows. Some people can see much better or worse than this. A piece of white paper with text on it has a ~30:1 contrast ratio, which is about as good as you can get with even illumination over a scene.

Let's compare this to a monitor--we'll use mine and call it typical. It advertises 500 cd/m^2 in brightness and a contrast ratio of 1000:1. Assuming that full brightness is 500 cd/m^2 that would put minimum brightness at .5 cd/m^2. But if you'll recall, the human eye can see values from .000001 to 10,000,000 cd/m^2, meaning that this display could both be improved by showing brighter brights and darker darks. Even in a setting which won't require your visual system to spend 30 minutes dark-adapting, things range from 1 cd/m^2 to 10 million cd/m^2, so there's room for improvement.

But let's get back to contrast. If you can tell the difference between something which is 1/1000th brighter than something else, and a monitor only has 256 levels of lightness, it's clear that it has only about 1/4th of the number of steps it should need to span that range. So only storing colors in 256 brightness values isn't enough. Let's pretend it is, though--let's say a person CAN only see something with 1/256th the difference. Why would we want to store any more values than that?

Several reasons. First, consider taking a picture inside of a church: the stain glass windows will be letting a lot of bright sunlight through, but the corners in the top of the roof, and any architectural details up there, will be in a great deal of shadow. If we only have 256 values to go between a really bright color and a very dark black, we have basically three choices: we focus on the dark details so we can see them and anything above a certain threshold just becomes white (a long exposure), we focus on the light details so we can see them and anything below a certain threshold just becomes black (a short exposure), or we try to map the darkest dark to black and the lightest light to white and we end up with very visible borders between brightness levels. Were we actually there, we could let our eyes adapt to the light or the dark regions, revealing either the fine details in the dark corners, or the fine details in the stain glass windows.

And in fact, for screens with a very high dynamic range and higher bit-depth images, that's exactly what can happen: you can look at a part of an image with a large contrast range, and your eyes will literally adjust and adapt to the brightness range you're looking at to bring it into clearer focus.

What's another reason you might want something--say, a photograph, or a texture for a game--to have more than just 256 levels of brightness? Well, usually with these things, you can do some degree of editing to them. Say we took a picture of that church before. If we used a large bit-depth, we could crop to just the dark corner of the ceiling and turn up the brightness (like our eyes adapting) to see fine details. If we only have 256 levels of brightness, all the dark values will probably be within a couple of each other, hiding fine detail. The same idea holds for a texture in a game, where lighting effects can be applied.

So what you'll sometimes see in games these days is advertising or options for HDR, and that typically means that they're doing tricks to make the lighting look more like what you would see in real life. Obviously if you're only using 24-bit color in a game and your monitor only supports 24-bit color, you can't truly get brighter whites and darker blacks, but keeping track of an extended range of colors with better precision lets lighting effects and calculations be carried out in a more realistic manner, without just cropping values when they're too high or too low. The game can, of course, choose to do certain cool things with this, such as light bloom (anything brighter than white floods out a little into neighboring regions), or simulating the change of pupil size, which Crysis and Half Life 2 do (so when you go into a darker area, the details come into focus, and light areas become washed out--and when you go into a bright area, things are initially bright and washed out, before settling back down to normal levels).

As display technology gets better and better, with contrast ratios in the ten-thousands-to-one range and with ever-brighter backlighting, these issues will become more apparent. Indeed, with a good bright display, if you try to use the full range of brightness values with a standard 24-bit color photograph, you CAN notice the steps between colors, and it can actually look better to tell the display to dim itself down, oddly enough.

If any of this stuff interested you at all I suggest you take a look at this Wiki article on a new JPEG format, which supports better color depths--I believe it even links to discussions which say some of this stuff better than I can.

Just feel that I should keep you guys in the know, and I'm learning this stuff, so why not share it? =D