Saturday, April 20, 2013


If you haven't seen this comic from the Oatmeal about mantis shrimp, please, go read it, now.  Because it is awesome.  It also has its problems, like for example the rainbow being a terrible example of how mantis shrimp vision is awesomer than ours.  That is our subject for today, because it's actually really interesting and not at all obvious why the rainbow wouldn't be a thermonuclear bomb of light and beauty to the mighty mantis shrimp.

But first, a primer on color vision.

If you are like most intelligent people, you have made it surprisingly far in life without properly figuring out how color vision works.  It's time to have The Talk!

Exhibit A: This is the color wheel we all learned about in kindergarten:

The color wheel is a sensible way of organizing the relationship between colors: red and yellow make orange, yellow and blue make green, blue and red makes purple.  That's what actually happened, when we mixed our fingerpaints together.

Exhibit B: This is the electromagnetic spectrum we all learned about in high school:

This is also sensible: light is a wave with a wavelength, so of course it should live on a line with varying wavelengths.  And again, orange is between red and yellow, green between yellow and blue, and purple between blue and...wait, what?

A paradox.  Is it a circle or a line? I mean, I know that in reality light lives on a line.  But I also know that when I mix red and blue together I get purple.  How does that make any sense given the above spectrum for visible light?  The color wheel's existence seems deeply inconsistent with how nature works.


And now, the answer to the paradox, which somehow almost nobody knows, even though we are intellectually curious people and vision is our most important sense.

When you look at an orange sweater, a bunch of light of various wavelengths comes into your eyes.  The question is how you process all this light and boil it down to the perception of a color.

The truth is that there are infinitely many different wavelengths coming in at varying intensities.  How much 500nm light is there? How much 501nm light? 501.5?  It's too much to completely record all this info.  So you take a massive shortcut.  In fact your eye has only 3 types of color-sensing cells: red cones, green cones, and blue cones.  A red cone is most sensitive to red light, although nearby wavelengths also trigger it to a lesser degree.  Similarly for green and blue cones.

All these infinitely many wavelengths coming in, then, are basically distilled down to three numbers, (R,G,B). How much are the red, green, and blue cones firing?  That's what gets passed on to the brain.
Observation: The pixels on your computer screen do not bother with other wavelengths besides red, green, and blue light.  Since our eyes are going to collapse incoming light to (R,G,B) anyway, this minimizes energy costs without even losing any generality.  Orange light is an inefficient way to stimulate red cones.  
The brain, in turn, makes up a color perception for each possible combination of (R,G,B).  The orange sweater triggers a big response from the red cones, a medium response from the green ones, and a small response from the blue cones.  The brain identifies this as some sort of orange. 

Note that our little (R,G,B) system is more than sufficient to uniquely identify each color in the rainbow.  That is, shine light of any one wavelength into your eye, and it will trigger a unique combination of (R,G,B) cones firing.

However...there are other combinations of (R,G,B) that no color in the rainbow will trigger...

For example, (high,low,high).  Suppose our red and blue cones are highly triggered, but the green cones are not.  No single color in the rainbow would do that...but why limit ourselves to single colors?  After all, a bunch of red wavelengths and blue wavelengths could be coming in, without any green.  It could happen.

And that is the sensation our brain has labeled "purple."

Colloquially we tend to think of "violet" as a purple, so let me be clear: There are purples that cannot be found anywhere in the rainbow.  Magenta, for example, but also other purples that are more similar to violet.  Pink, too, is a color you won't find in the rainbow.  Lots of red, but also a fair amount of green and blue light.

Actually, there are a lot of (R,G,B) combinations that won't be triggered by any single wavelength of light.  "All the colors of the rainbow" is not, in fact, all the colors.

So we perceive a lot more colors than are in the rainbow.  But even so, we do lose a lot by compressing the incoming infinitely many wavelengths down into just 3 numbers.  Necessarily, many different combinations of wavelengths will get mapped to the same exact color perception by our brains
Observation: We are tangentially now able to answer the age-old question of whether people "perceive colors differently."  The naive answer (which everyone gives) is that your "red" might be different from my "red" but that we would never be able to verify these subjective experiences.  But actually it should be easy to prove that colors are perceived differently.  Just find two bundles of wavelengths that trigger the same (R,G,B) experience in Person A but not in Person B!
For example, let's label:
Bundle A: 10 units of 400nm light + 20 units of 450nm light
Bundle B: 15 units of 370nm light + 24 units of 470nm light 
Each bundle of wavelengths triggers a color perception.  The idea is to carefully choose the bundles so that Person A perceives Bundles A and B as the same color, while Person B does not.   
This is a clever solution because it eliminates the need for subjective comparison across people.   Maybe Person A and B have the "same" subjective perception of Bundle A.  Maybe they have the same perception of B.  But these can't both be true simultaneously, if they also disagree about whether Bundle A and B are different! 
We all map many different light combos to each color, but surely we don't do so in the exact same way.  (Fact: my own left and right eyes don't even agree with each other...). 

OK.  So yeah, we are throwing away a lot of information about the world, compressing all the incoming visible light into 3 numbers.


Adding cones can accomplish two things.  Adding IR or UV cones will expand the range of visible light.  Adding more cones between existing cones doesn't expand the range of visible light, but it does increase the number of light bundles we can distinguish and hence the number of colors we perceive.

With our 3 cones, we can pick out combinations like (high,low,high) that have no analog in the rainbow.  As the number of cones grows, the number of distinguishable combinations which have no analog in the visible spectrum...grows exponentially.

With its sixteen cones, the mantis shrimp presumably sees a special color corresponding to
among many, many other colors.

However.  He does not see this color when he looks at the rainbow.  A rainbow is a collection of single wavelengths, not combinations!  The mantis shrimp may see some IR and UV around the edges of the rainbow, but overall his rainbow-viewing experience is not much different from ours, at least not on the basis of his extra cones.  It's not a THERMONUCLEAR BOMB of light and beauty.

The extra colors he sees in the rainbow are proportional to the expanded width of his visible spectrum.   It seems to me that this is nothing compared to the exponential increase in the number of different colors he can perceive as a function of having over 5 times as many cones as us.

Disclaimer: I am not a mantis shrimp.  I am not even a human with formal training in mantis shrimps or mantis shrimp optics.  The above represents only my current level of understanding of how color vision works.  If you are a mantis shrimp and you do equate rainbows to thermonuclear bombs of light and beauty, by all means, drop me a comment below!  I accept no liability for any keyboard destruction brought on by angry mantis shrimp leaving angry internet comments with their smashing claws.


  1. Color is like counting. The more you know about the subject, the harder it gets.

    For me, I think the hardest thing to reconcile is RGB versus the RYB on the typical color wheel. It's confusing because when you learn the RYB color wheel in art class you are learning a SUBTRACTIVE color system (when you add the three primaries you get black). When talking RGB you are usually talking electronics and you've switched to an ADDITIVE color system (when you add the three primaries you get white).

  2. Ryan,

    Yes. I think this is one of the main benefits of actually knowing how color vision works. It frees us to make sense of color systems we *didn't* learn as children. We have RYB hardwired into our brains. But then when we go to a different color system, we don't quite know what to make of it.

    I too have asked why RYB is used in art class and RGB in computers. I too have been told the magic words, "additive vs subtractive colors." But that doesn't really resolve the mystery, does it?

    The fact is there are many possible sets of additive and subtractive "primary colors," not just one. Hardwired knowledge aside, the additive RGB system is actually the simplest paradigm to understand, together with its complementary subtractive system that is equivalently easy to understand, namely Cyan, Magenta, Yellow (CMY). Cultural perceptions aside, RGB and CMY are the *natural* choices for additive and subtractive primary colors, based directly on the color receptors in our eyes.

    The CMY system used by many printers is actually *better* than RYB. CMY are the colors exactly "between" RGB. Cyan subtractively blocks out Red while letting Green and Blue through freely, and similarly Magenta blocks out Green, while Yellow blocks out Blue.

    The logic of mixing Cyan and Magenta is then that they directly block out Red and Green light, respectively, meaning only our Blue cones are triggered, meaning we see Blue. Yes, you're subtracting instead of adding, but the underlying logic is no different. So instead of reconciling RYB with RGB, maybe it's better to switch completely over to CMY and RGB!

    Again, there are many possible sets of primary colors. We could use orange, green, and purple paint if we wanted (that's right! Secondary colors as primary colors! Kindergarten mind blown!). It wouldn't be quite as good as CMY, but the qualitative mechanics are the same. So, the real answer to the question "Why RYB in art but RGB in computers?" is: Yes, why RYB in art? It's more due to culture than to "additive versus subtractive colors."

    As a disclaimer, I am not a painter so I'm not really qualified to say what are the best primary colors for painting. Mixing paints is probably more complicated than just overlaying cyan and magenta light filters. RYB could turn out to be a really good choice of primary colors for painting.

    Also, it is not literally true that RGB screens and CMY printing can reproduce every possible color perception, although they are pretty darned good. Ryan is correct that the deeper you get into color, the harder it gets...but let's stop here.

  3. I encourage my students to use CMY for mixing colors, at least in my Color Theory class. It will get them to the secondaries and tertiaries much quicker--especially purples--and I generally discourage overly opaque pigments in general such as the Cadmiums. I also encourage them to use Unbleached Titanium to lighten their colors instead of Titanium White, saving it mostly for highlights, as "pure" white tends to desaturate the color too quickly..

    In my Painting class, however, it's pretty much anything goes. The most important thing there is to be aware of the difference between mass and undertone, and opacity and translucency, and whether they want to directly paint (mix colors on the substrate itself) or indirectly (mixing it all on the palette first) or a combination of the two. Beyond that, I encourage them to buy whatever colors they need to save them the most time so they will look less at their palette and more on the image they are working.

  4. Dain, thanks for the comment! Very informative.