Thursday, March 10, 2011

Rules versus objectives

You've probably heard this one before: The principal has some complicated objective he wants the agent to pursue, but he can't get the agent to care about this objective directly. Instead, he can only create a bunch of rules that approximately incentivize the right behavior.

We face versions of this problem all the time, right? Any time we wish upon a genie, to cite one common example. Or, teachers may want their students to develop a deep understanding of the material, but they can only design exams that push students to learn approximately the right sort of stuff. (Some are more successful than others). Employers face a similar problem motivating their employees to do what's best for the company. The government, too, has complicated social objectives, but all it can do is create a set of laws.

The thing is that objectives are rarely simple, but rules generally need to be. Simple to communicate, simple to follow, simple to measure whether people are violating the rules or not. Therefore rules are biased towards being simple, whether the objective that generated them is complicated or not.

Now, with that as a prelude, what interests me is the problem of inferring objectives from rules. What if I, a lowly agent, actually care about the objective, but I can't observe it directly? What if I can only see the rules? Well, given the above, I should be especially leery of inferring simple objectives just because the rules are simple -- after all, the rules would be simple either way!

To be concrete, perhaps I am a benevolent citizen of society, and I'm trying to act in a way that is best for society (i.e. the actual objective of a benevolent government). I know what's legal, but not necessarily what behavior is best (for society). What can I infer from the laws? Hmm.

Well, here's something I can't infer. I can't infer that the laws themselves perfectly represent society's true goals. Is it socially optimal to come to a complete stop at every stop sign? Whether or not it is, you can bet the law will say "Come to a complete stop," because that's the simple statement closest in meaning to the true goal. So, reversing this, we can't infer from "Come to a complete stop" that a complete stop is actually what society wants us to do, right?

I worry that when the objective is not readily observable, people have a tendency to fixate on the rules themselves, to revere the rules as if they are the ultimate goal, as if following the rules is what makes you a "good" citizen. But legality is not the true benchmark of what's best for society, so it does not deserve our reverence. If you roll stop signs, on some level you may feel like you're committing a minor infraction against society -- an infraction that is perhaps justified by your right to care about your own wellbeing -- rather than recognizing that it is probably optimal for society if people don't always come to a complete stop. Society spends a lot of time inculcating this idea, "Obey the Law, it's the right thing to do," but we should really move beyond it where we can. Society doesn't really want people to focus on obeying its laws, just like teachers don't really want students to focus on learning the particular material they can feasibly be tested on.


As long as we're here, I will just post a provocative question. Like societal law, religion generally offers simple rules for how to behave. In this context people are very interested in figuring out the actual objective of God, so perhaps they should entertain the possibility that the rules may not perfectly represent his true objective. For many, the rules become the "definition" of right and wrong according to God. But might he not believe in a more complicated notion of right and wrong and still find it optimal to send the same message? Is what he says actually sufficient to determine what motivated him to say it?

Here I am being quite loose, so objections are expected.


  1. In response to your closing provocative question, I don't see how a close reading of the Bible could lead to a different conclusion. At one point, God commands "Thou shalt not kill" and in another time and place "Thou shalt utterly destroy." "Thou shalt not kill" is a very useful rule of thumb. God gave it as a commandment, and He expects his people to follow that commandment UNTIL TOLD OTHERWISE. But I digress on the topic of modern revelation...

    Also, Jesus tended to teach guiding principles rather than hard rules. He often clashed with Jewish leaders who wanted strict adherence to rules to define righteous living. Christianity has the doctrine of the Holy Ghost or Holy Spirit, which inspires Christians to know how to apply gospel principles in particular situations. So, at least within Christianity, I think there is certainly this recognition that God has a higher set of laws than the ones he has revealed to us in scripture. Thus access to the Holy Spirit and, dare I say it, living prophets is essential. (Shoot, there I go again...)

  2. Ian,

    Good. Although you are probably one of the more rational religious people out there, so I don't think of this as a sweeping defense of religious beliefs. A lot of religious people seem to have come to very different conclusions, no? That's an empirical claim; my measurements could just be off, in which case I would like to know.

    Question: in a sentence or 2, what's the qualitative difference between your "living prophets" and, say, the catholic pope? You talk about "living prophets" as if this is a central, controversial issue. (I'm going to guess that's why your church went and put "latter day saints" in the title). Just to say it, you certainly don't have to worry about sparking controversy here. I'm curious but I don't have a dog in this fight.

  3. So if you asked a Catholic, they would tell you that the Pope is not a prophet. While they consider his official statements authoritative, they would draw a distinction between Pope Benedict and, say, Moses or Isaiah. As for what the precise distinction is, I don't know that I'm qualified to say. But I think it at least has to do with the fact that official statements of the Pope are not considered scripture as the writings of Moses and Isaiah are.

    On the other hand, the LDS church has an open canon. We believe in modern day scripture revealed through modern day prophets just as ancient scripture was revealed through ancient prophets. For us, this modern day scripture carries equal weight with ancient scripture. Similarly, we believe that Christ is literally leading the LDS church through His current prophet (and 12 Apostles) just as was done in the New Testament. That we believe this quite literally, and not just metaphorically, bothers some people.

  4. OK.

    My understanding is that the Catholics believe the pope is in direct communication with God. His purpose largely *is* to be a latter day direct link, and he is treated as infallible because they believe he really is communicating God's will -- not just old news, but modern updates as well. I don't know if that would make him a "prophet" as the word is used, though if I were in charge of definitions, I would define a "prophet" as *anyone* who gets information from beyond the physical world. I wonder if the disagreement is semantic.

  5. Hi Xan,

    I just discovered your blog. The sentiment of this post sounds oh so Aristotelian. I think that rules and incentives often produce undesirable and unexpected side effects. Economist Fred Hirsch once said, “The more that is written in contracts, the less can be expected without them; the more you write it down, the less is taken, or expected, on trust.” When you tell a student that they only need to learn "x" for a test they will probably do the bare minimum. When you tell them the objective is to broadly learn, I think you will often see a different result.

    In order to answer your provocative question I would, of course, have to assume that there is indeed a God.

  6. Greg, thanks for the comment! There is something to that, perhaps I will write a post on it soon. I'm hoping to figure out if rules change the outcome for any other reason besides (a) which outcomes they leave feasible and (b) what *information* they convey.

    As a funny example, I once discovered that if you try to implement an optimal cake-splitting algorithm when there's actually more cake than people want to eat, they nevertheless focus intently on maximizing the size of their pieces.

    As for assuming there is a God, as a theory-head I am happy to condition on any hypothesis if it might lead to an interesting line of logic :)