"
. With Put that in your algorithm
Recently I read Stephen Baker’s book The Numerati, all about “the mathematical modeling of humanity,” and Bob Garfield’s long ode to data mining in Ad Age, which strikes very similar themes about the power of algorithms.
More recently I listened to a 60 Minutes report about the recent troubles on Wall Street and beyond (via podcast). Here’s a bit taken from the news show’s online textual recap of that segment:
These complex financial instruments were actually designed by mathematicians and physicists, who used algorithms and computer models to reconstitute the unreliable loans in a way that was supposed to eliminate most of the risk.
“Obviously they turned out to be wrong,” Partnoy says. [Frank Partnoy, a former derivatives broker and corporate securities attorney, who now teaches law at the University of San Diego.]
Asked why, he says, “Because you can’t model human behavior with math.”
Hm.
What say you? Can human behavior be mathematically modeled, or not?
.
.
Kim Fellner's book 
.
A
Reader Comments
Interesting thought. I believe that human behavior can be calculated and predicted with a formula, but it would have so many different variables that it would be impractical to even consider it. Anyway, great website and I am planning to buy the book, so keep up the good work.
You can model human behavior with math, in the same way you can model incidence of disease in populations, sunspot activity, or any other complex system. The question is whether there is enough predictive accuracy in these models to be useful, and more importantly, whether relying on these models poses a risk when they turn out to be wrong. Modeling is about defining what is most likely, but doesn’t prescribe what to do when an unlikely outcome occurs.
Well human behavior can be calculated, but only to a certain extent. Actually, it’s probably better vernacular to say human behavior can be predicted. The advertising industry shows this. For example, advertisors know that young people are attracted to cool things, so if young people are the target market then it is likely that the market will behave with a positive response if the ads effectively demonstrate the product as cool. However, a great deal of human behavior is random, and as Joseph said, there are too many factors in society to get specific mathematical figures. Predictions can be accurate, but the percent error would be too high for correct calculations.
I think that you can model human’s past behavior, but it is not smart to use that to predict their future behavior. Nassim Nicholas Taleb wrote a great book, Black Swan, which addresses this idea.
He uses a great example, which I’ll try to paraphrase. A turkey get fed every day. If he were to model his last 100 days, life is great. Then comes, Day 101, which is Thanksgiving.
The model of the first 100 days was as accurate as possible, but it only considered known events, not all possible events. Just like the turkey, the economic models are flawed because they are only take account for their limited observations, which is a small subset of possible events.
Black Swan is highly recommended, and gives insight on how these “impossible financial disasters” of “perfect storms” of events should not really be that surprising.’
Not sure that the problem is whether we can model human behavior with math. The problem was inherent to the risk management models instead. I read somewhere that the trouble with all those market-sensitive risk models was that they assumed each user is the only person using them. When you have a herd of people using the same risk model and flocking to the same supposedly undervalued/low-risk instruments, they become the very opposite of what the models say.
Thanks for the book tip Rob. Numerati sounds intriguing.
People reduced to numbers…mmm.
bonnieL
Ray, you’ve got it exactly right: Black Swan covers this topic so well that it’s just best to tell people to go and read it.
I think you can do the barbell strategy with human behavior: find the core subsets of behaviors that make up the majority of everyone’s personality and then try to find the 5% of really unusual stuff that distinguishes some people significantly from each other.
We are not all unique snowflakes, but no predictive method out there right now or in the near future (other than the occasional human for a brief time) can accurately predict what a person or people will do with a high rate of success.
Yeah, I’m of course familiar with Black Swan. I’m not sure I see what Partnoy is talking about is really a failure of the models to anticipate a surprising event, but rather a more across the board failure that actually *caused* the crisis. And re what Garfied/Baker are talking about, I don’t think it really applies — beyond the idea that you can model behavior but only up to a point, which I guess is the thrust of what everybody is saying here.
And quite possibly, what everybody is saying here is all that needs to be said on the matter…
Anyway, one thing to bonnieL: I actually read Numerati because I’m reviewing it for somebody, not because I personally sought it out or endorse it. When the review is published I’ll probably link.
Thanks all.
I don’t think it’s an issue of not being able to model human behavior. But also, I don’t know how algorithms might have made highly leveraged derivatives into low-risk securities. As we hear more about how big-house-buying Americans and now apparently math geeks are to blame, let’s remember that investment bankers with MBAs created a multi-trillion-dollar market perched (slip-sliding?) on the edge of the housing bubble. Almost as if they didn’t care if it all went wrong, because they knew taxpayers would pick up the tab.
If anyone ignored human behavior, it probably wasn’t the math geeks. They all know what zero is.
Bankers, particularly derivates brokers, learn somewhere along the way that all securities, including the super-leveraged ones, can go up and up, but they also could go in the other direction. Were they really assured by “the math guys” that nothing of the sort would ever happen? “Trust me, finance is your business, but I’m better at math than you. Tradeable securities that are leveraged on top of America’s mortgages can only go up forever. You can’t argue with my math, it’s right here on my whiteboard.”
Math profs make algorithms to program things like insulin pumps for diabetics. They program the machine to react when a person’s sugar levels go up, and also to react when their glucose levels crash downward. If a diabetic downs three sugary sodas, or forgets to eat for a full day, is the program somehow to blame?
Here’s what we can predict: every 20 or so years, Wall St will implode and endanger the world’s economy, on the basis of crazy securities they don’t understand themselves.
Jason I think you make a great point. And I guess maybe the issue might be that the situation is the reverse — the math guys have bosses, who might well say, “Adjust your algorithm in a way that gets us more business.”
Thanks for starting this thread. Just to respond to your original question. My feeling is that human behavior can be modeled, and like all models, it will sometimes (or even often) get us wrong. The test isn’t whether these models are right, but instead whether they help companies sell more stuff, make more money, squeeze more production out of workers, etc. If a relatively bad model moves these numbers, it works. Its challenge, looking at it a bit cynically, is less to describe and predict the truth than to outperform the status quo, whether its another predictive model or someone’s gut analysis. This process eventually leads closer to the truth–assuming that more accurate models do better work and win in the marketplace. But even here, truth may not always win. An eerily accurate model that devines our urges too well in the supermarket or the pharmacy might give us the creeps–and send us scurrying elsewhere. But at this stage of the science, that’s not yet a problem, at least as far as I know.