The Effect of Elections on Gasoline Prices

A quick web search shows that I’m not the only one who’s heard the talk about how gasoline prices always decline before an upcoming election.  Of course, this always gets mentioned when local gas prices are declining and an election is coming up.  But is this actually true?  Do gas prices in the United States decrease leading up to an election?  There are lots of articles written about this topic, and some even use numbers and statistics to back up their position, but I intend to be a bit more thorough here.

To get started, we’ll use the inflation-adjusted price of gasoline that we get from the U.S. Bureau of Labor Statistics.  We’ve looked at these data in a previous post, and if you’re at all interested in how constant-dollar costs are calculated, you should go read that post first.  This dataset goes back to 1976, so it includes a sizable number of election years.  Nineteen, to be precise.  It is important to use inflation adjusted data in this analysis because it compensates for the price changes from the changing buying power of the dollar.  Ten cent changes in the price of gas in 1980 and in 2010 aren’t the same, and inflation-adjusted prices account for this.

Unleaded regular gasoline prices from 1980 to 2014 in constant 2008 dollars.  Source:  U.S. Bureau of Labor Statistics.

Unleaded regular gasoline prices from 1980 to 2014 in constant 2008 dollars. Source: U.S. Bureau of Labor Statistics.

The first thing we should do is to look for annual trends.  It is entirely reasonable to expect that the price of gas shows some regular changes each year, so we should first understand if this happens and by how much so that we can account for it in our analysis.  To do this, we break the above graph up into its parts.  We separate the observed data into the parts that repeat on an annual basis (seasonal changes), the parts that change more slowly (long-term trend), and the parts that change more quickly (remainder).  If we add these all parts back together we will get the original observation.  For gasoline, this additive decomposition gives us the results plotted below.  Note the differences in the y-axis scales for the different plots.  The “trend” component is the largest.  Seasonal variations swing about seventeen cents from high to low, but the remainder, the fast-changing non-periodic fluctuations, can be in excess of a dollar, though they generally exist inside of a quarter of a dollar on either side of zero.

Decomposition of U.S. Gasoline prices into seasonal and other components.  Original data source:  U.S. Bureau of Labor Statistics.

Decomposition of U.S. Gasoline prices into seasonal and other components. Original data source: U.S. Bureau of Labor Statistics.

The seasonal component here is the part we’re interested in first.  There are a couple of ways to extract this component in a decomposition, but because we are using constant-dollar prices and looking for the true seasonal fluctuations, we don’t want to window the filter at all.  So let’s take a closer look at that seasonal component.  Recall that this is the component that occurs repeatably every year since 1976.

Seasonal change in U.S. gasoline prices (regular unleaded) in constant 2008 dollars.  Original source:  U.S. Bureau of Labor Statistics.

Seasonal change in U.S. gasoline prices (regular unleaded) in constant 2008 dollars. Original source: U.S. Bureau of Labor Statistics.

Not too surprisingly, we see there is a 7.7 cent increase in the summer, peaking in the June driving season, and a 9.2 cent decrease in gas prices in the winter, with the bottom arriving in December.  Interestingly, since elections are in November, they occur during the natural seasonal price decline.  This will not be a problem for us.

The key to determining whether or not non-seasonal conditions (i.e., elections) impact prices is simply to use the non-seasonal components of the prices for comparison.  That is, by considering only the trend and remainder components.  By excluding the seasonal fluctuations we can see how the non-seasonal prices changed, and from there the effect of elections on price changes will be able to be observed.

For the sake of simplicity, let’s define the time before the election that we are interested in to August through November.  This is an assumption on my part.  But the last quarter before the election seems to be the time we should pay the most attention to.  We can repeat the analysis using any other window of time if we desire.  Since we’ve already identified the seasonal changes in the decomposition, this is straightforward.  I’ve highlighted the pre-election time on the following graph for ease of viewing.

Non-seasonal changes in U.S. regular unleaded gasoline prices, in constant 2008 dollars during election years from 1976 to 2012.  Original source:  U.S. Bureau of Labor Statistics.

Non-seasonal changes in U.S. regular unleaded gasoline prices, in constant 2008 dollars during election years from 1976 to 2012. Original source: U.S. Bureau of Labor Statistics.

So when we do this we find the following for election year, non-seasonal gas price changes:

  • 1976:  +$0.08/gallon
  • 1978:  +$0.09/gallon
  • 1980:  -$0.02/gallon
  • 1982:  No Change
  • 1984:  +$0.10/gallon
  • 1986:  +$0.05/gallon
  • 1988:  +$0.02/gallon
  • 1990:  +$0.37/gallon
  • 1992:  +$0.09/gallon
  • 1994:  +$0.07/gallon
  • 1996:  +$0.10/gallon
  • 1998:  +$0.07/gallon
  • 2000:  +$0.14/gallon
  • 2002:  +$0.12/gallon
  • 2004:  +$0.21/gallon
  • 2006:  -$0.64/gallon
  • 2008:  -$1.40/gallon
  • 2010:  +$0.20/gallon
  • 2012:  -$0.10/gallon

So only four out of the last 19 election years have shown a drop in gas prices that were not part of the normal seasonal variation.  And only two of those were by more than a dime.  The average change here is a 2.3 cent drop, but that is very heavily influenced by the 2008 drop of $1.40/gallon, statistically an outlier.  The median value of an 8.1 cent increase is more in line with the typical behavior.  And of the years when the prices don’t drop, the average increase is 11.5 cents.  In other words, there is no election-year drop in gasoline prices using the BLS data.

We should ask ourselves how these election year results differ from those of non-election years.  This is also straightforward to answer.

Non-seasonal changes in U.S. regular unleaded gasoline prices, in constant 2008 dollars during non-election years from 1977 to 2013. Original source: U.S. Bureau of Labor Statistics.

Non-seasonal changes in U.S. regular unleaded gasoline prices, in constant 2008 dollars during non-election years from 1977 to 2013. Original source: U.S. Bureau of Labor Statistics.

And we find the following non-election year, non-seasonal gas price change results (August to November):

  • 1977:  +$0.06/gallon
  • 1979:  +$0.17/gallon
  • 1981:  +$0.04/gallon
  • 1983:  -$0.02/gallon
  • 1985:  +$0.04/gallon
  • 1987:  +$0.05/gallon
  • 1989:  -$0.01/gallon
  • 1991:  +$0.08/gallon
  • 1993:  +$0.11/gallon
  • 1995:  +$0.01/gallon
  • 1997:  +$0.04/gallon
  • 1999:  +$0.10/gallon
  • 2001:  -$0.09/gallon
  • 2003:  No Change
  • 2005:  -$0.09/gallon
  • 2007:  +$0.35/gallon
  • 2009:  +$0.13/gallon
  • 2011:  -$0.09/gallon
  • 2013:  -$0.20/gallon

And so we find that just six of the nineteen non-election years showed August to November gas price decreases, and only one of those was more than a dime drop.  The average price change in non-election years is a 3.6 cent increase with the median value of a 3.8 cent increase (it is nice when they agree).  And for the years that show an increase, the average change is a rise of 9.9 cents.  I think it is easier to grasp this visually.

Non-seasonal, August to November changes in U.S. regular unleaded gasoline prices from 1976 to 2013.  The comparison is made for election and non-election years.  Original data source is the U.S. Bureau of Labor Statistics.

Non-seasonal, August to November changes in U.S. regular unleaded gasoline prices from 1976 to 2013. The comparison is made for election and non-election years. Original data source is the U.S. Bureau of Labor Statistics.

This tells us that the non-seasonal median gas price change between August and November in an election year actually increases by 4.3 cents/gallon (in constant 2008 dollars) in an election year compared to the same time frame in a non-election year.  The caveat here is that we are dealing with national prices instead of local, but I think we can call this myth busted.

Advertisements

The Cost of Things: A Constant-Dollar Look at Common Goods

The other day I went to the grocery store to get some things to make tacos.  We often end up getting ground turkey these days, but I admit to preferring ground beef for this sort of thing.  But after I looked at the prices of ground beef, I believe I understand why we usually get turkey.  In case you haven’t noticed, the price for ground beef is on the rise, having crossed $4/lb in August of 2014 with no looking back.  This perplexes me somewhat as ground beef comes from cows, which are not dangerous, rare, or hard to kill.  Oh, for the cheap ground beef days of the 1980s, right?

Average U.S. prices for ground chuck (100% Beef) from 1980 to 2014.   Source:  U.S. Bureau of Labor Statistics.

Average U.S. prices for ground chuck (100% Beef) from 1980 to 2014. Source: U.S. Bureau of Labor Statistics.

Now any time you want to compare prices over some length of time you run into the same problem. The buying power of the dollar is not constant. It changes over time, and this makes it somewhat difficult to tell whether something is getting more or less expensive or if the buying power of the dollar is changing.  Or both.  The folks at the U.S. Bureau of Labor Statistics track the buying power of the dollar each month by finding out what it costs to purchase a basket of particular items in the marketplace. The result of their work is called the Consumer Price Index, or CPI.  So whether the cost of something actually changes depends on how it changes compared with the index.  To see this, we use the index to adjust past prices to reflect what they would have been if the dollar had kept a constant buying power.

This method isn’t perfect because it only reflects the buying power of the dollar at some point in time and doesn’t directly consider the change in the money supply.  You might have thought that the amount of money in the U.S. economy was a fixed number, and that would be a reasonable assumption.  But it would be a very wrong assumption.  An increase in the money supply will decrease buying power but takes some time to be recognized by the market.  Because of this, the CPI will lag somewhat behind actual values.  In any case, we won’t let perfection be the enemy of good enough.

In order to use the CPI to compare historic prices, we pick a year and normalize everything with respect to that year’s value.  Below is a plot of the CPI using 2008 as the reference year (where the CPI = 1).  To use the chart, we can look and see that in 1985 the CPI was one-half, and this indicates that prices doubled between 1985 and 2008.  Specifically, this means that the prices of the things in the BLS’s market basket doubled.  That’s another way of saying the buying power of the dollar was cut in half in that time period.  So much for your savings.

Consumer Price Index (CPI) from 1980 to 2014, using 2008 as the reference.

Consumer Price Index (CPI) from 1980 to 2014, using 2008 as the reference.

With this information in hand, we can have another look at the prices of ground beef in constant dollars.  And now we see that the cost of ground beef was surprisingly high in the 1980s but decreased until about 2000.  It then started its way up sharply around the middle of 2013 leading up to today’s prices.  So ground beef really is getting more expensive, and by no small amount.  Not good for taco lovers who prefer beef to turkey.  And before you ask, they don’t track ground turkey prices, and I don’t know why.

Historic pricing of ground chuck in the U.S. in constant 2008 dollars.  Source:  U.S. Bureau of Labor Statistics.

Historic pricing of ground chuck in the U.S. in constant 2008 dollars. Source: U.S. Bureau of Labor Statistics.

On a side note, who would have ever thought, at the time, that we would look back at the 1980s as the days of the strong dollar?

In any case, the BLS tracks the prices of a lot of other products.  We can get a feel for the health of the economy by looking at how the cost of things have changed using constant-valued currency.  So let’s continue with food.  Specifically, chicken.  The inflation adjusted cost of fresh, whole chicken in constant 2008 dollars shows that chicken prices have been stable since about 1990, which means the price at the register has increased at about the rate of inflation.  Hopefully your income has as well.

BLS data for U.S. fresh whole chicken prices in constant 2008 dollars. Source: U.S. Bureau of Labor Statistics.

BLS data for U.S. fresh whole chicken prices in constant 2008 dollars. Source: U.S. Bureau of Labor Statistics.

Milk is an interesting product to look at because it one of the items in the BLS market basket that is used to calculate the CPI.  Because of this, we should expect the cost of milk to move pretty much in line with the CPI and give us a flat cost curve, which it does—on average.  These numbers are nationwide averages, so some fluctuations should be expected.  But we also have (or have had in any case) minimum prices federally guaranteed at various times during this period.  Meaning there’s no limit to how high milk prices can go, but prices won’t fall below a set value.  And if that happens, your tax dollars are used to pay for the milk you didn’t buy in the store.

Milk prices from 1980 to 2014 in constant 2008 dollar prices. Source: U.S. Bureau of Labor Statistics.

Milk prices from 1980 to 2014 in constant 2008 dollar prices. Source: U.S. Bureau of Labor Statistics.

Sugar is the last of the food products that we’ll look at.  Sugar is another one of those agricultural products with price supports, but this one isn’t in the BLS market basket.  Interestingly, the supported prices don’t generally adjust in line with inflation.  So while the retail price of sugar (not shown) has been largely steady throughout the decades, the inflation-adjusted true cost of it has been coming down.  Interestingly, it was the production shortages in 1979 and and 1980 that lead to the soaring prices in the early 1980s that ultimately lead to the switch to high fructose corn syrup (HFCS) by food and beverage companies.  Other references show similar or larger spikes in sugar prices in the 1960s and 1970s.  It isn’t clear that it was worth it, but sugar hasn’t been quite so volatile since the switch.

Sugar prices from 1980 to 2014 in constant 2008 dollars. Source:  U.S. Bureau of Labor Statistics.

Sugar prices from 1980 to 2014 in constant 2008 dollars. Source: U.S. Bureau of Labor Statistics.

There are some other interesting products to look at. Electricity is one of them. I haven’t seen anything that shows seasonal price fluctuations quite as clearly as the electricity cost chart.  What is especially interesting here is that sometimes the retail price (not shown) of electricity increased while the cost in constant dollars came down.  Being a regulated public utility, the prices for electricity are generally not market-driven, but based on cost-recovery, so we can read from this that the costs of the electric utility business have been decreasing fairly consistently since the mid 1980s.  Here in New England they’re talking about 40% increases in electricity rates in the next year or so, which would take us back to the historically high 1980s costs.  There is no historic precedent for that kind of single year price increase.  At least, not in the last three decades.

The cost of electricity in the U.S. from 1980 to 2014 in constant 2008 dollars.  Source:  U.S. Bureau of Labor Statistics.

The cost of electricity in the U.S. from 1980 to 2014 in constant 2008 dollars. Source: U.S. Bureau of Labor Statistics.

Since we are on the topic of energy, it makes sense to look at gasoline.  Gasoline showed a fairly flat cost curve from the mid 1980s though the early 2000s before it went all to hell.  There was a significant correction in late 2008 which is curious.  And in spite of the fact that the prices are on the decline, they’re still historically high by around a dollar per gallon.  I always hear about how gasoline prices decline in election years.  That isn’t clear from this look, so that will be a topic I’ll dig into soon.

Unleaded regular gasoline prices from 1980 to 2014 in constant 2008 dollars.  Source:  U.S. Bureau of Labor Statistics.

Unleaded regular gasoline prices from 1980 to 2014 in constant 2008 dollars. Source: U.S. Bureau of Labor Statistics.

Staying with energy, for those who heat with natural gas, you’ll see that current rates are close to average lows.  But costs have clearly been on a roller coaster for the last 15 years.

Natural gas prices from 1980 to 2014 in constant 2008 dollars.  Source:  U.S. Bureau of Labor Statistics.

Natural gas prices from 1980 to 2014 in constant 2008 dollars. Source: U.S. Bureau of Labor Statistics.

So what does this tell us?  First, that someone has to be making some serious money in the ground beef industry.  The inflation-adjusted cost of ground beef is growing well above the rate of inflation, and I’ll bet that the difference is being pocketed by someone clever enough to pull it off.  Second, I was surprised to see how some of the inflation-adjusted prices have actually declined.  My instincts were to guess that I would see most costs on the rise, but flat or decreasing costs seem to be the rule and not the exception.  This is, of course, a simple survey rather than an all-encompassing study.  For all I know, every other product is skyrocketing.  Given the slope of the inflation curve, retail prices are doing a good job of increasing at the register.  May your income increase ever faster.

Feeding America: The Extraordinary Increase in US Farm Productivity (Part 2)

After my last post on the remarkable increase in field crop yields in American agriculture, I was interested in seeing where else yield improvements were observed and hopefully getting a better understanding of the causes.  I’m interested in understanding how much of the impact mechanization (i.e tractors and harvesters) had as opposed to improvements in seed, fertilizers, and pesticides.  Certainly all of these came together to boost productivity, but what had the greatest impact?

So on to (non-field crop) vegetables.  The same USDA website has statistics for vegetables as well as for field crops, but to get any data of any historical significance, we can’t afford to be choosy.  Only two crops have data going back to the pre-1970s days, and thankfully they go back to the 1860s.  So here we have the crop yield data in cwt/acre (cwt is a centum weight, or hundredweight—a one hundred pound increment) for potatoes and sweet potatoes.  Given what we have already seen, this isn’t terribly surprising.  It looks much the same as the graphs for corn or wheat.  That is, a relatively flat yield curve until the 1930s followed by a sharp upturn and a monotonic increase spanning seven decades.  I still find this to be remarkable.

Potatoes and Sweet Potatoes yields in centum weight/acre (1868-2014).

Potatoes and Sweet Potatoes yields in centum weight/acre (1868-2014).

And what about how the yield has grown?  Looking below we can see that potatoes are 8 times more productive than they were some seventy years ago.  Considering the crops we have looked at so far, that is the record (beating rice’s sevenfold yield increase).

Yield growth for Potatoes and Sweet Potatoes (1868-2014).

Yield growth for Potatoes and Sweet Potatoes (1868-2014).

Explaining this is somewhat of a problem.  Every crop we have looked at has shown the same yield curve behavior.  But certainly not every crop had a sudden, massive successful hybridization improvement at the same time.  And if pesticides and fertilizer were primarily responsible, then why wouldn’t we see a large step change in the yield curve instead of steady incremental growth?  We do know that the tractor and other mechanized equipment came to popular use in the 1930s, but isolating its impact on crop yields is difficult.

To understand the effect of mechanization, we have to look at something else.  We have to look at a product that doesn’t rely on fertilizers or pesticides.  We have to look at an agricultural product where hybrid seed isn’t a factor.  We have to look at a product where mechanization is the primary driver of yield growth.  We have to look at milk.

Milk yield in lbs/head from 1924-2014.

Milk yield in lbs/head from 1924-2014.

And milk shows the exact same behavior.  In 2014, a single dairy cow in the United States could be expected to produce close to 22,000 pounds of milk in a year.  That is no small feat considering that the same cow was producing only about one-fifth of that at just over 4,000 pounds annually in 1924.  (Well not the same cow…)  This suggests that mechanization, and the knowledge behind it, has been the primary driving factor in the increase in farm productivity over the last 70 years.

Milk yield growth from 1924-2014.

Milk yield growth from 1924-2014.

And by knowledge I mean that it isn’t enough to make a milking system that pumps faster or has larger storage tanks to increase the time between handling.  Today’s automated milking systems enable a cow to decide when she needs to be milked to enter a milking stall and have the system automatically attach and begin.  This is an interesting video.  I don’t know the language, but it shows we are a far cry from a three-legged stool and a bucket.

Now you might say, “Not so fast!  The dairy industry gives cows hormones to increase milk production!”  And while there is an element of truth there, it doesn’t explain what we see in the yield.  In the 1930s it was found that Bovine Growth Hormone (BGH, also bovine somatotropin, or BST) boosted milk production. But to get BGH they had to extract it from the pituitary glands of cow cadavers, so it wasn’t able to be widely used.  It wasn’t until the 1970s that the gene for producing it was identified.  Recombinant bovine somatotropin (rBST) didn’t receive FDA approval for usage until 1983, and wasn’t commercially available until 1994.  It also isn’t clear that it has always made great economic sense, and adoption has been less than commercial producer expectations, at least at times.  So hormones can’t explain this observation.

This takes us back to mechanization.  Knowledge, ingenuity, and the desire to improve have lead us to build equipment to do the job better on the farm.  Better tractors, seed planters, harvesters, milkers, irrigation systems, and so on have allowed us to do more with less.  And the most interesting thing of it all is that in most of the data that we’ve seen, there is no leveling off in sight.  We haven’t hit the maximum yet; production yields continue to increase each year, on average.

Predicting the future is hard, but it is very easy to look backward in time to see how we got to where we are today.  I don’t imagine a farmer alive in the 1920s would ever have imagined that it would be possible to be getting the production levels we are getting today.  Likewise, today it seems impossible that these numbers can continue to increase for another seven decades.  In spite of that, I’m going to bet on the impossible coming to reality when it comes to feeding America.

Feeding America: The Extraordinary Increase in US Farm Productivity

A few nights ago I was re-reading a P.J. O’Rourke book on terribleness.  In the chapter on famine, everyone’s favorite bedtime reading subject, something he said struck me.  “In most of the world, food production has well outpaced the growth of population.  In the 1930s American wheat growers had an average yield of thirteen bushels per acre.  By 1970 the yield was thirty-one bushels.  In the same period the corn yield went from twenty-six bushels per acre to seventy-seven.”  This was unexpected. I have the picture in my mind of the differences between arithmetic and geometric progressions when it comes to comparing food production and population.  This is courtesy of Rev. Malthus, whose treatise was nicely summarized in the same book on terribleness in the chapter on overpopulation: “…there’s no end to the number of babies that can be made, but you can only plant so much wheat before you run the plow into the side of the house.

According to data from the fine folks at the USDA, the story here is rather interesting.  Yes, the wheat yield in the 1930s was in the low teens.  But what is surprising is that it had been there at least since the USDA began recording wheat crop yields in 1866!  In other words, the amount of wheat an acre of farmland produced showed no significant improvements for at least some seventy years, until the early 1930s.  Since then, however, it has increased at a roughly constant rate, reaching its all time high of 47.1 bushels per acre in 2013.  And it hasn’t settled at that yield; it continues to increase.

U.S. wheat crop yields, in bushels per acre.  1866-2014.

U.S. wheat crop yields, in bushels per acre. 1866-2014.

It should come as no great surprise, then, that corn yield numbers tell essentially the same story.  But I was surprised by the fact that the upturn in the yield happens at about the same date.  Corn production was a flat 25 bushels per acre for decades and started its way upward at about 1930.  For the curious, it peaked at 173.4 bushels per acre in 2014 and continues to climb as well.  That is a huge number.  A corn farmer in 1900 working hard to get his 28 bushels per acre never in his most fantastic dreams thought yields like this were possible.

U.S. corn crop yields, in bushels per acre. 1866-2014

U.S. corn crop yields, in bushels per acre. 1866-2014.

We can look at this another way, by plotting the corn yield against the wheat yield.  And what we see is that the relationship is well behaved.  When wheat yields increase, so do corn yields, though not necessarily in the same proportion.  After a point, every 10 bushels/acre growth in wheat equals about a 40 bushel/acre growth in corn.  This suggests there is more to the story — that there is some common factor that drives this effect.

The crop yields for U.S. corn plotted against the yields of U.S. wheat for the years 1866-2014.

The crop yields for U.S. corn plotted against the yields of U.S. wheat for the years 1866-2014.

And there is more to the story.  The USDA doesn’t just measure wheat and corn production.  They determine the yields for all of the field crops.  So that we can compare like things, first are the crop yields that are measured in bushels per acre: barley, corn, flaxseed, rye, sorghum, soybeans, and wheat.   Of those, only rye and sorghum yields look as though they have stopped increasing.  The rest show this continuously increasing trend over time, starting around the same year — 1930.

USDA crop yields (bushels/acre) for barley, corn, flaxseed, rye, sorghum, soybeans, and wheat.

USDA crop yields (bushels/acre) for barley, corn, flaxseed, rye, sorghum, soybeans, and wheat.

And then we have the crop yields that are measured by weight (pounds per acre).  These are: beans, cotton, hay, hops, peanuts, peppermint oil, rice, spearmint oil, sugarbeets, and tobacco.  These also show the same yield growth since about 1930 behavior.  Interestingly hay and tobacco yields seem to have joined rye and sorghum yields in leveling off (showing classic error function behavior).

USDA crop yields (pounds/acre) for beans, cotton, hay, hops, peanuts, peppermint oil, rice, spearmint oil, and tobacco.

USDA crop yields (pounds/acre) for beans, cotton, hay, hops, peanuts, peppermint oil, rice, spearmint oil, sugarbeets, and tobacco.

If it wasn’t clear before, it is now.  Sometime around the 1930s, something quite dramatic began to happen in field crop production.  An agricultural revolution of sorts.  And while the effect is present for all of them, it varies in its impact depending on the crop.  To show this, we normalize the yield rate by the average of the first few years in the dataset for a given crop.  This shows us how the yield rate has grown in time.  And we can compare them all if we plot them all with the same y-axis scale (conveniently done below).  So the biggest crops in terms of yield rate growth are:  corn (~7x), cotton (~6x), peanuts (~5.5x), rice (~7x), sorghum (4-6x).

Yield growth in US field crops.  Plotted is the multiplier in the yield rate since the start of data collection for the given field crop.

Yield growth in US field crops. Plotted is the multiplier in the yield rate since the start of data collection for the given field crop.

I find it interesting that the curves for soybeans and cotton begin to increase in 1920, which is a few years before the others.  Peanuts seem to be among the last to join the group as their yield didn’t start taking off until about 1950.  Did some experimentation take place with soybeans and cotton crops and then once successful transition to other crops such as wheat, corn, and then peanuts?  Soybeans became quite important in the US around 1910, so this is plausible.  Though I haven’t found anything to suggest this is what actually happened.

We do know that hybrid seed became all the rage starting around 1930.  Gregor Mendel demonstrated plant hybridization in the 1860s with peas, but it wasn’t until the 1930s when hybrid seed was able to produce a corn crop that was well suited to mechanical harvesting.  One report says: “The tractor, corn picker, and hybrid seed corn came together to raise labor and land productivity in corn production in the late 1930s.”  Tractor development (power takeoff, rubber tires) and sales really started to take off in the 1920s, which fits the timeline perfectly.  And I’ll bet that improved chemical fertilizers and pesticides factor in as well.

In any case, I should go back to the original statement where O’Rourke said food crop yield rates were increasing faster than the population.  In 1927 the world’s population is estimated to have been about 2 billion people.  This is convenient for our comparison as it about coincides with our 1930 start of this agricultural revolution.  Corn and wheat yields both doubled by about 1960, when the global population was about 3 billion (a 1.5x increase).  So far, so good.  And we reached about 7 billion people in 2012, an increase of a factor of 3.5 since 1930.  This about matches the rate of increase in wheat production.  But corn and rice have managed to stay ahead by a considerable margin, so O’Rourke is right and Reverend Malthus has been wrong.  At least, since this “revolution” started.

But these crop yield numbers aren’t global numbers, they’re for domestic production.  How do they compare against the growth in the U.S. population?  Quite favorably, it turns out.  In 1930 the U.S. population was about 123 million.  It grew by about 50 percent to about 180 million in 1960, which is in line with global population increase we just considered, where crop yields for corn and wheat doubled.  But in 2012 the U.S. population reached 313 million, a growth factor of only 2.5x over 1930.  The slowest of the food crop yields shown above have at least grown at the same pace as the domestic population.  But the major food crops like corn and rice are out in front by a mile, with their yields growing some 2.8 times faster than the growth in domestic population.

This is startling, but in a good way.  It is a win for science and innovation and demonstrates how humans can manipulate their situation to work to their advantage.  The primary food crop yields show no signs of leveling off.  It makes sense that they probably will at some point.  But for now it is nice to know that the Reverend Malthus is wrong.

Book Review: Thinking Fast and Slow – Daniel Kahneman

There are people who spend their lives peeling an onion.  If they are lucky, it is a sweet bulb, and offers up its layers without too many tears.  If they are very lucky, what the peeling reveals is interesting enough that others are interested and pay attention.  And if they are very, very lucky, well, then the Royal Swedish Academy of Sciences honors their efforts with the most prestigious award that onion peelers can receive.

There were two people peeling this particular onion, but one passed away before their work was fully recognized.  In telling their story on their work on human decision making psychology, Daniel Kahneman recognizes his longtime collaborator Amos Tversky while sharing their work that was recognized with the 2002 Nobel Prize in Economics.

Since before the 1970s, it was well understood that humans were rational decision makers, and that when we strayed from this rational behavior, it was driven by some strong emotion.  Anger, fear, hatred, love — these were the things that pushed us into irrationality.  This makes perfect sense.  This “Utility Theory” was well known and not really challenged because it was, well, obvious.  Tversky and Kahneman, however, challenged its depiction of rational human decision making in their 1979 paper, “Prospect Theory: An Analysis of Decision Under Risk.”  We, it turns out, don’t behave very rationally at times (the Ultimatum Game is a very good example of this).  But what made this paper special was that they went beyond documenting the failure of Utility Theory and pointed their fingers at the design of the machinery of cognition as the reason, rather than the corruption of thought by emotion.  They argued that heuristics and biases were the key players in judgement and decision making.  This was a revolutionary idea.  The first layer of the onion had been peeled back, and as you might expect, it revealed more questions.

“Jumping to conclusions on the basis of limited evidence is so important to an understanding of intuitive thinking, and comes up so often in this book, that I will use a cumbersome abbreviation for it: WYSIATI, which stands for what you see is all there is.  System 1 is radically insensitive to both the quality and the quantity of the information that gives rise to impressions and intuitions.”

Kahneman describes the cognitive machine with a cast of two players.  These are, as he calls them, “System 1” and “System 2.”  It would be easy to call these the subconscious and the conscious; this would be a good first approximation, but it wouldn’t be entirely correct.  System 1 does operate quickly, automatically, and with no sense of voluntary control, as you would expect the subconscious to do.  But it is responsible for feeding System 2 with things like feelings, impressions, and intuitions.  System 2 generally takes what System 1 gives it without too much fuss.  But System 1 behaves in funny ways sometimes, producing some very interesting results.

Interestingly, you can give System 2 a specific task that taxes the cognitive resources and some interesting things happen.  For example, watch the video below and count the number of times the players wearing white shirts pass the ball.

Did you get the correct number?  Did you see the gorilla?  About half of the people who watch the video simply do not see the gorilla.  System 2 is busy counting, while System 1 is supporting that task and not distracting it with irrelevant extraneous information.  Sometimes, it would appear, we are quite blind to the obvious.  And blind to our own blindness.  This “tunnel vision” of sorts happens not only when we are cognitively busy, but also in times of stress or high adrenaline.

I experienced this personally a few years ago during an Emergency Response training exercise.  I was part of the two-man team entering a room where we had to assess the scene and respond accordingly.  The instructor running the exercise had taped a sign to the wall with information that would provide some answers to questions we would have in the room; things that would be obvious in a real situation.  Except it wasn’t obvious.  I didn’t see it.  At all.  A coworker was playing the part of an injured person and lying on the floor and safely removing him from the scene was all we could think about.  As we debriefed I was asked why I did not address the issues on the sign.  I had to go back to see the sign for myself to convince myself they were serious.  I was shocked.

After Prospect Theory, discerning the rules for the cognitive machine became a hot research area in the field of cognitive psychology.  And what they found is astounding.  A brief overview of the heuristics and biases can be found online, and these are discussed in detail in the text.  Some, like the affect heuristic, make perfect sense.  Emotion and belief or action are tied together.  This is a significant influence in how you create your beliefs about the world.  But priming, on the other hand, is downright scary because we have no conscious knowledge that it is taking place.

One of the interesting things that comes out of this research is that not only are humans are not rational thinkers, we aren’t very good statistical thinkers either.  Kahneman and Tversky’s first paper together was “Belief in the law of small numbers.”  The “law of small numbers” asserts, somewhat tongue in cheek, that the “law of large numbers” applies to small numbers as well.  There is much truth about this in how we build very lasting first impressions, quickly finding rules where random chance is a better explanation.  Kahneman’s insights into the day-to-day decisions and judgements that we make, with no thought into them, are priceless.

“The idea that large historical events are determined by luck is profoundly shocking, although it is demonstrably true…It is hard to think of the history of the twentieth century, including its large social movements, without bringing in the role of Hitler, Stalin, and Mao Zedong.  But there was a moment in time, just before an egg was fertilized, when there was a fifty-fifty chance that the embryo that became Hitler could have been a female.  Compounding the three events, there was a probability of one-eighth of a twentieth century without any of the three great villains and it is impossible to argue that history would have been roughly the same in their absence.  The fertilization of these three eggs had momentous consequences, and it makes a joke of the idea that long-term developments are predictable.”

It is pleasant to find an academic that can write a general interest book.  Too frequently the result of such an effort is a dense tome that is closer to a textbook.  Thinking Fast and Slow is enjoyably readable.  But it is more than that.  It is a very complete book—a dissecting of the machinery of the mind.  It pulls back, in 38 chapters, the covers to reveal in plain language the mechanisms that operate our minds every day.  The sorts of things that go on behind the scenes in every decision we make.  But also the myriad ways that advertising professionals can and do manipulate us.

This is not a weekend quick read.  The paperback version weighs in at 512 pages.  That shouldn’t hold you back.  After all, this is a review of an entire lifetime of Nobel Prize-winning research, in clear language without jargon, told with its historical perspective.  There is gold on every page and I’m grateful for every one of them.

Eleven Weeks of Counting the Mail

For no good reason outside of my own curiosity I have taken on a project of analyzing what shows up in my mailbox.  After eleven weeks of collecting information, there are some interesting things that I’ve observed.   How useful this project is, or will be, remains questionable.  But I can find no signs online of anyone doing this sort of project, so perhaps it will reveal something valuable.  Even with eleven weeks of data, the numbers are still relatively small, so we should be careful not to draw any concrete conclusions just yet.

First, let’s start by looking at what has shown up in the mailbox each week.  As much as some folks rant against stacked bar charts, I think they can be useful in some situations.  Situations like this, for example.  Here we can see the weekly volume of mail that my local carrier delivered to the box, how that number changed over time, and what the majority category was for each week.  Each week’s bar is sorted by quantity, with the largest quantity category placed at the bottom.  This makes large changes easy to spot.  It should be easy, for example, to grasp when the last election took place from the sudden rise (and subsequent disappearance) in the purple bars.

Eleven weeks of USPS mail, by category.

Eleven weeks of USPS mail, by category.

In brief, over these eleven weeks I received anywhere from 13 to 33 pieces of mail each week.  The average was 22 pieces of weekly mail with a standard deviation of about 6.5.  I’m not sure that a Gaussian model is the right one for this type of data (Poisson is probably more reasonable), but it gives a reasonable summary for a smallish data set.  Examining the distribution statistics is something I intend to do once the dataset grows a bit larger.

The relative breakdown between the categories is not surprising.  Advertisements dominate the composition.  I trust the political ads will be competitive for second place as the November election gets closer.  What I think this shows pretty clearly is how the Post Office gets paid.  Fully half of the mail I have received has been advertising, either the normal (34.5%) or the political flavor (17.8%).  I don’t think I would have guessed it to be that high.  If there is ever legislation passed to limit junk mail, the USPS will be in bigger financial trouble than they’re in already.

Mail_totals_percent_by_category (11 weeks)

What I find terribly interesting, though, is the breakdown by weekday. Meaning, the day of the week that the piece of mail was received.  Eleven weeks is not the biggest sample size, and so this may not hold true in the long run.  But Friday is absolutely a dominant day for the quantity of mail.  And not just for a single category.  The three major categories all have big mail days on Friday.

Mail totals by day of the week.

Mail totals by day of the week.

We can see this more clearly by looking at the categories separately.  I’ve separated out the weekly totals in actual pieces of mail for these categories.  First are “Advertisements,” otherwise known as “junk mail.”  Friday stands above the crowd with 22 above Wednesday’s 19.  Monday, Tuesday, and Saturday all come in at 10, so this difference appears to be significant.

Advertisement_by_Weekday (11 weeks)

Then we have “Information.”  This is mail from people or businesses that I have a relationship with.  Doctors’ offices, schools, banks, etc.  This is a very big Friday category with 18.  The next largest day is Saturday with 9.

Information_by_Weekday (11 weeks)

And then “Political Advertisements.”  I’m somewhat conflicted on how valid this category is. There was an election held on a Tuesday, and the direct mail spike happened on the Thursday (7 pieces) and Friday (6 pieces) of the week before.  Since the election I have received at most one political advertisement on any given day.  So this contributes to the “big Friday” effect, to be sure, but it was just one week out of eleven, so its impact is mitigated by its relative rarity.  But note that eliminating the category altogether would still result in Friday being the dominant weekday for mail delivery.

Political Advertisement_by_Weekday (11 weeks)

So this “big Friday” effect seems like it might be real.  But the raw numbers are still relatively small and the time relatively short.  So I’m cautious about drawing too many conclusions just yet, but it certainly is an interesting, if preliminary, result.

Book Review: A More Beautiful Question – Warren Berger

Questions play a significant role in how we learn.  The quest for knowledge and just plain human curiosity are natural drivers for the questions we ask.  But what is different about the most creative, the most innovative—the designers, the inventors, the engineers?  What sets them apart?  The author and journalist Warren Berger sought to find out and details his findings in A More Beautiful Question:  The Power of Inquiry to Spark Breakthrough Ideas.  What he found, in brief, was that these people differ in their exceptional ability to ask questions.  Specifically, they ask the types of questions that force a different perspective.  Something ambitious.  Something actionable.  He calls these “beautiful questions,” a la E.E. Cummings.

“Always the beautiful answer

Who asks a more beautiful question.”  — E.E. Cummings

So what is a beautiful question?  Berger gives us some examples.

  • When Intel co-founders Andrew Grove and Gordon Moore were contemplating the future of Intel, they faced a difficult decision.  Should they stick with making memory chips, or switch to the (perhaps) more promising world of microprocessors?  Their beautiful question: “If we were kicked out of the company, what do you think the new CEO would do?”  Without the emotional attachment to the company’s product history, the right decision was clear, and arguably vindicated by history.
  • When taking a photograph of his three year old daughter in 1943, Edwin Land was asked by her, “Why do we have to wait for the picture?”  Land went on to co-found Polaroid and developed the instant photograph.
  • After losing a foot in a waterskiing accident, Van Phillips asked his beautiful question. “If they can put a man on the moon, why can’t they make a decent foot?”  His answer revolutionized the world of prosthetics.

“So then, one of the primary drivers of questioning is an awareness of what we don’t know—which is a form of higher awareness that separates not only man from monkey but also the smart and curious person from the dullard who doesn’t know or care.” — Warren Berger, A More Beautiful Question

Not satisfied simply to understand this type of questioning, Berger dives into related issues.  The discussion on the decline of question asking as we age was particularly enlightening (Chapter 2 – Why We Stop Questioning). He cites a Newsweek article from 2010 that made an revealing, but mostly overlooked observation:

“Preschool children, on average, ask their parents about 100 questions a day. By middle school, they’ve pretty much stopped asking.” —Po Bronson and Ashley Merryman, The Creativity Crisis, Newsweek, 7/10/2010

Not surprisingly, he asks why this happens in a well-researched and stimulating chapter that is mostly focused on the changes in education.  It is insightful and not too preachy.

Although he points out that there isn’t a specific roadmap to follow, Berger does boil down the fundamentals of the “beautiful” questioning process into its characteristic stages of Why, What If, and How.  I don’t believe too many will find that series of questions shocking, or even anything but expected.  As someone who does R&D for a living I found myself nodding in agreement but hoping for something less…obvious.  But Berger makes it real by including specifics from innovation leaders and how questioning is part of how they work.

Perhaps too much emphasis is placed on building an empire around solving the problem that caused the question to be asked.  Early on in the book, the key observation is that innovative questioners “give form to their ideas and make them real.”  But later, the emphasis is on good questions and how they are formed, which is persistent through the last few chapters.  I’m going to say he ended up in the right place.

Questioning in Business and Questioning for Life are the last two chapters and are full of questions you haven’t likely thought about or asked yourself.  I have a personal quest to think thoughts I haven’t had before, and these were stimulating chapters.  Questions such as “what should we stop doing?” or “what should we learn?” in business (instead of “what should we do?”).  And techniques like “thinking wrong” to “jiggle the synapses” are introduced.  It is a clever read that takes you beyond simply reading for pleasure and challenges the way you look at the world each day.

“Jeff Weiner, the chief executive of LinkedIn, observed that he often asks prospective employees this reasonable and fairly straightforward question: Looking back on your career, twenty or thirty years from now, what do you want to say you’ve accomplished?…You’d be amazed how many people I meet don’t have the answer to that question,” Weiner said.”

Berger has put together a quality work.  The breadth of the content means you’re going to see something you haven’t seen before, and the things you have seen before will likely have a new perspective.  Cap that off with twenty-five pages of notes organized by chapter, and indices for both the questions and the questioners featured in the book, and this one is worth having in hardcover.

“We must embrace the notion that answers are in fact quite boring.  The Irish are especially good or infuriating in this respect.  We answer questions with questions.  But in my opinion that’s a good place to be.  A little perplexed by the perplexity of life.” —Colum McCann