Why ProPublica's auto insurance report is inaccurate, unfair and irresponsible

This Op-Ed Originally appeared in Insurance Journal, April 5, 2016

It looks like ProPublica failed its first actuarial exam.

The renowned investigative journalism website has, along with Consumer Reports magazine, published reports that auto insurers systematically charge unfairly high rates to people in minority and low-income communities.

It is an explosive charge – to say that in, for example, Illinois, 33 out of 34 companies the journalists looked at (including the nation’s largest insurers) all systematically price-gouged minority communities and areas with predominantly low income households.

And the charge is inaccurate.

ProPublica’s analysis makes an unfair comparison (and a fundamental actuarial mistake) by comparing the losses of all drivers within a ZIP code to the premium charged to a single person. Again, this is a basic error in statistical analysis, one so elemental that all actuaries learn about it early in their rigorous curriculum. Underwriters learn about it, too. In fact, most people working in the insurance industry understand this mistake and are taught not to make it. Most people in insurance would put it this way: The reporters are not making an apples-to-apples comparison.

ProPublica’s investigation is so faulty, it is being criticized not only by U.S. auto insurers but the state governmental agencies that regulate them. If their failure isn’t obvious to you, don’t worry; I’ll explain it shortly.

ProPublica’s journalists claim that bias permeates the industry, all supposedly perpetrated under the nose of state regulators, who monitor and often approve in advance the rates insurers charge. And ProPublica said it found discrimination in California, where industry skeptics can earn literally millions of dollars challenging auto insurers rate filings.

Discrimination is a charge the insurance industry takes seriously, and in fact, the industry works meticulously with its regulators to ensure that the millions of Americans paying for their products and services are provided fair and accurate pricing. To be clear, insurance companies do not gather information on race or income, nor do they set rates based on this information.

As American society grows increasingly diverse, insurers want to reflect that diversity, both in the people they employ and in the customers they protect. Embracing diversity is good for business. More importantly, it is the right thing to do.

If insurers were to systematically discriminate by race or income, it would be an alarming betrayal of much of what the industry stands for.

Here is why ProPublica’s reporters are wrong:

Insurance departments in four states – California, Illinois, Missouri and Texas – provided information on the accident rate and on the average size of claim by ZIP code. The reporters multiplied accident rate times size of claim within each ZIP code to calculate what they call a risk factor. (Insurance professionals will recognize that the accident rate times the average size of claim is the “pure premium,” or as I will refer to it, the “loss cost.”)

The loss cost in each ZIP code is then compared to the rate an insurer would charge a single risk in that ZIP code: a 30-year-old schoolteacher who has been driving for 14 years. This leads to a finding that the ratio of the fictional schoolteacher’s premium to loss cost varies by ZIP code and that the ratio is higher in minority and low income areas than elsewhere.

That, the reporters conclude, is evidence of discrimination.

Having read that – if you work in the insurance industry – you may be dumbfounded. You might be aghast. You are probably thinking the obvious: “They haven’t made an apples-to-apples comparison.” You are right.

Why is this bad? Well, the loss experience in any ZIP code is driven by all of the factors that make up that area’s drivers. And those drivers together are an eclectic mix. People in some ZIP codes drive, on average, more miles than people elsewhere. People in some ZIP codes get in fewer accidents than people elsewhere.

Here is a super-simplified example to make the point:

Suppose that people who drive “a lot” – say 10,000 miles a year – cost insurance companies $200 in losses per year. And suppose people who drive “a little” (5,000 miles a year) cost $100 to insure. (And assume there are no expenses or taxes. Including them complicates the math here needlessly.)

Further, suppose that there is a ZIP code in which everyone drives a lot and a minority-majority ZIP code in which everyone drives a little. Other than that, drivers are identical in either ZIP code.

How much will it cost to insure a person who drives a lot? $200 in both ZIP codes.

Would the insurer be discriminating? According to ProPublica’s reasoning, the answer is yes.

In the first ZIP code the ProPublica “risk factor” would be $200 and in the minority-majority ZIP code it would be $100.

And the ratio of premium to risk score – ProPublica’s metric for measuring discrimination – would be 1.0 in the first ZIP code ($200/$200) and 2.0 ($200/$100) in the minority-majority ZIP code. People in the minority-majority area are charged twice as much as the other people, using ProPublica’s flawed logic.

This, according to ProPublica, would be evidence of discrimination. However, the fact is that race or income didn’t create the disparity – it is the fact that the hypothetical driver behaves differently from the typical person in the minority-majority ZIP code.

ProPublica was told about this error – not just once, but several times. As chief actuary at the Insurance Information Institute, I explained it to reporter Julia Angwin in a telephone conversation February 13 and in a series of emails dated February 16 and 17. I also know that some of the insurers that stand accused told reporters the same thing.

To its credit, ProPublica shared its data with us. We hired a respected actuarial firm, Pinnacle Actuarial Resources, to independently review the study. They have come to the same conclusion. The journalists had made a rookie mistake.

Because of this, however, ProPublica has concluded that EVERYONE – actuaries, underwriters, executives, regulators – is doing something awful.

And that is why ProPublica journalists found something to accuse insurers of, something that had eluded teams of state regulators, apparently for decades, and scores of industry skeptics.

When you look at what ProPublica did, though, a more likely explanation emerges. ProPublica had a theory – some insurers refused in the 20^th century to sell its policies to minority consumers – and updated it for a 21^st century audience. Their premise today is that, in 2017, insurers are willing to sell policies to minority consumers but only on the condition that regulators allow insurers to overcharge minorities.

There are other issues I have with ProPublica’s study, including:

• In most states, it doesn’t seem to account for different auto insurance policy limits.
• It doesn’t consider that an auto insurer’s individual loss costs – the amount they pay out in claims – could vary from the statewide averages.
• It doesn’t seem to address how auto insurers priced policies where data about the policyholders and a ZIP code’s loss costs was thin.

These errors could all help produce the result ProPublica has reported, but the failure to understand the simple idea of the apples-to-apples comparison is the most obvious error, as well as the least forgivable.

Upon this faulty analysis, ProPublica and Consumer Reports hang a charge of minority discrimination. And they were told, repeatedly, that they were analyzing their data incorrectly. Confronted with this fact, they did nothing, which makes it clear they chose sensationalism over realty.

As a former journalist, and a practitioner of the actuarial science, I cannot help but to take this manner of reporting personally. It is irresponsible at the least.

James Lynch, FCAS MAAA, is Chief Actuary and Head of Research and Education at the Insurance Information Institute.