Glassbro - How Not to Rank Transit

From 2016 to 2021, NJ.com published a series of articles:

To this day, these articles get cited by local and national media and politicians.

These articles are also very bad. A big, statewide transit agency like NJ Transit will always break down more times than the Port Authority of Nowhere, Kansas.

Consider the MTA’s Long Island Rail Road (LIRR). In 2021, they reported 78 breakdowns, while the Minneapolis METRO had 2. Is the LIRR worse than METRO? Well, LIRR is the largest commuter rail system in the country by miles traveled, driving almost 368 times more miles than METRO. It’s meaningless to compare their total breakdowns, and misleading to call one worse for it. This could be a textbook example of how to lie with statistics.

Instead, we should compare breakdown frequency. Transit agencies often track this as Mean Distance Between Failures, or \(\frac{Vehicle And Passenger Car Revenue Miles}{Breakdowns}\). This tells us how many miles an agency’s trains or buses travel, on average, before breaking down. The higher the better.

Agency	Vehicle/Passenger Car Revenue Miles	Breakdowns	Mean Miles Between Breakdowns
MTA Long Island Rail Road	58,398,960	78	748,705
Minneapolis METRO	158,717	2	79,359

[Source: Author calculations using the 2021 National Transit Database, FTA. See data and methodology [here]({{< ref "post/ntd-2021" >}} “The National Transit Database: Technical Intro”).]

Now we see the LIRR breaks down less often than METRO, despite breaking down more times. LIRR trains travel over 748,000 miles before breaking down, on average, while METRO trains manage just over 79,000. LIRR is both much bigger than METRO and way more reliable.

Take it from a more reputable news source like… NJ.com. The same author¹ of the above articles wrote this one comparing NJ Transit and LIRR’s Mean Distance Between Breakdowns, not their total breakdowns.

Thankfully, NJ.com isn’t publishing more of these bad articles.² Even more fortunately, the NJ Democrats, who hold a trifecta, seem to correctly believe that years of under-funding caused NJ Transit’s issues, and that the solution is full, stable funding.³ Then again, their leader got ousted and they’re sliiiiiding to the right on other issues.

But a problem lurks. Imagine a future where we fully fund NJ Transit. It’d be easy for a reactionary shithead to convince people that NJ Transit remains the Worst In The Nation despite all the money we pumped into it. They’d seem legit because NJ.com did the same analysis for years. Don’t lend credibility to austerity boosters by sanitizing bad analysis in our state’s Paper of Record.

I humbly request that:

NJ.com either delete these bad breakdown articles or preface them with “This article is wrong” at the top in big, bold font;
The American Prospect and everyone else citing these articles stop doing so;
If we must, rank transit by breakdown frequency, not total breakdowns. I demonstrate how to do that here.

Footnotes

The author otherwise does good journalism. I ultimately blame editorial pressure for and approval of this type of reporting.↩︎
The author’s latest article on NJT breakdowns uses Mean Distance Between Breakdowns instead, but leads with a barbed comment:

When a train breaks down, statistics don’t matter if you’re the one waiting in the rain.

Odd coming from an outlet that published breakdown statistics for the last 5 years.↩︎
I say NJ Transit is owed some serious back-pay, but that’s for another post.↩︎