Turning to Twitter to Track Signal Delays on Subway

NEW YORK — If train signals are one of the principal ailments afflicting the New York City subway system, then how often do the mechanisms, some of which date back to the Truman White House, fail?

That question is surprisingly hard to answer given the Metropolitan Transportation Authority’s quixotic bookkeeping on the subject: It sometimes singles out signals as the cause of delays, but at other times lumps them into other and broader categories of problems.

One place that routinely reports when signals are on the fritz: @NYCTSubway, New York City Transit’s twitter account.

Now, a city councilman, Brad Lander, frustrated at what he believes is a lack of transparency by the agency that runs the subways, has embarked on an effort to get a more comprehensive picture of how often signals fail. The MTA does report signal failures on its online dashboard, but only when they are the root cause of delays that affect 50 or more trains.

So Lander asked his staff to use the transit system’s Twitter account to come up with his own count. Based on what is an admittedly unscientific survey, in a report released this week, Lander’s office found that the account sent out tweets about 479 signal failures of all sizes from December through February. On its website, the MTA, hewing to its own criteria of when to list signal failures, posted such breakdowns 66 times during the same time period.

“We can’t possibly develop a real plan to fix what ails the subway if we’re not being honest about what’s broken,” said Lander, a Democrat who represents parts of Brooklyn.

MTA officials scoffed at Lander’s criticism, noting that the agency is being transparent about signal problems since it is the one revealing them on Twitter as they happen. And in response to a request from The New York Times for a more thorough accounting, the transit agency provided what it said was a complete list of signal-related delays for last December and January. February numbers were not yet available.

The final tally? Signal issues delayed 11,555 trains.

The divergence from Lander’s count is partly explained by the fact that the agency tabulates individual trains impacted by signal troubles, while the agency’s Twitter feed focuses on a broken signal, which can affect many trains on multiple lines.

But the confusion over what role signals play in the subway’s dismal performance underscores the ongoing debate — and news media scrutiny — over the data that the MTA provides the public and how upfront it has been about identifying the causes of riders’ upended commutes.

“Being candid, we have archaic systems, multiple sources of the truth, databases galore, but it is very hard to pin down and get from multiple sources, true data,” said Andy Byford, president of New York City Transit, which runs the subway and public buses. Byford has already expressed his belief that blaming delays on “overcrowding,” is woefully imprecise, and not in fact, a root cause of delays.

Byford said he had commissioned a review of the terminology the agency uses such as what constitutes a major delay or a power-related failure, and how it categorizes the various causes of delays.

“There is a huge amount of data to crunch and properly categorize,” Byford said. “But that doesn’t mean we are not fixing the right things. We must tackle equipment that we know is failing, but then also categorize these failures in an appropriate manner.”

For Lander’s report, the data was collected by a social media bot that scraped digital information from the subway’s Twitter feed and downloaded each mention of a signal failure into a spreadsheet. Anna Levers, the councilman’s policy director, then pored over the data to cull any duplicated tweets or tweets the bot may have pulled in error. She then cross-referenced each delay with the subway stop nearest to where it occurred. Next, based on MTA data on hourly ridership numbers at subway stops across the city, Levers estimated how many riders were likely impacted by each delay. She posted her estimates periodically over the past few months on SignalFail.com.

For the three-month period, using these methods, she estimated that 11.1 million riders were delayed by the signal failures reported on the subway’s Twitter feed.

Of course, this analysis falls far short of any sort of reliability or scientific standard, Levers said, which was part of the point of the exercise.

“It’s best information we have to go on, and which is in part what we are objecting to,” Levers said. “It’s ridiculous that the way users are getting real time/real-time feedback in performance is by me scraping data from a Twitter feed.”