The City That Never Deletes: How New York City Became a Real Estate Data Mecca

Reonomy found this out the hard way. The company launched in New York City in 2013 with a product that pulls information on properties from different sources and compiles them on one website, and its first New York iteration was smooth sailing. That was not the case when it tried to expand to Los Angeles in 2015.

“New York is 20 years ahead of the rest of the country in terms of the data that’s available,” said Patrick Rafferty, the vice president of product for Reonomy.

To get the mortgage history of properties here, Reonomy simply had to download the data from the New York City Department of Finance’s Automated City Register Information Systems, and the data came in a format the start-up could easily parse for relevant data, according to Rafferty.

In Los Angeles, however, Reonomy had to buy the images of all those documents from the County Clerk’s office. That is, once they were able to finally get in touch with somebody there, Rafferty said. Then it had to convert those thousands of pages into the optical character recognition format so the text could be searched by computer.

Only then could Reonomy start pulling relevant information from the files to plug into its database.

“That’s a massive barrier to entry,” Rafferty said. “The amount of work that you have to do to structure the information in other counties is immense compared to what you start off with in New York.”

Thanks in part to legislation passed nearly a decade ago by Mayor Michael Bloomberg, New York City has become one of the best spots in the country to access free data in an easily accessible format, experts said.

The city freely publishes more than 1,600 different sets of public information, including deeds, building violations, zoning codes of neighborhoods, and even every complaint logged to 311, that are constantly updated.

“New York has the highest quality data and the most amount of data of any other location in the United States, and as far as I know the world,” said L.D. Salmanson, a co-founder of the real estate data platform Cherre. “They make it available to you in a way that’s really helpful to people.”

The Bloomberg administration bet on open data has paid off big in recent years. The treasure trove of information has turned the city into the epicenter of the burgeoning real estate tech start-up market.

“It’s a big driver of why New York City is the home of proptech,” said Adrian Mercado, the chief information officer at B6 Real Estate Advisors. “It just makes it the perfect recipe for start-ups and other technology companies trying to innovate in the space.”

Aside from helping the proptech industry flourish in New York, the flow of data also made investment sales brokers’ lives easier when researching a property, said Ackman Ziff’s Andrew Sasson. Gone are the days when Sasson had to trust a building owner who was claimed his or her property had no violations, that he was not involved in any lawsuits and that yes, she definitely got the go-ahead from the city to redevelop the parcel.

“You can see if the permits are actually approved or if they weren’t,” Sasson said. “You don’t have to just take someone for their word.”

It’s not just L.A. that lags behind the city with open data. Brokers pointed to areas like New Jersey and Detroit — which doesn’t even have a public property record database — as being especially difficult to cull information from.

To make things worse, New York City’s five boroughs generally act as one when it comes to data, but L.A. County has 88 cities all with separate offices that each handle real estate information differently, Rafferty said.

Even in places that put out a decent amount of data, the sheer volume published by New York City makes it stand out.

“The amount of depth that you can get on a property and on a [land] lot in New York is second to none,” Mercado said. “New York has hundreds of data points. In any given city, you might have 20 to 25 data points.”

The situation was so dire before that when Salmanson had the idea for Cherre — which uses artificial intelligence to consolidate public and private real estate data — about eight years ago, his quest was frozen the moment he looked beyond Gotham.

“At the time, New York was the only one that provided that data,” Salmanson said. “When we saw it was only New York, we stopped.”

He eventually founded Cherre in 2016 once other cities began to catch up with New York, but the gap is still wide.

New York’s unchallenged status as the real estate data capital of the world is a relatively recent phenomenon. Just a decade ago, it could be nearly impossible to find much information about properties online and sometimes even offline.

“I remember the moment in August of 2006 when co-op sales became public record,” said Jonathan Miller, the president of the residential appraisal and consulting firm Miller Samuel. “That was the first time I’ve ever experienced greater transparency in the New York market, which traditionally had been in the 1950s as far as data sharing goes.”

Sasson said he might have been able to find out who signed a deed before, but it could be an exercise in frustration trying to find court cases or zoning restrictions.

“The permit information wasn’t necessarily available on the Department of Buildings website,” he said. “Emails [of owners] were impossible to find.”

All that started to shift in 2009 when — after a federal push by newly elected President Barack Obama to make government more transparent — Mayor Michael Bloomberg launched the NYC BigApps competition, according to Craig Campbell, the assistant director for policy operation of the New York City Mayor’s Office of Data Analytics, writing in a 2017 column for the Harvard Data-Smart City Solutions web publication.

The competition challenged developers to come up with applications using public data, and for it, the city released 170 datasets that included property sales and restaurant inspections, according to Campbell.

But what Mercado called the “watershed moment” happened in 2012 when the city passed the Open Data Law. Before that, the city gave out public information like police data and noise complaints only after a Freedom of Information Law request was filed. The new legislation stated it should be simply published without prompting.

“When New York City leaders passed comprehensive open data legislation in 2012, they set a high bar for open information and government transparency,” Campbell wrote. “Not only would the city open its data: It would open it all.”

The city has since released more than 1,600 datasets. Mayor Bill de Blasio built on the program in 2015 when he launched new tools and websites to make it more accessible and easier to visualize.

“It will be easier for people, even those with no programming experience — like myself — to find the information they want, and better ways to utilize that information,” de Blasio said in a statement at the time.

Not everybody was thrilled about the law. It became much easier to pull up detailed information about a property, leaving some landlords feeling overexposed and others fearful of investing in multifamily, said Shaun Riney, an investment sales broker at Marcus & Millichap.

“You’re expected to, number one, be perfect as a property owner in New York City,” Riney said, noting that the city puts complaints online even if an inspector hasn’t checked it yet. “Number two, every potential imperfection is right out there in the open and you’re guilty until proven innocent. It’s not like that in many places.” 

And even with the amount of data available, there are still some blind spots. Mercado added that public records in Staten Island aren’t as robust as the other boroughs, and the city also doesn’t log the majority of leasing activity.

“Leasing data is difficult to acquire,” Rafferty said. “If you have lease-level data, then you can build up from there to sort of a bigger picture to what revenue looks like for a property.”

When it comes to accuracy, specifically, Miller added that New York’s data pales in comparison to cities like Miami, which has boosted its public data efforts in recent years.

“There’s a real quality issue here,” Miller said. “It’s still extremely primitive compared to many housing markets.”

Miami isn’t the only one challenging New York City’s spot as the top data city. San Francisco, Seattle, L.A. and Chicago have all improved their data sets in recent years. In 2016, Seattle Mayor Ed Murray signed an executive order requiring all city departments make the information more accessible to the public, CityLab reported.

“The other cities are kind of getting hip to the game,” said Zach Aarons, a co-founder of proptech venture firm MetaProp. “Those cities are moving in a good direction as it relates to getting all the data up online and allowing companies to use it.”

But even if they do manage to catch up, there’s plenty of reasons experts think New York will remain at the top of the proptech scene.

“There’s an ecosystem here to support a proptech company that doesn’t exist anywhere else,” Cherre’s Salmanson said, adding he thinks the city would be a proptech mecca even without the open data.

The city already had a robust real estate market, with most of the major players either entirely based in New York City or having a major presence.

“We’re right down the street from a number of our clients,” said Rafferty. “That ability to go out and meet with your customers is definitely a bigger advantage.”

And plenty of investors have popped up in recent years to fund start-ups and cement the city’s place as the proptech hub.

“That’s one major factor,” said Aarons. “There’s a confluence of all that stuff.”