ElasticSearch server exposed the personal data of over 57 million US citizens

An ElasticSearch server that was left open on the Internet without a password has leaked the personal information of nearly 57 million Americans for almost two weeks, ZDNet has learned.

The leaky server was spotted by Bob Diachenko, Director of Cyber Risk Research for cyber-security firm Hacken, during a regular security audit of unsecured servers indexed by the Shodan search engine.

Also: Cathay Pacific breach leaks personal data on 9.4 million people CNET

The researchers said the ElasticSearch server --a technology used for powering search functions-- was leaking over 73GB of data, and that several databases were cached inside the server's memory.

Inside one of these databases, Diachenko said he found 56,934,021 records holding the personal data of US citizens.

In most cases, these records contained personal information such as first name, last name, email address, home address, state, ZIP code, phone number, and IP address.

But the leaky ElasticSearch server also contained a second cached database named "Yellow Pages," which Diachenko said held an additional 25,917,820 records, which appeared to be business entries. These latter records contained a little bit more information, such as names, company details, ZIP codes, carrier routes, latitude/longitude coordinates, census tracts, phone numbers, web addresses, email addresses, employees counts, revenue numbers, NAICS codes, SIC codes, and a few other fields.

While Diachenko said he spotted the server on November 20, he later discovered that the ElasticSearch instance had been first indexed by Shodan a week earlier, on November 14.

The researcher told ZDNet he was not able to identify who owned the exposed server but based on clues contained within the leaked databases, he said today in a report that he believes that Canadian data firm Data & Leads might be connected to the data, directly or indirectly. The company didn't respond to separate inquiries made by Diachenko, and later by ZDNet, before this article's publication. The company's website also went down after ZDNet reached out yesterday.

Also: Why 31% of data breaches lead to employees getting fired TechRepublic

The good news is that the server was eventually taken down, although it's unclear if Diachenko's tweets or emails might have had something to do with it. "The exact date when the server was secured is not known - it was just shut down a couple of days after our discovery," Diachenko told ZDNet.

In the meantime, the security researcher has provided a copy of the leaked data to data breach index service Have I Been Pwned (HIBP), and users will be able to search the site and see if their personal details leaked during this incident --and possibly take precautionary measures against a possible rise in email spam or automated robo-calls.

"HIBP will start sending alerts after our blog's publication," Diachenko said. Users who have HIBP accounts and have enabled email notifications should get an alert if their email address was leaked during this incident.

This is not the first time when an Internet-exposed ElasticSearch server has leaked a company's information. In the last two months, two other organizations--FitMetrix and Brazil's Federation of Industries of the State of São Paulo-- suffered similar leaks.

The root cause of all these ElasticSearch-based leaks is that server administrators don't set up passwords for their servers, which they later leave exposed on the Internet, where everyone can take a peek or download the data cached inside it.

In a blog post published in 2013, five years ago, Elastic, the company behind the ElasticSearch technology, said that ElasticSearch servers aren't meant to be exposed on the Internet, and they've been developed to be deployed for use in internal networks primarily, hence the reason servers don't perform authentication or authorization in default setups.

These are 2018's biggest hacks, leaks, and data breaches

Related stories:

• Amazon leaks users' email addresses due to 'technical error'
• Uber fined £900,000 by UK, Dutch privacy regulators over 2016 data breach
• Popular Dark Web hosting provider got hacked, 6,500 sites down
• Vision Direct reveals customer credit card leak, fake Google script may be to blame
• Security firm doxxes hacker who sold MySpace and Dropbox databases in 2016
• AWS rolls out new security feature to prevent accidental S3 data leaks