Breach Notification Letter: Company Gives Minute-by-Minute Account of Its Cyber Attack

Social media memory company Timehop has a positive tone to its brand. "Timehop helps you celebrate the best moments of the past with your friends." 

Now, however, the company is remembering and sharing several of its worst moments, in a minute-by-minute account of its July 4, 2018, security breach that compromised 21 million records.

This is noteworthy because it is the most transparent breach notification letter we've ever read.

And really, the company has posted two letters: a breach notification overview and a minute-by-minute cyber attack timeline. The timeline is what we'll focus on here.

Breach notification letter: minute-by-minute cyber attack breakdown

For background, you'll want to know the following: Timehop says the attacker gained initial access to its systems in 2017 by compromising an employee's credentials, and then the hacker created his own administrative user account in the system.

The attacker then checked back occasionally but found no personally identifiable information (PII) available. A table on Timehop's "users" was still empty at that point. 

Then on June 22, 2018, the hacker discovered that the "users" table was in use, had been populated with PII, and that his credentials gave him access. After doing some reconnaissance, the hacker logs out.

And waits.

Now it is July 4th. The company's employees, like the rest of the United States, are out of the office for the national holiday. That helps ensure the success of this attack, as you will see.

July 4, 2018: the cyber attack begins

The hacker logs in at 2:04 p.m. on the 4th of July, and here is what happens next, minute-by-minute.

• 2:04 p.m. - [The Unauthorized User] logs in and begins to restore an existing snapshot of the database containing the Users table into a cluster they created, called 'Reusers.'
• [The Unauthorized User] continually checks alarms and monitors while the restore process is progressing.

  • The restoration of the snapshot has taken around 30 minutes and [The Unauthorized User] spun down the restoration process.

• 2:43 p.m. - [The Unauthorized User] resets the password to the production Users database.

• 2:43 p.m. - Internal alerts report the ‘Reusers’ database is available.

• 2:44 p.m. - [The Unauthorized User] initiates a deletion of the ‘Reusers’ cluster

• 2:49 p.m. - Internal alerts report the ‘Reusers’ cluster is closing down.

• 2:50~4 p.m. - Massive spike in DB reads of the Users production cluster is reported by internal alerting tools. Timehop application users are reporting black screens.

• 4:04 p.m. - Internal alerts report the service being down. A Timehop engineer investigates and tries to restart the database.

• 4:13 p.m. - The Timehop engineer discovers that the password has been changed.

• 4:16 p.m. - The Timehop engineer discovers that the database, while still password protected, is not behind a firewall and can be accessed by anyone with the password.

• 4:23 p.m. - The Timehop engineer resets the password to the database, Services start to come back up.

July 5, 2018: incident investigation and response 

• 6:09 a.m. - [The Unauthorized User] logs in using the API access key on [Employee]'s account and lists Cloud Computing Environment users, and logs out.
• 12:10 p.m. - Timehop engineers begin investigation into the event.
• 12:30 p.m. - Timehop engineers reviewing logs recognized suspicious patterns and conclude that we had been attacked. An incident is declared.
• 1:25 p.m. - Other Timehop engineers examining logs and artifacts confirm the incident, and began removing access to [The Unauthorized User]’s account and the [Employee's] account.
• 1:35 p.m. - Engineers begin enforcing MFA policy on all access to all accounts
• 2:02 p.m. - Cloud Computing Environment is considered secure.
• 2:47 p.m. - COO notifies Federal Law Enforcement
• 2:55 p.m. - CEO contacts cyber incident response company
• 3:11 p.m. - CEO notifies Board of Directors
• 3:22 p.m. - CEO and Incident Responder begin discussing engagement
• 4:45 p.m. - Incident Responder arrives on scene and begins investigation

National holiday helps hacker succeed

I have lost track of the number of sessions I've been in at our SecureWorld cybersecurity conferences where a CISO or Director of Information Security talks about attack attempts (and phishing efforts) ramping up around holidays, likely because they know IT security teams may be out of pocket.

In this case, the strategy worked. Kudos to Timehop for being so transparent:

"Timehop engineers had begun to implement security measures to restore services, however, they still believed that they were dealing with a maintenance issue. They did not immediately suspect a security incident for two reasons that in retrospect are learning moments. First, because it was a holiday and no engineers were in the office, he considered it likely that another engineer had been doing maintenance and changed the password. Second, password anomalies of a similar nature had been observed in past outage. He made the decision that the event would be examined the next day, when engineers returned to the office."

Incident response questions to ponder

There are certainly some things to think about, and the Timehop breach is a great reminder. Here are three questions that come to mind:

1. If you feel good about your incident response plan, have you taken a second look at how it might become less effective on a holiday or when key employees are on vacation?
2. How transparent will your incident response be to internal stakeholders, customers, and the public? Avoid a repeat of the 2018 Allscripts breach, when a failure to communicate made the reputational harm even worse.
3. Does your communications team have a plan to handle damage on social media channels, or will your company get burned like this after a breach?

Hopefully, Timehop's candor can help you improve at least one thing about your incident response plan or your breach notification strategy.