The attack on US-based software provider Kaseya by notorious Russia-linked ransomware group REvil in July 2021 is estimated to have affected up to 2,000 global organizations. REvil targeted a vulnerability (CVE-2021-30116) in a Kaseya remote computer management tool to launch the attack, with the fallout lasting for weeks as more and more information on the incident came to light.
The event served as a reminder of the threats posed by software supply chains and sophisticated ransomware groups. Following is a timeline of the attack and the ramifications for the affected parties based on Kaseya’s incident update page and other sources.
[Editor’s note: This article, originally published on August 3, 2021, will be updated as new events occur.]
Kaseya REvil ransomware attack timeline
Friday, July 2: Kaseya’s incident response team detects a potential security incident involving its remote computer management tool Kaseya VSA
With an investigation underway, the company advised all on-premises customers to shut down their VSA servers until further notice, while also shutting down its SaaS servers as a precautionary measure. Kaseya’s internal team, alongside security experts, worked to determine the cause of the issue, alerting enforcement and government cybersecurity agencies, including the FBI and CISA. Kaseya said early indicators suggested that only a small number of on-premises Kaseya customers (40) were affected and that they had identified the vulnerability source. A patch was being prepared as of 10 p.m. EDT.
Saturday, July 3: Kaseya confirms that it was the victim of a cyberattack
Kaseya continued to strongly recommend its on-premises customers to keep VSA servers offline until it released a patch. It also advised any customers that were experiencing ransomware and had received communication from the attackers to avoid clicking on any links. The company announced it was making a compromise detection tool available to VSA customers to help them assess the status of their systems. Kaseya continued to contact impacted users and stated that CEO Fred Voccola would be interviewed on the incident on Good Morning America the following day.
Sunday, July 4: Kaseya announces delay in bringing data centers back online, releases compromise detection tool
Kaseya’s executive committee met and determined that, to best minimize customer risk, more time was needed before bringing data centers back online. In an interview on Good Morning America, Voccola said, “We are confident we know how it happened and we are remediating it.” The compromise detection tool was made publicly available via download, while the FBI and CISA issued their own joint guidance for MSPs and their customers impacted by the attack, urging them to take action such as ensuring backups are up to date and stored in an easily retrievable location that is air-gapped from the organizational network, reverting to a manual patch management process and implementing multi-factor authentication. REvil operators boasted on the group’s “Happy Blog” that more than a million individual devices were infected, and that they would provide a universal decryption key to Kaseya for $70 million in Bitcoin.
Monday, July 5: Kaseya claims fewer than 60 customers compromised, patch being tested
Kaseya promised that the patch for on-premises users was being tested and would be made available within 24 hours. Amid widespread media reports of the attack, the company estimated that it would be able to bring its SaaS severs back online between 4 p.m. and 7 p.m. EDT on July 6.
Tuesday, July 6: Kaseya adds security layers to SaaS infrastructure
Kaseya began configuring an additional layer of security to its SaaS infrastructure to change the underlying IP address of its VSA servers, allowing them to gradually come back online. However, upon rollout, an issue was discovered, delaying the release. Operations teams worked through the night to fix the issue with an update due the following morning. An update on the on-premises patch stated that 24 hours or less remained the estimated timescale. Across the pond, the UK’s National Cyber Security Centre said the impact of the attack on UK organizations appeared to be “limited”, though it advised customers to follow Kaseya guidance as a precaution.
Wednesday, July 7: Kaseya apologizes for SaaS and on-premises fix delays
Kaseya published a guide for on-premises customers to prepare for the patch launch and stated that a new update from Voccola was to be emailed to users clarifying the current situation. The company apologized for ongoing delays with SaaS and on-premises fix deployment.
Thursday, July 8: US government tells Russia it will be held accountable; Kaseya pushes back patch release
White House press secretary Jen Psaki said that a “high level” of US national security had contacted top Russian officials about the Kaseya attack to make clear its intentions to hold Russia responsible for criminal actions taking place within its borders. She also said that another ransomware-focused meeting between the two countries was scheduled for the following week.
Meanwhile, Kaseya set a new estimate of Sunday July 11 for the launch of the on-premises patch, while it was starting deployment to its SaaS infrastructure. Kaseya released two update videos, one from Voccola and another from CTO Dan Timpson, addressing the situation, progress, and next steps. The company also warned of spammers exploiting the incident by sending phishing emails with fake notifications containing malicious links and attachments. It stated that it would not send any email updates containing links or attachments.
Friday, July 9: Kaseya updates VSA hardening advice
Kaseya updated its VSA On-Premise Hardening and Practice Guide while executive vice president Mike Sanders spoke of the team’s continued work towards getting customers back up and running. He also raised awareness of ongoing, suspicious communications coming from outside Kaseya.
Saturday, July 10: Report says Kaseya was warned of security flaw
Kaseya said it remained on course to release the on-premises patch and have its SaaS infrastructure online by Sunday July 11 at 4 p.m. EDT. The latest video update from Sanders outlined steps companies could take to prepare for the launch. Meanwhile, a Bloomberg article reported that, according to ex-employees of the company, executives at Kaseya were warned of critical security flaws in its software on several occasions between 2017 and 2020, which they failed to address
Sunday, July 11: Kaseya release patch, begins SaaS restoration
Kaseya launched the on-premises patch and began restoring its SaaS infrastructure ahead of the 4 p.m. target. As of 10 p.m. EDT, it claimed to have 60% of SaaS customers live and servers due online for the rest of its customers in the coming hours. Support teams were working with any on-premises customers requiring assistance with the patch.
Monday, July 12: SaaS restoration completed
The restoration of Kaseya’s SaaS infrastructure was complete as of 3:30 a.m. EDT. However, it was forced to carry out unplanned maintenance due to performance issues, causing a short downtime. It continued to support on-premises users with patch assistance.
Tuesday, July 13: REvil websites disappear
All REvil ransomware gang websites suddenly went offline, leaving security experts to speculate potential action by US or Russian governments. This left some victims unable to negotiate with REvil to recover data through a decryption key to unlock encrypted networks. At Kaseya, advisors prompted users to continue to review its various customer guides to dealing with the incident and getting back online.
Wednesday, July 14: Kaseya issues patch install check advice to customers
“When running the Kinstall patch on your VSA, if you chose to reinstall VSA and either unchecked the default option to install the latest patch, or reran the Reinstall VSA process a second time without the ‘install patch’ option selected, it’s possible your patch was not re-applied,” the company wrote. “While these are rare edge cases, we recommend that you verify that the latest patch was installed properly. We have made a tool that enables you to ensure the patch is properly installed.”
Friday, July 16: Victims struggle with decryption tool, Kaseya releases non-security patch
With REvil’s websites still offline, some victims struggled to unlock files and systems despite having paid for the decryption tool but with no way of contacting REvil for support. Kaseya announced it was releasing a non-security-related patch (9.5.7.3011) to fix functionality issues caused by enhanced security measures and other bugs. Deployments were estimated to begin on July 17 (SaaS) and July 19 (on-premises).
Saturday, July 17: First updated SaaS patch deployments go live
Monday, July 19: Remainder of updated SaaS patch deployments go live
Tuesday, July 20: New functionality patches released
Kaseya provided further patch updates (9.5.7.3015) to fix functionality issues and bugs, and made the updated on-premises patch available.
Wednesday, July 21: SaaS functionality updated
Kaseya again updated SaaS instances to remediate functionality issues and provide minor bug fixes. This resulted in a brief interruption (2 to 10 minutes) as services were restarted.
Thursday, July 22: Kaseya acquires universal decryption key
Kaseya announced it had obtained a universal decryption key for ransomware victims. “We can confirm that Kaseya obtained the tool from a third party and have teams actively helping customers affected by the ransomware to restore their environments, with no reports of any problem or issues associated with the decryptor,” the company wrote. “Kaseya is working with Emsisoft to support our customer engagement efforts, and Emsisoft has confirmed the key is effective at unlocking victims. Customers who have been impacted by the ransomware will be contacted by Kaseya representatives.” Across the industry, mass speculation arose as to exactly how Kaseya accessed the decryption tool and whether a ransom payment was involved.
Friday, July 23: Another functional patch and SaaS update released, Kaseya reportedly requests non-disclosure for decryptor
As news of the decryption key made global headlines, details of how it became available remained unclear. Meanwhile, Kaseya released a quick fix patch 9.5.7b (9.5.7.3015) for on-premises customers to resolve three non-security issues. All SaaS instances were also updated. According to a CNN report, Kaseya was requesting the signing of a non-disclosure agreement for customer access to the decryptor.
Saturday, July 24: Kaseya declines to comment on ransom payment
Security sources and outlets continued to speculate as to the details of how the decryption key was obtained, with Kaseya declining to comment on whether it had paid a ransom.
Monday, July 26: Kaseya says decryption tool “100% effective,” no ransom paid
Kaseya released the following statement on the decryption key: “Throughout this past weekend, Kaseya’s incident response team and Emsisoft partners continued their work assisting our customers and others with the restoration of their encrypted data. We continue to provide the decryptor to customers that request it, and we encourage all our customers whose data may have been encrypted during the attack to reach out to your contacts at Kaseya. The decryption tool has proven 100% effective at decrypting files that were fully encrypted in the attack.”
Despite claims that Kaseya’s silence over whether it had paid attackers a ransom could encourage additional ransomware attacks, the company argued that nothing was further from its goal. “While each company must make its own decision on whether to pay the ransom, Kaseya decided after consultation with experts to not negotiate with the criminals who perpetrated this attack and we have not wavered from that commitment. As such, we are confirming in no uncertain terms that Kaseya did not pay a ransom—either directly or indirectly through a third party—to obtain the decryptor.”
Friday, September 10: REvil resurfaces on Exploit to explain universal decryptor key error
As detailed in a blog post from cybersecurity company Flashpoint, REvil reappeared on Exploit on September 10, claiming to being back online through the use of backups. A REvil representative also explained how an error made by a REvil coder led to the decryptor tool being inadvertently released to Kaseya. “Our encryption process allows us to generate either a universal decryptor key or individual keys for each machine,” they said. “Then, in the process of generating the keys, we had to generate between 20 and 500 decryption keys for each [individual] victim [because the victims of the Kaseya attack all had networks of different sizes]. One of our coders misclicked and generated a universal key, and issued the universal decryptor key along with a bunch of keys for one machine.”
According to Flashpoint, REvil appeared to be fully operational after its hiatus, with evidence also pointing to the ransomware group making efforts to mend fences with former affiliates who have expressed unhappiness with the group’s disappearance.
