It was 8:30 a.m. last Friday before a long weekend when Missouri's state court system learned it might have a cyber problem. IT staff discovered the state court system's cybersecurity software had detected unusual activity coming from a system administrator's account at 2 am, well outside business hours. Also suspicious? That system admin was on vacation, said the director of IT services for Missouri State Courts, during the recent National Center for State Courts' (NCSC) Court Technology Conference.
What ensued was days of activity from court staff and a tech vendor working to contain the threat and recover. This included removing two accounts and a compromised virtual server as well as requiring more than 5,000 court personnel to reset passwords. They also blocked outbound data traffic to an increasing number of countries.[1]
A picture emerged of an apparent reconnaissance campaign, conducted by threat actors who'd managed to exploit the Log4j vulnerability despite the court system adopting recommended mitigations. After three days working long hours, the courts' response teams and the vendor were satisfied that the threat had been purged, enabling the courts to open as normal again.
The ordeal highlighted both opportunities to improve as well as strengths that helped stop the incident from becoming a full-blown crisis. For one, it showed just how fortunate Missouri courts were to have had continuity of operations and IT emergency response plans in place.
Courts need to know ahead of time what they'll do during an incident to avoid wasting time and making decisions on the fly. It also means that if a key person is unavailable, someone else is ready, the Court said.
This includes identifying all kinds of details, like who IT contacts first after discovering an incident, be it the chief justice, law enforcement or vendor. Missouri opted to first reach out to its vendor to ensure assistance was coming right away, skipping the extra step of having the justice tell them to call. "It's not any disrespect in any way, shape or form, but it's better to get help on the way," it said. "No one's going to come and yell at me and say, 'Why did you call Microsoft before you came to talk to me?'"
Plans also should outline details like which systems to prioritize restoring, for Missouri, payroll tops the list, and who'll be on the emergency response team. It also means listing contact information for key groups. Missouri, for example, realized it had not pre-identified anyone at the FBI, said retired Judge Gary Lynch, who chairs the Missouri Court Automation Committee. Instead, during the incident they'd resorted to contacting the FBI via a supreme court justice who had a friend in the bureau, a fortuitous event but not something to rely on going forward.
Judge Lynch said plans should also make clear who'll make decisions, whether government will pay during a ransomware incident and whether to prioritize restoring systems even if doing so would hinder investigations into culpability. There were plenty of other lessons learned from this cyber incident, too.
On the technical side, bolstering defenses meant adopting new restrictions around data traffic. Previously, the court had blocked inbound traffic only. But as employees went through logs to understand the incident, they found indications of outbound traffic going to various other country IP addresses. After progressively blocking additional country IP addresses, the court decided to default to geoblocking all traffic directed outside North America, while allowing exceptions as needed should a judge be traveling, for example. "What viruses do is that they tend to come in through an IP in the United States and then they call the mother ship ... and so when we do the geoblocking out, it's like they can't call," the Court said.
Sophisticated attackers still could get around this by leveraging a US-based IP address to receive the outbound traffic, but such a measure can help thwart less-dedicated attackers. Geoblocking also isn't a precaution everyone can take. State executive branches may need to avoid such measures to do business with overseas parties, the Court said. But state judiciary branches generally deal with domestic court case participants.
There is a human dynamic to incident response, too. Managing employee stress was important, and they did this by going on food runs and managing shifts to ensure no one was on for 24 hours, it said. Working with a vendor with a large, global workforce also helped, because it meant the vendor had the staffing to continue working around the clock. Regular vendor progress reports also made the incident seem less bleak, Judge Lynch said.
Finally, the IT director advised courts to make sure their contracts with primary tech vendors include cyber assistance options.
This article is presented at no charge for educational and informational purposes only.
Red Sky Alliance is a Cyber Threat Analysis and Intelligence Service organization and has reported extensively on AI technology. For questions, comments or assistance, please contact the office directly at 1-844-492-7225, or feedback@redskyalliance.com
Weekly Cyber Intelligence Briefings:
- Reporting: https://www. redskyalliance. org/
- Website: https://www. redskyalliance. com/
- LinkedIn: https://www. linkedin. com/company/64265941
Weekly Cyber Intelligence Briefings:
REDSHORTS - Weekly Cyber Intelligence Briefings
https://attendee.gotowebinar.com/register/5504229295967742989
[1] https://www.securityinfowatch.com/cybersecurity/news/53073237/what-missouri-courts-learned-from-a-cyber-attack
Comments