The October Okta Hit

Okta Security has identified adversarial activity that leveraged access to a stolen credential to access Okta's support case management system. The threat actor was able to view files uploaded by certain Okta customers as part of recent support cases. It should be noted that the Okta support case management system is separate from the production Okta service, which is fully operational and has not been impacted. In addition, the Auth0/CIC case management system is not impacted by this incident.

Note: All customers who were impacted by this have been notified. If you’re an Okta customer and you have not been contacted with another message or method, there is no impact to your Okta environment or your support tickets.

Within the course of normal business, Okta support will ask customers to upload an HTTP Archive (HAR) file, which allows for troubleshooting of issues by replicating browser activity. HAR files can also contain sensitive data, including cookies and session tokens, that malicious actors can use to impersonate valid users. Okta has worked with impacted customers to investigate, and has taken measures to protect our customers, including the revocation of embedded session tokens. In general, Okta recommends sanitizing all credentials and cookies/session tokens within a HAR file before sharing it.[1]

Attacks such as this highlight the importance of remaining vigilant and being on the lookout for suspicious activity. We are sharing the following Indicators of Compromise to assist customers who wish to perform their own threat hunting activity. We recommend referring to our previously published advice on how to search System Log for any given suspicious session, user or IP. Please note that the majority of the indicators are commercial VPN nodes according to our enrichment information.

IP Addresses:

23.105.182[.]19

104.251.211[.]122

202.59.10[.]100

162.210.194[.]35 (BROWSEC VPN)

198.16.66[.]124 (BROWSEC VPN)

198.16.66[.]156 (BROWSEC VPN)

198.16.70[.]28 (BROWSEC VPN)

198.16.74[.]203 (BROWSEC VPN)

198.16.74[.]204 (BROWSEC VPN)

198.16.74[.]205 (BROWSEC VPN)

198.98.49[.]203 (BROWSEC VPN)

2.56.164[.]52 (NEXUS PROXY)

207.244.71[.]82 (BROWSEC VPN)

207.244.71[.]84 (BROWSEC VPN)

207.244.89[.]161 (BROWSEC VPN)

207.244.89[.]162 (BROWSEC VPN)

23.106.249[.]52 (BROWSEC VPN)

23.106.56[.]11 (BROWSEC VPN)

23.106.56[.]21 (BROWSEC VPN)

23.106.56[.]36 (BROWSEC VPN)

23.106.56[.]37 (BROWSEC VPN)

23.106.56[.]38 (BROWSEC VPN)

23.106.56[.]54 (BROWSEC VPN)

User-Agents / While the following user-agents are legitimate, they may be rare in your environment given the release of Chrome 99 in March 2022.

Mozilla/5.0 (Windows NT 10.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.7113.93 Safari/537.36 (Legitimate, but older user-agent)

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.83 Safari/537.36 (Legitimate, but older user-agent)

Analysis:[2]

On 2 October 2023, the BeyondTrust security teams detected an identity-centric attack on an in-house Okta administrator account. The team immediately detected and remediated the attack through our own Identity Security tools, resulting in no impact or exposure to BeyondTrust’s infrastructure or to our customers. The incident was the result of Okta’s support system being compromised which allowed an attacker to access sensitive files uploaded by their customers.

The incident began when BeyondTrust security teams detected an attacker trying to access an in-house Okta administrator account using a valid session cookie stolen from Okta’s support system. Custom policy controls blocked the attacker's initial activity, but limitations in Okta's security model allowed them to perform a few confined actions. BeyondTrust’s own Identity Security Insights tool alerted the team of the attack, and they were able to block all access and verify that that attacker did not gain access to any systems.

The initial incident response indicated a possible compromise at Okta of either someone on their support team or someone in position to access customer support-related data. Concerns were raised regarding a breach to Okta on 2 October. Having received no acknowledgement from Okta of a possible breach, analysts persisted with escalations within Okta until 19 October when Okta security leadership notified BeyondTrust that they had indeed experienced a breach and we were one of their affected customers.

Okta has now issued a statement confirming the breach that we detected nearly three weeks ago. Again, while there was no exposure to BeyondTrust or our customers, analysts are sharing details of the attack to educate other Okta users and infosec professionals. For BeyondTrust customers who leverage our Identity Security Insights product, experts have also outlined the various detections that would alert you to this type of attack and recommendations to better control your attack surface and limit the possibility and impact of Okta-focused attacks.

Timeline Overview

2 October 2023 – Detected and remediated identity centric attack on an in-house Okta administrator account and alerted Okta.

3 October 2023 – Asked Okta support to escalate to Okta security team given initial forensics pointing to a compromise within Okta support organization.

11 October 2023 and October 13, 2023 – Investigators held Zoom sessions with Okta security team to explain why we believed they might be compromised.

19 October 2023 – Okta security leadership confirmed they had an internal breach, and BeyondTrust was one of their affected customers.

Attack Details:

On 2 October 2023, an Okta support agent requested a BeyondTrust Okta administrator generate a HAR file to assist in resolving an ongoing support issue the administrator was working on. HAR files are HTTP archives that can be generated by a web browser to log interactions with a website, in this case used for debugging an issue with the site. The administrator complied with the request and generated a HAR file containing an API request and a session cookie which was uploaded to the Okta support portal.

The Okta administrator’s account was protected with FIDO2 authentication, and policies within BeyondTrust’s Okta only allowed access to the admin console from managed devices with Okta Verify installed.

Within 30 minutes of the administrator uploading the file to Okta’s support portal an attacker used the session cookie from this support ticket, attempting to perform actions in the BeyondTrust Okta environment. BeyondTrust’s custom policies around admin console access initially blocked them, but they pivoted to using admin API actions authenticated with the stolen session cookie. API actions cannot be protected by policies in the same way as actual admin console access. Using the API, they created a backdoor user account using a naming convention like existing service accounts.

Our own instance of BeyondTrust’s Identity Security Insights, and tailored detections from our security teams, alerted us to several aspects of the intrusion. Analysts immediately disabled the backdoor user account and revoked the attacker’s access before the account could be used and preventing any further actions. They saw no evidence of other irregular activity across all other privileged Okta users in Identity Security Insights, no evidence of other suspicious Okta accounts being created, and no evidence of any unusual activity in the targeted user’s account before this incident.

Detailed Attack Timeline - Below is the detailed timeline of events:

2 October 2023 - A BeyondTrust Okta administrator uploads a browser recording (HAR file) at the request of Okta support related to ongoing troubleshooting of a non-security related support issue.

Within 30 minutes of the support file upload there was an attempt to access the BeyondTrust Okta admin console as the BeyondTrust Okta administrator using an IP address in Malaysia linked to anonymizing proxy/VPN services. Okta events are logged from this Malaysian IP however there were no prior authentication events or activity from this user in this location as we would normally expect.

Attacker was authenticated, but access to the Okta console was denied due to a non-default Okta security policy configuration enforced by BeyondTrust security teams:

Default deny access and only allow access if specific criteria is met.
Attacker denied console access due to policy requirement of requiring Okta Verify on a managed device.
Attacker attempts to generate a password health report using the underlying API of the Okta admin console.
The attacker attempts to gain access to main Okta dashboard but receives a policy challenge.

Note: It is important for Okta customers to enhance security policies through settings such as prompting admin users for MFA at every sign in. While this was within an existing session the attacker hijacked, Okta still views dashboard access as a new sign in, and prompts for MFA. Attacker uses Okta official API to create a fake service account named “svc_network_backup” to make it look like existing service accounts.

Note: Session cookies can be used to authenticate to official Okta API and in many cases, these lack the policy restrictions that apply to the interactive admin console.

The attacker acted quickly, but detections and responses were immediate, disabling the account and mitigating any potential exposure. BeyondTrust initiated an incident response process, immediately isolating and forensically investigating all systems and accounts associated with the administrator.

The investigation did not discover any indication of compromise however it did uncover the HAR file that had been generated for the support case. This was notable as these are only created in exceptional circumstances, in this instance for troubleshooting a support case. BeyondTrust contacted Okta support to inform them of our concerns while we continued to investigate.

3 October 2023 - Further investigation ruled out the possibility of the compromise originated from a BeyondTrust system leading us to conclude the Okta support system was likely compromised. Requested Okta support to escalate to their information security team given our concern that Okta was likely compromised, and other Okta customers might be exposed. No known compromise or ongoing security incident was communicated by Okta.

11 October 2023 - Okta Support Zoom meeting with a member of their information security team where we shared our findings and requested additional log data from Okta related to support case data access. Okta committed to providing the requested logs and working with us. No known compromise or ongoing security incident was communicated by Okta.

13 October 2023 - Okta support logs were received but contained several discrepancies. BeyondTrust requested more detailed logs relating to the discrepancies and reiterated concerns that there was a high likelihood of compromise within Okta support and that we were likely not the only customer impacted. No known compromise or ongoing security incident was communicated by Okta.

19 October 2023 - Call with Okta Security Leadership who notified investigators that there was a breach at Okta and we were one of the customers exposed during that breach.

20 October 2023 - Coinciding with Okta’s public announcement, a decision was made to publish this blog with detailed information including indicators of compromise to provide information to the security community and protect mutual customers.

BeyondTrust would like to thank Okta for working with us to protect mutual customers. We appreciate their transparency in reporting this breach, notifying affected customers, and highlighting further investigative steps.

Identity Security Insights Detections Specific to this Discovery - The following are detections and recommendations available within BeyondTrust’s Identity Security Insights solution would have triggered for Insights customers if they were targeted by the techniques used in this Okta attack.

Okta session hijacking: Attackers steal Okta session cookies and use them to access Okta from infrastructure they control, allowing them to bypass most MFA and security controls related to authentication. This detection looks for suspicious sessions appearing without an authentication event that are consistent with session hijacking.

Okta user performed administrative action using a proxy: This attacker, and other Okta-focused attackers like Scattered Spider often use proxies to login as privileged users and perform sensitive administrative actions, but legitimate users rarely do.

Okta admin privileges were granted to a user: Attackers often attempt to escalate privilege, or grant privilege to backdoor accounts. This information-level detection highlights all Okta admin assignments. These assignments are typically rare and usually occur within an established process.

Okta password health report generated: This report is generated rarely in most environments we monitor. This information-level detection highlights when that happens in case the activity is suspicious.

Okta user with some level of admin access uses MFA vulnerable to SIM swapping: Our incident response process was significantly faster because the admin user used FIDO2 for MFA, allowing us to rule out attacker-in-the-middle phishing as the mechanism for the token theft. Posture recommendations for privileged users give identity security professionals incremental changes they can make to better protect these crucial accounts.

If you are currently using Insights, please review your findings for any applicable detections based on the details below and feel free to reach out to BeyondTrust Support for help reviewing your own environment’s potential exposure from Okta’s breach. If you are not currently an Identity Security Insights customer but would like to leverage our free trial to assess your environment, please contact us.

Indicators of Compromise:

Access to Okta admin functions through proxy (isproxy: true in Okta log events)
Access to Okta from IPs 202[.]59.10.100 or 23[.].105.182.19
Access to Okta, especially Okta admin functions, from VPS/hosting providers. (Especially: VPS Malaysia, LeaseWeb.)
Access to Okta with this user agent for an outdated version of chrome for MacOs: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.3538.77 Safari/537.36
Okta account created via REST API with name svc_network_backup, or another name mimicking existing, legitimate accounts.
Activity against endpoints like /reports/password-health/async_csv_download_schedule?, which are typically used from Okta Admin Console UI only, without any corresponding admin console login
Okta activity for a user without any clear indication that the user authenticated (e.g. a user.session.start event for that user from a similar geographic area)
Admin console login attempts that are denied by policy without a subsequent successful login to admin console from the same user within an hour

Other Notes - Okta have recently updated their KB articles relating to the creation and sanitization of HAR files, we recommend reviewing these. We did not see any failed attempts to use the stolen session after its expiration, but this may be because actions attempted with expired sessions do not appear in Okta logs.

X-Industry

The October Okta Hit

Comments