Duration: 1 hour and 43 minutes

Affected Components: Regulatory AI

May 26, 21:41:00 GMT+0
Investigating - New Reg AI sessions are failing to be created for any sessions, including Maryland PSC files. The team is investigating..

May 26, 22:12:07 GMT+0
Identified - We've identified an issue affecting Reg AI session creation for the Maryland PSC (MDPSC) catalog only. New MDPSC sessions are not indexing correctly and may return incomplete results. All other catalogs are operating normally. Engineering has narrowed the root cause and is working on a fix. .

May 26, 23:24:24 GMT+0
Resolved - A hotfix was deployed, and new sessions involving Maryland PSC files are now progressing as intended. Our apologies for any inconvenience caused..

Duration: 1 minute

Affected Components: Platform

Mar 10, 15:15:50 GMT+0
Investigating - HData's platform cannot be accessed at the moment. This incident was created by an automated monitoring service..

Mar 10, 15:16:56 GMT+0
Resolved - HData platform is now operational! This update was created by an automated monitoring service..

Duration: 2 hours and 49 minutes

Affected Components: Platform, Library

Feb 20, 13:02:00 GMT+0
Identified - The team was made aware of a performance issue on our OpenSearch clusters, which is responsible for the search capabilities of many of our Catalog datasets. The issue has been identified and the team is working to resolve..

Feb 20, 15:26:46 GMT+0
Monitoring - The cluster is recovering, and 75% of shards are now assigned. Search functionality is steadily recovering, and error rates are dropping..

Feb 20, 15:51:00 GMT+0
Resolved - The OpenSearch clusters have now recovered, no errors have been detected for the last 18 minutes, and our QA team has confirmed that Library and Search are functioning normally. The issue is now resolved. Our apologies for any inconvenience caused..

Duration: 1 hour and 46 minutes

Affected Components: Regulatory AI

Feb 14, 19:58:17 GMT+0
Identified - The team has implemented a mitigation for all affected sessions over the past 48 hours. Book reports are now downloading as expected. The team is continuing to implement the mitigation while we investigate a fix..

Feb 14, 20:43:03 GMT+0
Resolved - This incident has been resolved. Our apologies for any inconvenience caused..

Feb 14, 18:57:00 GMT+0
Identified - Our team has been notified that book reports are intermittently failing to download. The team has identified the source of the problem and is actively applying mitigations to resolve it while investigating paths to a more permanent fix..

Duration: 49 minutes

Affected Components: Regulatory AI

Oct 24, 16:00:00 GMT+0
Monitoring - We have detected an issue with our Regulatory AI indexing process. A fix has been pushed and we are monitoring for resolution. Until resolved, any AI indexing is failing..

Oct 24, 16:34:04 GMT+0
Monitoring - The AI indexing queue is processing now. The latest ETA is 15 minutes until the queue clears. After that time, recent requests will process successfully at the typical speed. We are continuing to monitor the situation..

Oct 24, 16:48:49 GMT+0
Resolved - The underlying issue has been resolved, and the AI indexing queue has fully caught up. The issue is now resolved..

Duration: 2 hours and 12 minutes

Affected Components: Platform

Oct 20, 14:42:16 GMT+0
Monitoring - Our platform has mostly recovered, but AWS is still reporting elevated error rates in addition to continuing platform recovery. We are continuing to monitor the situation and have been running a platform-wide smoke test to ensure functionality is restored..

Oct 20, 15:12:08 GMT+0
Resolved - The latest error we recorded was at 10:39 AM ET. Our checks have been completed and we are no longer detecting any issues. .

Oct 20, 13:00:00 GMT+0
Monitoring - One of our platform infrastructure providers, AWS, began having issues early Monday morning in the us-east-1\. We have detected intermittent issues within the HData platform until about 9:30 AM. Our platform is showing signs of recovery. We are running checks on our platform and monitoring progress..

Duration: 21 minutes

Affected Components: Platform

Jul 15, 18:10:00 GMT+0
Investigating - Our monitoring detected an issue where we were returning HTTP 500s for a certain number of customers accessing our platform. We are investigating the issue..

Jul 15, 18:23:00 GMT+0
Monitoring - We have found the root cause and implemented a fix. We are now monitoring the situation to ensure it is resolved..

Jul 15, 18:31:00 GMT+0
Resolved - This incident has been resolved..

Duration: 3 hours and 19 minutes

Affected Components: Regulatory AI

Jun 12, 18:26:04 GMT+0
Identified - An issue has been identified with RegAI Pro. The root cause is a widespread Google Cloud Platform outage. Our team is opening a call bridge with Google resources now, and we are monitoring the situation..

Jun 12, 18:58:09 GMT+0
Monitoring - Google Cloud Platform is making progress on the outage, and our systems are showing signs of recovery. Our QA team is running tests now to confirm the operation of our Reg AI Pro system..

Jun 12, 19:44:56 GMT+0
Monitoring - Reg AI Pro sessions are visible again. Documents previously indexed by our system are queryable, and Reg AI Pro will respond to questions about them. However, our indexing system remains affected by the GCP outage. Our team is continuing to monitor the situation..

Jun 12, 20:28:35 GMT+0
Monitoring - Reg AI Pro is able to index some files now, but functionality is still impaired. We are continuing to monitor the situation and run tests after every update from Google or every 30 minutes, whichever comes first..

Jun 12, 21:31:07 GMT+0
Monitoring - Google is now estimating the time to restore functionality as less than 1 hour. We will report back the moment we detect service as restored..

Jun 12, 21:45:00 GMT+0
Resolved - Functionality has been restored as of 5:45 PM ET. Our apologies for any inconvenience caused..

Duration: 1 minute

Affected Components: Platform

May 9, 03:12:44 GMT+0
Investigating - HData's platform cannot be accessed at the moment. This incident was created by an automated monitoring service..

May 9, 03:13:49 GMT+0
Resolved - HData platform is now operational! This update was created by an automated monitoring service..

Duration: 2 hours and 45 minutes

Apr 17, 16:38:00 GMT+0
Investigating - User permissions are not associating correctly with Compliance usernames after a recent change. Many customers are unable to complete FERC filings as a result. The team is investigating the root cause..

Apr 17, 19:01:08 GMT+0
Identified - The root cause was identified as a security library update in the recent deployment. It will take several hours to resolve going that route, so the team is proceeding with a rollback to resolve the issue. .

Apr 17, 19:23:00 GMT+0
Resolved - The rollback completed, and testing shows that the incident is now resolved. Our apologies for any inconvenience caused..

Duration: 1 minute

Affected Components: Platform

Apr 13, 03:22:28 GMT+0
Investigating - HData's platform cannot be accessed at the moment. This incident was created by an automated monitoring service. The incident is caused by a failure of our Customer Identity Access Management provider to propagate SSL certificate updates. The issue was escalated to the provider..

Apr 13, 07:42:00 GMT+0
Resolved - HData platform is now operational after a workaround was deployed. Issue resolved..

Duration: 8 minutes

Affected Components: Gateway, Library, Regulatory AI, Data Hub, Platform

Feb 26, 22:12:00 GMT+0
Resolved - This incident has been resolved..

Feb 26, 22:04:00 GMT+0
Identified - An issue was detected with a routine deployment. The cause is identified, and the team is rolling back the associated change..