Summary

On April 8, 2026, customers in certain production environments experienced degraded performance and intermittent failures while accessing the platform. This impacted login functionality and execution of new and existing tasks.

Root Cause

A spike in internal task processing caused excessive load on the service, leading to resource exhaustion and degraded performance across multiple service instances.

Impact

Customers in affected environments experienced:

• Slowness and failures during login
• Inability to start new tasks in some cases
• Failures in ongoing executions

Remediation

‌

Immediate:

Stabilized the system by resetting affected components and restoring service capacity, which allowed the platform to recover.

Permanent:

Introduced safeguards to limit resource-intensive operations and prevent unbounded processing under high load conditions.

Action Items

To prevent such issues from happening again, Harness will

• Add limits to high-volume internal processing paths
• Audit and enforce safeguards across similar workflows
• Improve system resilience under burst load scenarios
• Enhance monitoring to detect abnormal load patterns earlier