Hosted Mac offering is not functional

Incident Report for Harness

Postmortem

Summary

One of our scheduler services experienced resource issues that temporarily impacted Hosted Cloud Mac builds. The issue was detected by our internal monitoring, and corrective action was taken to restore service.

Root Cause

One our services got impacted to to resource issues, on investigating further the GC job was misconfigured causing the impact to our Hosted Mac builds. These issue were proactively alerted by our internal systems and corrective measure have been put in place to resolve the issue.

Resolution

Our engineering team took immediate remediation steps to address the resource contention, which restored normal operation.

Next steps

We are adding additional monitoring and safeguards to detect and prevent similar issues from occurring in the future.

Posted Sep 18, 2025 - 15:33 PDT

Resolved

This incident has been resolved.
Posted Sep 15, 2025 - 10:28 PDT

Monitoring

A fix has been implemented and we are monitoring the results.
Posted Sep 15, 2025 - 10:08 PDT

Investigating

We are currently investigating this issue.
Posted Sep 15, 2025 - 09:49 PDT
This incident affected: Prod 3 (Continuous Integration Enterprise(CIE) - Mac Cloud Builds), Prod 1 (Continuous Integration Enterprise(CIE) - Mac Cloud Builds), and Prod 2 (Continuous Integration Enterprise(CIE) - Mac Cloud Builds).