Code Repository and Artifact registry are impacted in Prod2/Prod3

Incident Report for Harness

Postmortem

Summary

At 08/15/2025 13:15PST, Due to a bug which affected the secrets logic , got deployment in Prod2 and Prod3 2 customers were impacted due to incorrect mapping of accountId.

Mitigation

We released a hot fix to resolve the issue by updating Github secret.

Root Cause:

We had invalid secrets due to a recent secret rotation as a part of compliance. One of the services did not get the updated secrets.

Next Steps

Add Monitoring and automation to ensure services are able to validate secrets early in the deployment lifecycle.

Posted Sep 08, 2025 - 22:28 PDT

Resolved

This incident has been resolved.
Posted Aug 15, 2025 - 14:33 PDT

Monitoring

A fix has been implemented and we are monitoring the results.
Posted Aug 15, 2025 - 14:20 PDT

Identified

The issue has been identified and a fix is being implemented.
Posted Aug 15, 2025 - 14:00 PDT

Investigating

We are currently investigating this issue.
Posted Aug 15, 2025 - 13:54 PDT
This incident affected: Prod 2 (Code Repository, Artifact Registry) and Prod 3 (Code Repository, Artifact Registry).