Login issues and propagation delays on FME

Incident Report for Harness

Postmortem

Summary

On February 27, 2026, at 22:46 UTC, customers using Feature Management experienced brief UI and Admin API unavailability. The issue lasted approximately 14 minutes and was fully resolved at 23:00 UTC. Feature flag evaluation and SDK operations were not impacted during this time.

Impact

  • Customers could not access or make changes in the Feature Management console or API.
  • All other operations, including flag evaluation via SDKs, continued to function normally.

Root Cause

A configuration change deployed during a release introduced database connection errors.

Remediation

Upon detection, we immediately reverted the configuration change. We then initiated data synchronization and cleared relevant caching layers to restore normal authentication and operation.

Action Items

  • We have implemented additional automated validation and safeguards to ensure configuration changes are validated prior to deployment.
  • We have also enhanced service health monitoring and alerting to proactively detect authentication latency and flag propagation delays, enabling faster mitigation in the future.
Posted Mar 04, 2026 - 09:23 PST

Resolved

Customers using FME would have experienced logging issues in addition to their Flag propagation delay from 22:46 UTC to 23:00 UTC. Services are back to normal now
Posted Feb 27, 2026 - 14:46 PST