FME API write operations started returning 499 errors

Incident Report for Harness

Postmortem

Summary

On August 20, 2026, between 10:24 and 14:55 UTC, a subset of FME writes failed. Writes made from the FME UI and writes made with Harness access tokens (PATs and SATs) were not affected. Runtime flag evaluation continued to work normally. The issue was mitigated by reverting a recent authentication change in a shared governance service, and affected writes returned to normal by 14:55 UTC. Status: https://status.harness.io/incidents/rhthgm7d5dkz

Root Cause

A change in how a shared governance service authenticated inbound calls resulted in some FME writes being rejected. Those writes used service-to-service credentials that the governance service could no longer verify after the change. FME surfaces a governance failure to the client as HTTP 499, the same status used when a governance policy intentionally denies a change. Because 499 is a valid, expected response in that deny path, the failures did not look like an outage on our alerts, and the incident was identified from customer reports rather than internal detection.

Impact

  • A subset of FME writes failed during the window, primarily those made using legacy Split API keys or change request scheduling.
  • Writes made from the FME UI were not impacted.
  • Writes using Harness access tokens (PATs and SATs) were not impacted.
  • Runtime flag evaluation continued normally.
  • No data loss occurred. Failed writes did not apply.

‌

Remediation

Reverted the governance-service authentication change. Affected writes returned to normal immediately.

Action Items

To prevent such issues from happening again,

  • Harness will return a distinct error (not 499) when a write fails because governance could not be evaluated, so it is not confused with an intentional policy denial.
  • Add alerting on the governance evaluation call itself, rather than relying on the client-facing status code.
  • Expand authentication support for policy evaluations.
  • Expand automated coverage for additional write scenarios.
Posted Aug 23, 2026 - 14:46 PDT

Resolved

This incident has been resolved.
Posted Aug 20, 2026 - 08:02 PDT

Monitoring

A fix has been implemented and we are monitoring the results.
Posted Aug 20, 2026 - 07:56 PDT

Identified

The issue has been identified and a fix is being implemented.
Posted Aug 20, 2026 - 07:47 PDT

Investigating

We are currently investigating this issue.
Posted Aug 20, 2026 - 07:32 PDT
This incident affected: Prod 1 (FME).