Summary

On January 19, 2026, some customers experienced missing logs for certain CI pipeline steps with very short execution times. The issue was related to increased latency in the log streaming process, which caused log data for affected steps to be unavailable after execution completed. The issue was identified and mitigated promptly. 

Impact

During the incident window:

• CI pipeline steps with very short execution durations may have shown incomplete or missing logs after completion.
• The issue primarily affected steps that completed execution before log streaming was fully established.
• Pipeline execution itself was not impacted, and builds continued to run successfully.

There was no impact to pipeline correctness, execution results, or customer data, aside from log visibility for the affected steps.

Root Cause

The issue occurred due to increased latency in the log streaming initialization process. For very short-running steps, execution completed before the log stream was fully established. As a result, the system attempted to finalize logging before the stream was ready, leading to an empty log snapshot being recorded.

This behavior caused valid log data to be unavailable for a subset of short-duration steps under specific timing conditions.

Mitigation

As an immediate mitigation:

• The recent log service change contributing to increased stream latency was rolled back.
• This action restored normal log streaming behavior and prevented further log loss.

Service stability and log visibility were fully restored following the mitigation.

Action Items

To prevent recurrence and strengthen resiliency, the following improvements are being implemented:

‌

• Add safeguards to prevent log streams from being finalized before initialization is complete.
• Improve validation logic to ensure valid log data is not overwritten by empty or incomplete snapshots.
• Introduce additional protections for short-duration steps to ensure consistent log availability.
• Enhance monitoring around log streaming latency to enable earlier detection of similar conditions.