Few Pipelines are running slow and some failures

Summary

Due to capacity issues on our primary database, pipelines latency for couple of our customers got degraded and errored out.

Root Cause:
One of the databases was under provisioned for the high workload experienced

Mitigation:
We scaled out our database and made sure there is enough headroom and capacity for the workload.

Next Steps:

• Add additional granular monitoring for our databases.
• Update thresholds for alerting for warning us on increase in utilization/under capacity.