High system load across all nodes
5:22am We are currently experiencing a high load on our systems, we are working to resolve this.
5:36am We have located the issue. This was caused by 2 drives on the storage cluster causing a high IO wait time, but their state remained active rather than go into failed mode. The drives were forced into a failed mode and the cluster is now returning to normal load levels. We are investigating with our vendors as to why the drives didn’t register as failed in a more timely manner.