Delayed delivery of webhooks to a subset of stores

Incident Report for BigCommerce

Resolved

The backlog of webhook events that failed to send between 3 PM UTC and 5:30 PM UTC has now been processed.
Posted Nov 04, 2019 - 19:42 CST

Update

The backlog of webhook events that failed to send between 3 PM UTC and 5:30 PM UTC is still processing. The ETA for all of them to be delivered is now 30 minutes.
Posted Nov 04, 2019 - 17:58 CST

Update

The backlog of webhook events that failed to send between 3 PM UTC and 5:30 PM UTC is currently being processed. The ETA for all of them to be delivered is 1h30min.
Posted Nov 04, 2019 - 16:00 CST

Update

BigCommerce Engineers are continuing to work towards processing the backlog of webhook events that failed to send between 3 PM UTC and 5:30 PM UTC. New webhook events are being delivered normally.
Posted Nov 04, 2019 - 14:17 CST

Monitoring

New webhooks are being delivered normally. We are currently processing a backlog of messages that failed to send between 3:00 PM UTC - 5:30 PM UTC.
Posted Nov 04, 2019 - 11:52 CST

Identified

BigCommerce engineers have identified an issue that is causing delayed delivery of webhooks to a subset of stores. The issue has been identified and work is underway to mitigate the issue. We will have another update shortly.
Posted Nov 04, 2019 - 10:58 CST

Update

We are continuing to investigate this issue. We will provide another update shortly.
Posted Nov 04, 2019 - 10:25 CST

Investigating

BigCommerce engineers are investigating an issue that is causing delayed delivery of webhooks to a subset of stores. We will provide another update shortly.
Posted Nov 04, 2019 - 09:54 CST