Lonza

All stalled indicator computations have been performed, and the incident is now over.

Redis capacity has been increased. Ticket creation is back to normal and indicator computations that were stalling and now being processed

We have identified the root cause of the incident. It is due to a lack of capacity in Redis, the technology we use for caching and to perform long-running operations, such as exports and computation of dynamic indicators. While the infrastructure was under capacity, computation of dynamic indicators stalled. We have increased the capacity of Redis and the new capacity is being deployed.

Indicators are stuck in a recomputing state and tickets are failing to get created

We have identified the root cause and have reverted the changes made that affected our API. The root cause was linked to a routine server dependency update, that was reverted to resolve the incident

We are currently experiencing an incident affecting our API, causing it to be unavailable. Our team is investigating the issue and working to restore service as quickly as possible. We will provide further updates shortly.

The performance issues (including slowness and potential access problems) reported earlier this morning, following our recent system maintenance, have now been fully resolved. Our team has completed the necessary corrective actions, and all services have returned to normal operation. We have confirmed that system performance is stable. We apologize for the disruption this may have caused and thank you for your patience and understanding while we worked to resolve it.

Our monitoring systems have detected performance issues impacting services following overnight maintenance completed early on April 23rd. Users may experience slowness or difficulty accessing the platform. Our engineers are investigating urgently. We apologize for the inconvenience and will share more information shortly.

Incident has been mitigated by doubling the number of server instances. We are carrying an investigation to identify the root cause.

Some requests take multiple minutes instead of less than a second. Several of our server instances are unhealthy. We are currently investigating.