[osc-fr1] App and and database logs are not available

» View Event Details | Created Thu, 04 Jul 2024 07:45:00 +0000

Resolved

The situation is now back to normal.

The incident is considered as close on our side. If you are still facing issues, you can contact our support

Posted: Thu, 04 Jul 2024 09:04:00 +0000

Update

The logs are being ingested again, there may be a little delay in displaying the logs in real time in the next few minutes, we will inform you when the situation will be completely back to normal.

Posted: Thu, 04 Jul 2024 08:32:00 +0000

Update

The root cause has been identified and our operators are working on a fix.

We will keep you updated.

Posted: Thu, 04 Jul 2024 08:11:00 +0000

Investigating

A minor incident is ongoing: Logs are currently not displayed as expected. Our operators have been notified and are investigating the issue.

We will keep you updated

Posted: Thu, 04 Jul 2024 07:45:00 +0000

[osc-fr1] A DB node is misbehaving

» View Event Details | Created Fri, 05 Jul 2024 15:12:00 +0000

Resolved

All databases are up and running. If you still have issues please contact us via the support.

Posted: Fri, 05 Jul 2024 16:20:00 +0000

Update

All databases have been restarted. We are now executing post-incident checks.

Posted: Fri, 05 Jul 2024 16:05:00 +0000

Update

The server has been restarted and databases are being restarted.

Posted: Fri, 05 Jul 2024 15:32:00 +0000

Update

The server will be restarted. Databases on a starter plan will face downtime during this operation. Databases on a business plan will failover to their secondary instance without downtime. We will keep you updated

Posted: Fri, 05 Jul 2024 15:17:00 +0000

Investigating

One of our database server is mis-behaving. Our operators have been notified.

Posted: Fri, 05 Jul 2024 15:12:00 +0000

[osc-fr1] App and and database logs are not available

» View Event Details | Created Mon, 08 Jul 2024 07:15:00 +0000

Resolved

The situation is now back to normal. The incident is considered as close on our side. If you are still facing issues, you can contact our support

Posted: Mon, 08 Jul 2024 08:55:00 +0000

Update

The logs are being ingested again, there may be a little delay in displaying the logs in real time in the next few minutes, we will inform you when the situation will be completely back to normal.

Posted: Mon, 08 Jul 2024 08:26:00 +0000

Update

The root cause has been identified and our operators are working on a fix. We will keep you updated.

Posted: Mon, 08 Jul 2024 07:49:00 +0000

Investigating

A minor incident is ongoing: Logs are currently not displayed as expected. Our operators have been notified and are investigating the issue. We will keep you updated

Posted: Mon, 08 Jul 2024 07:15:00 +0000

[osc-fr1] Application and database metrics access is down, impacting autoscaling functions

» View Event Details | Created Tue, 09 Jul 2024 13:19:00 +0000

Resolved

Metrics and logs are processed correctly. The incident is now over.

Posted: Tue, 09 Jul 2024 13:42:00 +0000

Update

Metrics are visible again in the dashboard. Access to logs is also impacted and may take some time to appear in the application/database dashboard.

Posted: Tue, 09 Jul 2024 13:41:00 +0000

Investigating

Access to application and database metrics is currently unavailable. This incident also impacts autoscaling functionalities.

Posted: Tue, 09 Jul 2024 13:19:00 +0000

[osc-secnum-fr1] Incident impacting deployments /scale up

» View Event Details | Created Mon, 15 Jul 2024 09:20:00 +0000

Resolved

The incident has been resolved on Outscale side. We have confirmed that their fix is indeed working and rolled back the temporary workarounds.

Posted: Mon, 15 Jul 2024 12:37:00 +0000

Update

We've set up a workaround for the "one-off" part. All our services are now up and running again. The incident is still considered open at this point until the root cause has been resolved on the infrastructure provider's side.

Posted: Mon, 15 Jul 2024 12:01:00 +0000

Update

The problem has been identified as caused by our infrastructure provider, an incident is open on their side: https://status.outscale.com/incidents/d467qvkf4qrd We have implemented a workaround to restore service. Deployments and scale up operation are working as expected. Booting a One-off container is currently impossible.

Posted: Mon, 15 Jul 2024 09:42:00 +0000

Investigating

An incident is currently impacting application deployment. Scale-up operations are also impacted.

Posted: Mon, 15 Jul 2024 09:20:00 +0000

[osc-fr1] Deployments are not working as expected

» View Event Details | Created Thu, 18 Jul 2024 13:00:00 +0000

Resolved

The deployment pipeline is working normally and deployments are processed as expected. We consider the incident closed. If you are still facing issues, you can contact our support.

Posted: Thu, 18 Jul 2024 13:20:00 +0000

Update

The situation seems to be almost back to normal, Deployments are being progressively processed.

Posted: Thu, 18 Jul 2024 13:13:00 +0000

Investigating

The deployment pipeline is misbehaving and some jobs are being queued. Our operators are working on fixing the issue. We will keep you updated of the situation evolution.

Posted: Thu, 18 Jul 2024 13:00:00 +0000

[osc-fr1] A DB node is misbehaving

» View Event Details | Created Fri, 19 Jul 2024 00:26:00 +0000

Resolved

All databases are up and running. If you still have issues please contact us via the support.

Posted: Fri, 19 Jul 2024 00:52:00 +0000

In Progress

One of our database server is misbehaving. The server will be restarted. Databases on a starter plan will face downtime during this operation. Databases on a business plan will failover to their secondary instance without downtime. We will keep you updated

Posted: Fri, 19 Jul 2024 00:26:00 +0000

[osc-fr1] A DB node is misbehaving

» View Event Details | Created Sun, 21 Jul 2024 00:23:00 +0000

Resolved

All databases are up and running. If you still have issues please contact us via the support.

Posted: Sun, 21 Jul 2024 01:02:00 +0000

Investigating

One of our database server is misbehaving. Our operators have been notified.

Posted: Sun, 21 Jul 2024 00:23:00 +0000

[osc-fr1] Applications unavailable

» View Event Details | Created Wed, 24 Jul 2024 06:53:00 +0000

Post-Mortem

A postmortem regarding this incident is available. If you have any questions, don’t hesitate to contact our support team. https://doc.scalingo.com/security/postmortems/spm-2024-003

Posted: Fri, 02 Aug 2024 16:51:00 +0000

Resolved

The situation is now stable.

The issue has been traced back to an automatic upgrade of one of our DNS service. A workaround has been deployed and a patch is being written.

Posted: Wed, 24 Jul 2024 07:32:00 +0000

Update

The situation is now returning back to normal, we continue to monitor the situation.

Posted: Wed, 24 Jul 2024 07:07:00 +0000

Update

The incident also impacts the osc-secunum-fr1 region. The root cause has been identified, and we are in the process of applying a patch.

Posted: Wed, 24 Jul 2024 07:06:00 +0000

Investigating

An incident is currently in progress on the osc-fr1 region. Applications are slow and/or unavailable. Our team has been alerted and we are working on it.

Posted: Wed, 24 Jul 2024 06:53:00 +0000

[osc-fr1] DB Node misbehaving

» View Event Details | Created Wed, 24 Jul 2024 10:23:00 +0000

Resolved

All databases are up and running.

If you're still experiencing difficulties, please contact us via the support.

Posted: Wed, 24 Jul 2024 10:48:00 +0000

Update

All databases has been restarted; our team is running post incident checks.

Posted: Wed, 24 Jul 2024 10:48:00 +0000

Update

Databases are restarting on the impacted node.

Posted: Wed, 24 Jul 2024 10:33:00 +0000

Investigating

One of our database hosting node is mis-behaving. Our team has been alerted.

Posted: Wed, 24 Jul 2024 10:23:00 +0000

[osc-fr1] Planned network maintenance on July 24th

» View Event Details | Created Wed, 24 Jul 2024 19:00:00 +0000

Resolved

The maintenance is now over. Access to the application is back to normal. If you still experience issue please contact us via the support.

Posted: Wed, 24 Jul 2024 19:03:00 +0000

Update

The maintenance is in progress access to the application will be degraded for the next few minutes.

Posted: Wed, 24 Jul 2024 19:02:00 +0000

Update

The maintenance will start in the next few minutes.

Posted: Wed, 24 Jul 2024 18:59:00 +0000

Scheduled

Planned maintenance will occur in the osc-fr1 region on July 24, 2024, from 21:00 CEST to 21:05 CEST. During this time, network access to the region will be degraded.

Impact:

- Network access to apps and databases exposed to the internet will be degraded for a few minutes during the maintenance.
- Internal networks (including app-to-database traffic) will not be affected.

If you have any questions, please respond to this email or reach out to us through the in-app support.

Sincerely,

Scalingo

Posted: Wed, 17 Jul 2024 14:29:00 +0000

[osc-fr1] A DB hosting node is misbehaving

» View Event Details | Created Fri, 26 Jul 2024 00:38:00 +0000

Resolved

All databases have been restarted.

If you still encounter issues, please contact our support

Posted: Fri, 26 Jul 2024 00:56:00 +0000

Update

The misbehaving server have been restarted.
Databases instances hosted on it are being restarted.

Posted: Fri, 26 Jul 2024 00:55:00 +0000

Investigating

One of our database server is misbehaving. Our operators have been notified.

Posted: Fri, 26 Jul 2024 00:38:00 +0000