← Go back to Sincoolka status page

Services offline (internet resolved)

May 14, 2025 at 11:28 UTC

Síťové prvky SIN Weby sin.cvut.cz, wiki, sincoolka.cz SINIS

Resolved after 29h 44m of downtime. May 15, 2025 at 17:12 UTC

Update 15.5. 19:15: All of the services have been restored and are running again. Unfortunately we have lost data from 2025-05-14 4am to approx 13:17 due to the last backup being at approx. 4am of that day. This also concerns e-mails. If you were expecting an important e-mail, please check if you have received it, and ask the sender to resend, if necessary.
We will implement replication across the VM cluster on the critical services to prevent such major data loss in the future; the backups will also be performed more often.

16:10 SINIS and websites should be online, albeit a bit slower now due to DNS resolving only half-functional. All changes performed from 4am have been lost for the time being. The secondary DNS resolver is working but the primary is being restored; once that is up, operations should speed up significantly. E-mail restore will also be performed in the evening hours to recover e-mails sent/received from approx 4am to 1pm.

14:39 Connectivity should be up now; if not, please disable and then reconnect your Wi-Fi, or disconnect and reconnect your network cable. Other services will take longer to restore.

The internet is currently down because of a critical server drive malfunction. We are currently trying to restore the systems from backup but it might take time. Thank you for bearing with us.

E-mails @sin.cvut.cz are also down! Please, do not e-mail admin (at) sin.cvut.cz at this moment. We will only receive your messages once the services are back up, and the message delivery might be delayed by a few hours.