Activity Streams, Search & Downtime 01/02/2022

[Grant]

Apologies for the downtime earlier today and the issues with activity streams and search.

Everything should be fixed now but the search database needs to be rebuilt which means searches may be slow and not complete for the next few hours but, activity streams should already be caught up.

The SGL server ran out of disk space today - I hold my hands up, I knew it was running high and had a plan in place to upgrade shortly but what I didn't realise, is when it reached a certain percentage,  the search engine system we use would refuse to index any more and went into read only mode - this in turn caused over 100GB of error log messages to be generated thus taking the server from having 1-2 months of storage space left to 0 in a matter of hours.

Thanks to our forum sponsor FLO, we now have an additional 1tb of storage on the server so this issue is now resolved and should not crop up again anytime soon (famous last words!).

Sorry for any disruption in your viewing pleasure. If you notice any issues ongoing, please report as usual in the bugs forum.

Cheers,

Grant