Recent Community Downtime
-
Hi everyone,
You might have noticed the community forum has been experiencing intermittent downtime the last couple of weeks. We're working on mitigating this and minimizing any potential downtime!At first glance it looks like we're getting a lot of bot traffic that's clogging things up. We've implemented some additional monitoring and logging on the server running the forum to understand exactly what's going on before implementing any fixes or solutions.
I'll drop updates in this thread as they become available.
-
Update:
- Determined the downtime is being caused by the database used by the community saving snapshots of itself to disk
- The db snapshot uses up all of the available server memory -> forcing the server to run a process to swap memory to disk -> the swap process uses up all of the available server CPU -> making the community unreachable
- Have not yet determined why the database is now saving snapshots more frequently
Next Actions:
- Move the community to a server instance with more available memory to see if that helps with uptime
- What this means:
- We'll be making a snapshot of the community and launching it on a new instance
- Any posts made between now and the launch of the new instance won't be saved!
- I'll post an update in this thread when the migration is complete
Update Aug 29, 2024 at 2:30PM EST
- Successfully migrated community server to an instance with more memory
- Will continue to monitor the community and take additional action as necessary!
- Determined the downtime is being caused by the database used by the community saving snapshots of itself to disk
