In wake of an ongoing protest against Reddit’s new API pricing plan and hostility towards third-party clients, many Redditors from private communities have begun exploring the fediverse as an alternative. The good news is that Lemmy and Kbin are relatively stable and ready for general use – the bad news is that there are far too many people trying to squeeze into far too few servers.

With the revelation that many subreddits containing millions of users may continue the blackout indefinitely, there is a strong possibility that the influx of thousands of new people could come pouring in to a part of the network that’s relatively young. Many instance admins, flagship owners included, are struggling to stay above water.

At the moment we are urgently working to solve major issues, such as optimizing slow database queries, ripping out the inefficient websocket API, and fixing a major security vulnerability (big thanks to deadcade). In addition we suddenly have to manage dozens of pull requests. To give us time to work on these priorities, it would be very beneficial if users could refrain from interacting with issue trackers when possible.

Lemmy Project Statement

The statement goes on to declare that the active monthly user count for the platform increased 25 times over the course of the month, and the daily amount of issues in the project’s tracker has gone from a handful any given day, to over a hundred tickets daily. The project has suddenly become the second biggest project on FediDB, in terms of user adoption and number of instances, right behind Mastodon. The project’s flagship instance, Lemmy.ml, also had to migrate to a beefy new server to handle the onslaught of daily service disruptions.

Kbin, which is compatible with Lemmy, was also feeling the heat with an extreme growth cycle. The project has spiked in the past month from an average of roughly 300 active users to over 42,000. Although its user population is much smaller than that of Lemmy’s, that kind of growth in such a short time has put a strain on their flagship server, kbin.social.

The first step was to upscale the VPS to a higher version (66.91EUR). It quickly turned out that it wasn’t enough. I had to enable CF protection just to keep the website responsive, but the response times were still very slow. At this stage, the instance was practically unusable.

The next step was a full migration to a dedicated server (100EUR, the current hardware). It can be done relatively quickly, so it resulted in a 5-minute technical break. Despite the much higher parameters, it didn’t get any better. It became clear that the problem didn’t lie there. I’m really frustrated when it comes to server administration. That was the moment when I started looking for help. Or rather, it found me.

Statement on Kbin’s growth, by Ernest

Kbin was so overwhelmed at one point, that it needed to reach out to Fastly for caching assistance and Cloudflare to keep the overwhelming traffic at a steady drip. Things have started to settle down somewhat, but for a lot of admins, this is now the new normal.

 Share