# I am the Watcher. I am your guide through this vast new twtiverse.
#
# Usage:
# https://watcher.sour.is/api/plain/users View list of users and latest twt date.
# https://watcher.sour.is/api/plain/twt View all twts.
# https://watcher.sour.is/api/plain/mentions?uri=:uri View all mentions for uri.
# https://watcher.sour.is/api/plain/conv/:hash View all twts for a conversation subject.
#
# Options:
# uri Filter to show a specific users twts.
# offset Start index for quey.
# limit Count of items to return (going back in time).
#
# twt range = 1 17
# self = https://watcher.sour.is/conv/verezuq
@darch This is an interesting read, thanks for sharing! π
@darch This is an interesting read, thanks for sharing! π
@darch This is an interesting read, thanks for sharing! π
@darch This is an interesting read, thanks for sharing! π
From the article:
> Unfortunately, the combination of these three factors creates a perfect storm6 which means that now, every time I post something that gets lots of engagement, I essentially end up carrying out a denial of service attack on myself.
π³
From the article:
> Unfortunately, the combination of these three factors creates a perfect storm6 which means that now, every time I post something that gets lots of engagement, I essentially end up carrying out a denial of service attack on myself.
π³
From the article:
> Unfortunately, the combination of these three factors creates a perfect storm6 which means that now, every time I post something that gets lots of engagement, I essentially end up carrying out a denial of service attack on myself.
π³
From the article:
> Unfortunately, the combination of these three factors creates a perfect storm6 which means that now, every time I post something that gets lots of engagement, I essentially end up carrying out a denial of service attack on myself.
π³
@prologic the solution, of course, is to have many servers, each with less users. Anything that grows too big is bound to either have problems, or become too expensive to maintain, or both. So, servers need to be architected to be light weight, and relatively simple to operate.
@bender Yeah
> So, servers need to be architected to be light weight, and relatively simple to operate.
Precisely's yarnd's design and design choices π
@bender Yeah
> So, servers need to be architected to be light weight, and relatively simple to operate.
Precisely's yarnd's design and design choices π
@bender Yeah
> So, servers need to be architected to be light weight, and relatively simple to operate.
Precisely's yarnd's design and design choices π
@bender Yeah
> So, servers need to be architected to be light weight, and relatively simple to operate.
Precisely's yarnd's design and design choices π
@prologic It's nuts! I don't know much about sidekiq but it sounds to me like every single time someone posts, they generate a job per instance just to propagate that post out to others. It's such a heavy-handed way to do message passing, which you can do at a rate of 10k or 100k per second on not great hardware using appropriate middleware (or corresponding libraries).
@bender With the architecture they have, more servers might make the performance worse. If someone joins your server and has a million followers, then every time they post, the mastodon software generates one sidekiq job on your server for each server among the servers used by that user's million followers. More servers in the mastodon network, more sidekiq jobs on your server each time heavily-followed person posts. Assuming there's a fair amount overhead just in the job creation/completion part of the process, increasing the number of servers increases the fraction of compute time spent on overhead. There's some optimal balance point somewhere between one gigantic server and many many one-person servers. I don't know enough to guess what that is, but given what I do know it sure sounds like the architecture they have has some edge cases where scaling is very bad with respect to number of servers.
@abucci right, how Mastodon does it, because how it is architected, is problematic. I donβt think ActivityPub was meant, and made, to scale to the thousand/millions of followers either.