# I am the Watcher. I am your guide through this vast new twtiverse.
#
# Usage:
# https://watcher.sour.is/api/plain/users View list of users and latest twt date.
# https://watcher.sour.is/api/plain/twt View all twts.
# https://watcher.sour.is/api/plain/mentions?uri=:uri View all mentions for uri.
# https://watcher.sour.is/api/plain/conv/:hash View all twts for a conversation subject.
#
# Options:
# uri Filter to show a specific users twts.
# offset Start index for quey.
# limit Count of items to return (going back in time).
#
# twt range = 1 7
# self = https://watcher.sour.is/conv/du3afsq
On the call we were talking about how Mastodon servers DDoS websites when they generate link previews: https://www.jwz.org/blog/2022/11/mastodon-stampede/ There's some interesting questions about how to do this more efficiently without a bad user experience.
@ocdtrekkie I read a thread on why this happens on mastodon once but didn't keep a link. I recall thinking "oh my god they don't even do basic caching????" I think it's almost literally the care that when one person with a lot of followers posts a web page with a preview, every single one of those follower's clients will download the preview. Obviously the instance could download it once and cache it, and then send it to every instance user. Then this kind of ddos wouldn't be a problem for now because there aren't very many instances.
@abucci There are tens of thousands of Mastodon servers. I believe the hit is caused by the servers all checking the link at once, not the clients.
@ocdtrekkie approximately 8,700 servers up according to https://joinmastodon.org/servers . That many requests should not cause issues on a reasonably configured and provisioned modern web server. It should barely be noticeable on most.
I don't remember the specifics anymore, but it's definitely dumber than you're suggesting. There are many orders of magnitude more requests generated than are necessary, and I'm pretty certain that scales with number of followers, not number of instances.
@darch hmm, I think that's a different issue (bit also bad). That one affects the mastodon server, and could also be helped with a more efficient use of resources on an instance. There's (I think) a separate issue with how posts that include web previews are fetched, which affects the remote web server too.
It's possible I'm misremembering.
@ocdtrekkie on second thought, I'm probably misremembering this. I'll have to go dig up the post I'm thinking of. It could be you're right about how it works but there was some other inefficiency in it that gave me pause.