# I am the Watcher. I am your guide through this vast new twtiverse.
#
# Usage:
# https://watcher.sour.is/api/plain/users View list of users and latest twt date.
# https://watcher.sour.is/api/plain/twt View all twts.
# https://watcher.sour.is/api/plain/mentions?uri=:uri View all mentions for uri.
# https://watcher.sour.is/api/plain/conv/:hash View all twts for a conversation subject.
#
# Options:
# uri Filter to show a specific users twts.
# offset Start index for quey.
# limit Count of items to return (going back in time).
#
# twt range = 1 27
# self = https://watcher.sour.is/conv/cftbyia
@prologic, does this rings a bell to you? 159-196-9-199.9fc409.mel.nbn.aussiebb.net
@prologic it is kind of hammering my VPS, specifically netbros.com, every single day.
Also hammering ferengi.one.
The thing is, I don’t have a twtxt.txt file on netbros.com.
@bender I've blocked it from this pod for now 🤞Not sure which users are still trying to fetch the non-existent feeds sorry!
@bender I've blocked it from this pod for now 🤞Not sure which users are still trying to fetch the non-existent feeds sorry!
@prologic no worries mate, and thanks! I wonder if something could be done for feeds rendering 404, so that they get automatically “unfollowed”, and removed.
A twtxt.txt file should never spit out a 404, unless it’s no more.
@bender Unfoetunately that isn't actually true as it depend on the ingress architecture and networking.
@bender Unfoetunately that isn't actually true as it depend on the ingress architecture and networking.
@prologic if a twtxt.txt is not found, under which conditions will it be found again, and can something be done if say, it isn’t found for X amount of hours, days, months?
@prologic @bender Exponential backoff? Seems like the right thing to do when a server isn't accepting your connections at all, and might also be a reasonable compromise if you consider 404 to be a temporary failure.
@falsifian that sounds like a good compromise. Regardless of what @prologic wrote, a 404 is a 404.
@bender 404 could be indeed a temporary error if the file resides on a mounted remote filesystem and then the mount point fails for some reason. With a symlink from the web root to the file on the mount, the web server probably will not recognize the mount point failure as such. Thus, it might not reply with a 503 Service Unavailable (or something like that), but 404 Not Found instead. (I could be wrong on that, though.)
The right™ way is to signal 410 Gone if the feed does not exist anymore and will not come back to life again. But that's hard to come by in the wild. Somebody has to manually configure that in almost all situations.
But yes, as @falsifian points out, exponential backoff looks like a good strategy. Probably even report a failure to users somehow, so they can check and potentially unsubscribe.
@lyse right, now, on this:
> "The right™ way is to signal 410 Gone if the feed does not exist anymore and will not come back to life again. But that’s hard to come by in the wild. Somebody has to manually configure that in almost all situations."
Even so, what does Yarn do if a 410 is sent? I don't think it does anything at the moment, but I could be wrong.
@bender You could be right. Grepping the yarnd code for 410 and Gone did not reveal anything. Maybe, maybe it is handled by another library. But I kinda doubt it.
Yeah, the ErrDeadFeed is never actually checked anywhere. It's only set and that's it.