# I am the Watcher. I am your guide through this vast new twtiverse.
#
# Usage:
# https://watcher.sour.is/api/plain/users View list of users and latest twt date.
# https://watcher.sour.is/api/plain/twt View all twts.
# https://watcher.sour.is/api/plain/mentions?uri=:uri View all mentions for uri.
# https://watcher.sour.is/api/plain/conv/:hash View all twts for a conversation subject.
#
# Options:
# uri Filter to show a specific users twts.
# offset Start index for quey.
# limit Count of items to return (going back in time).
#
# twt range = 1 70
# self = https://watcher.sour.is?uri=https://twtxt.net/user/zvava/twtxt.txt&offset=70
@bender@alexonit funnily enough i came up with the idea for a banner field when i started working on bbycll too (though i still need to implement avatar uploads themselves first)
@itsericwoodward@bender this is vaguely concerning...does yarn refresh feeds every minute or two? or is there some special "notify twtxt.net to refresh my feed" that i don't know about
@lyse i would like to ditch hash addressing but as was pointed out it would be a pain in the ass to get clients currently working off of hashv1 to suddenly switch to location-based addressing instead of just hashv2 with the option to eventually phase it out — unless we can rally together all active client developers to decide on a location-based addressing specification (i still think my original suggestion of #<https://example.com/tw.txt#yyyy-mm-ddThh:mm:ssZ> is foolproof)
@bender technically it's still the same, but the brackets are different, and the # symbol is on the outside of the brackets, but it makes more sense with @<...> being mentions
@movq huh, firefox actually does seem to tolerate the dashes in the fragment. also, i did propose simply using an anchor link first, but prologic was not a fan of this :p
@alexonit prologic has me sold on the idea of hashv2 being served alongside a text fragment, eg. (#abcdefghijkl https://example.com/tw.txt#:~:text=2025-10-01T10:28:00Z), because it can be simply hacked in to clients currently on hashv1 and provides an off-ramp to location-based addressing (though i still think the format should be changed to smth like #<abc... http://example.com/...> so it's cleaner once we finally drop hashes)
is the first url metadata field unequivocally treated as the canon feed url when calculating hashes, or are they ignored if they're not *at least* proper urls? do you just tolerate it if they're impersonating someone else's feed, or pointing to something that isn't even a feed at all?
and if the first url metadata field changes, should it be logged with a time so we can still calculate hashes for old posts? or should it never be updated? (in the case of a pod, where the end user has no choice in how such events are treated) or do we redirect all the old hashes to the new ones (probably this, since it would be helpful for edits too)
@alexonit yeah, i didn't even consider this as an option lol, though if it works it works! will you be writing a compatible proxy for self-hosting separate from the custom backend you were thinking of? :o
@alexonit i tried making a webapp initially but i didn't even get into the initial stages of testing because no one sets the Access-Control-Allow-Origin header, so i just jumped into building a backend instead. did you find away around this limitation? :o
plus, if hashv2 was implemented in combination with text fragments the way you proposed that would solve both scripting and human readability woes!!
...though, the presence of the text fragments then makes reversing the replied-to twt (and therefore its hash) trivial, which could allow clients to tolerate the omission of the hash — and while it would be 'non-standard' this would be the best of both worlds; potential to *tolerate* (or pave a glacial path toward? :o) human writable twts whilst keeping a unique id for twts that is universal across all pods
plus, if hashv2 was implemented in combination with text fragments the way you proposed that would solve both scripting and human readability woes!!
...though, the presence of the text fragments then makes reversing the replied-to twt (and therefore its hash) trivial, which could allow clients to tolerate the omission of the hash — and while it would be 'non-standard' this would be the best of both worlds; potential to *tolerate* (or pave a glacial path toward? :o) human writable replies whilst keeping a unique id for twts that is universal across all pods
@prologic to clarify the i meant the ability to parse feeds using unix command line utilities, as a prinicpal of twtxtv1's design. im not sure how feasible it is to build a simple feed reader out of common scripting utilities when hashing is in play, and;
i concede, it does make a lot of sense to fix up the hashing spec rather than completely supplant it at this point, just thinking about what the rewrite would be like is dreadful in and of itself x.x
@prologic to clarify: i meant the ability to parse feeds using unix command line utilities, as a principal of twtxtv1's design. im not sure how feasible it is to build a simple feed reader out of common scripting utilities when hashing is in play, and;
i concede, it does make a lot of sense to fix up the hashing spec rather than completely supplant it at this point, just thinking about what the rewrite would be like is dreadful in and of itself x.x
@prologic the simplest thing to do is to completely forgo hashing anything because we are communicating using plain text files right now :3 while i agree hashes are incredibly helpful in the backend im not sure it has a place outside of it, it basically eliminates two core design principals of twtxt (human readability and integrating well with unix command line utilities) and makes new clients more difficult to build than it should be
@alexonit@lyse i really don't understand why this was not the solution in the first place, given how simple twtxt is (mean to be), a reply should be as simple as #<https://example.com/twtxt.txt#2025-09-22T06:45Z> with the timestamp in an anchor link. the need for a mention is avoided like this as well since it's already linking to the replied-to feed!
🐀💭 i should just implement it into bbycll and force it into existence
@lyse@movq bbycll's nickname regex is /^([-_\p{N}\p{L}])+$/iu because i don't like how english-centric only allowing ascii letters/numbers is though this only applies to local users as of now, currently all nicknames are tolerated when parsing remote feeds and i just do mentions how yarn does (just the feed url)
in the wild, i've noticed a texedus feed with spaces in the nick (where its spec explicitly disallows whitespace in the nick) and feeds with other symbols in the nick too. honestly, i think we should just tolerate arbitrary nicknames for sake of user expression (while stripping or converting unreasonable characters) and just leave them out of mentions
is there consensus on what characters should(n't) be allowed in nicks? i remember reading somewhere whitespace should not be allowed, but i don't see it in the spec on twtxt.dev — in fact, are there any other resources on twtxt extensions outside of twtxt.dev?
@lyse i dont mind if the hash is not backward compatible but im not sure if this is the right way to proceed because the added complexity dealing with two hash versions isnt justified
regular end users wont care to understand how twt hashes are formed, they just want to use twtxt! so i guess i could work in protecting users from themselves by disallowing post edits on old posts or posts with replies, but i'm not fond of this either really. if they want to break a thread, they can just delete the post (though i've noticed yarn handling post deletes dubiously...)
on activitypub i do genuinely find myself looking through several month or even year old posts sometimes and deciding to edit/reword them a little to be slightly less confusing, this should be trivial to handle on twtxt which is an infinitely simpler specification
@bender@thecanine well now this has me thinking abt the feasibility of making an android twtxt app for pods, the actual apis of pods would have to be standardized (or the fucked up way that activitypub does it, where the "mastodon api" is the defacto client api (does yarn even have an api reference?)) or the client is just simply..a client..but editing feeds via PUT, PATCH, DELETE etc. is standardized!...? (not to mention i dont even know where to begin making an android app lmao)
@bender@thecanine well now this has me thinking abt the feasibility of making an android twtxt app for pods, the actual apis of pods would have to be standardized (or a fucked up version of how activitypub does it, where the "mastodon api" is the defacto client api (does yarn even have an api reference?)) or the client is just simply..a client..but editing feeds via PUT, PATCH, DELETE etc. is standardized!...? (not to mention i dont even know where to begin making an android app lmao)
im unable to figure out why bbycll is not generating posts hashes for @lyse's feed correctly (or at least different from the ones generated by yarn)
i'm pretty sure the timezone is stripped off the offset correctly (2025-09-14T12:45:00+02:00 → 2025-09-14T12:45:00Z) though messing with how the hash is generated i can't get it to make one that matches...but all other hashes for all other feeds seem to be correct? does yarn use a different canonical url for lyse internally? is there a bug in the libraries im using? bwehhh
@prologic im unsure how i feel about the hash v2 proposal, given it is completely backward incompatible with hash v1 it doesn't really solve any of the problems with it. it only delays collisions, and still fragments threads on post edits
i skimmed through discussions under other the proposals — i agree humans are very bad at keeping the integrity of the web in tact, but hashes in done in this way make it impossible even for systems to rebuild threads if any post edits have occurred prior to their deployment
wait....so i'm like nearly done? it just _works_? and it's _fast_? this feels like the end of the first all-nighter i pulled where i just got post creation done, unaware of the three weeks that would follow — like looking at the roadmap i'm definitely not _done_ but bbycll is like actually kind of usable now o.o
ok so i have found a genuine twt hash collision. what do i do.
internally, bbycll relies on a post lookup table with post hashes as keys, this is really fast but i knew i'd inevitably run into this issue (just not so soon) so now i have to either: 1) pick the newer post over the other 2) break from specification and not lowercase hashes 3) secretly associate canonical urls or additional entropy with post hashes in the backend without a sizeable performance impact somehow
we are now parsing and recursively fetching remote feeds somewhat successfully, gotta work on the media proxy and markdown way more, so so many fucky edgecases....my friend's feed with like four posts parsed correctly so i tried this account's feed and well now im not going to bed on time
we are now parsing and recursively fetching remote feeds somewhat successfully, gotta work on the media proxy and markdown way more, so so many fucky edgecases....my friend's feed with like four posts parsed correctly so i tried this account's feed and well now im not going to bed on time
@bender ty for attempting to test, though, it means a lot! lmk if u find any more silly things i need to clarify or fix :> be prepared for everything breaking during beta :p
@bender i also recently discovered there was a bug causing new users to initialize wrong leading to their posts not being saved :p ..and made breaking changes to how the config and database files are stored so um, make sure to clear your local tree before updating!
@bender as the host (eg twtxt.net) determines the canonical url of the instance in generated feed url metadata as well as every hash of every post made on the instance internally, i added this error message to make sure people don't accidentally set up their instance on localhost :p for testing i set it to localhost:31212 and protocols to ["http"], it's a recent addition that could definitely do with documenting in the getting started section
@lyse retwts are a discovery feature! on federated platforms with no algorithm where you only ever see posts from accounts you explicitly follow, the element of "hey look at this!" helps users to find other accounts they might like organically
i agree quoting and replying forum-style is generally a much better way of doing things even though im a heathen and i revel in the dark patterns inspired by quote posts but when you have nothing to add and you just want to share a twt with your followers it'd be good to have a standardized way of linking to twt
@bender@kat after implementing cws in bbycll (parsing them the way i described in op) i understand why <details> aren't rendered. firstly they're not actually markdown, and secondly they style weird — details are forced onto a new line as a block element but you can't make the <summary> inline because it is inside <details>, and making <details> inline will also indent everything inside it
in the same vein, i think content warnings can be faithfully implemented by parsing CW: ..., tw // ..., etc. from the first line of a post followed by two newlines, like how they're used on platforms that don't have content warnings
at first i dismissed the idea of likes on twtxt as not sensible...like at all — then i considered they could just be published in a metadata field (though that field could get really unruly after a while)
retwts are plausible, as "RE: https://example.com/twtxt.txt#abcdefg", the hash could even be the original timestamp from the feed to make it human readable/writable, though im extremely wary of clogging up timelines
i _thought_ quote twts could be done extremely sensibly, by interpreting a mention+hash at the end of the twt differently to when placed at the beginning — but the twt subject extension requires it be at the beginning, so the clean fallback to a normal reply i originally imagined is out of the question — it could still be possible (reusing the retwt format, just like twitter!) but i'm not convinced it's worth it at that point
is any of this in the spirit of twtxt? no, not in the slightest, lmao
search page, bookmarks page, improved thread view (that i will probably improve further), as well as a logo and a whole ui redesign. it is truly all coming together...were i to mark any items off the roadmap :p
is there someone (ideally not in the opposite timezone to me) who'd be willing to let me bother them with technical questions abt twtxtv2 and/or yarn's inner workings? :3
beginnings of remote feed parsing..! the fact hashing just sort of works with the minuscule libraries i found for base32 and blake2b still amazes me (mentions are being eaten as html tags)
@lyse hihi ^^ i did that at first, but i personally i don't like it when websites don't let me change my password when i am already authenticated — fwiw you can view and log out other sessions, if that diminishes this attack vector at all
replies and following implemented! next step is further parsing of post contents, rendering threads, and then maybe i can finally start adding remote feeds...! though i kinda wanna redo the whole ui ^^'
demo showing two users following and unfollowing eachother, showing up in eachothers timelines, and replying to one another
@prologic haven't had too much time to really try it out yet ^^' i'm um too busy staring at code i wrote while sleep deprived and wondering why i did the things i did, while sleep deprived \@.@
