# I am the Watcher. I am your guide through this vast new twtiverse.
# 
# Usage:
#     https://watcher.sour.is/api/plain/users              View list of users and latest twt date.
#     https://watcher.sour.is/api/plain/twt                View all twts.
#     https://watcher.sour.is/api/plain/mentions?uri=:uri  View all mentions for uri.
#     https://watcher.sour.is/api/plain/conv/:hash         View all twts for a conversation subject.
# 
# Options:
#     uri     Filter to show a specific users twts.
#     offset  Start index for quey.
#     limit   Count of items to return (going back in time).
# 
# twt range = 1 196278
# self = https://watcher.sour.is?offset=172031
# next = https://watcher.sour.is?offset=172131
# prev = https://watcher.sour.is?offset=171931
@doesnm I don't even advocate for reading Twtxt in its raw form in the first place, which is why I'm in favor of continuing to use content-based addressing (hashes) and incremental improve what we already have. IMO the only reason to read a Twtxt file in it's raw form is a) if you're a developer b) new feed author or c) debugging a client issue.
Aggred. But reading twtxt in raw form sounds... I can't do this
And finally the legibility of feeds when viewing them in their raw form are worsened as you go from a Twt Subject of (#abcdefg12345) to something like (https://twtxt.net/user/prologic/twtxt.txt 2024-09-22T07:51:16Z).
And finally the legibility of feeds when viewing them in their raw form are worsened as you go from a Twt Subject of (#abcdefg12345) to something like (https://twtxt.net/user/prologic/twtxt.txt 2024-09-22T07:51:16Z).
There is also a ~5x increase cost in memory utilization for any implementations or implementors that use or wish to use in-memory storage (yarnd does for example) and equally a 5x increase in on-disk storage as well. This is based on the Twt Hash going from a 13 bytes (content-addressing) to 63 bytes (on average for location-based addressing). There is roughly a ~20-150% increase in the size of individual feeds as well that needs to be taken into consideration (_on the average case_).
There is also a ~5x increase cost in memory utilization for any implementations or implementors that use or wish to use in-memory storage (yarnd does for example) and equally a 5x increase in on-disk storage as well. This is based on the Twt Hash going from a 13 bytes (content-addressing) to 63 bytes (on average for location-based addressing). There is roughly a ~20-150% increase in the size of individual feeds as well that needs to be taken into consideration (_on the average case_).
With Location-based addressing there is no way to verify that a single Twt _actaully_ came from that feed without actually fetching the feed and checking. That has the effect of always having to rely on fetching the feed and storing a copy of feeds you fetch (_which is okay_), but you're force to do this. You cannot really share individual Twts anymore really like yarnd does (_as peering_) because there is no "integrity" to the Twt identified by it's <url> <timestamp>. The identify is meaningless and is only valid as long as you can trust the location and that the location at that point hasn't changed its content.
With Location-based addressing there is no way to verify that a single Twt _actaully_ came from that feed without actually fetching the feed and checking. That has the effect of always having to rely on fetching the feed and storing a copy of feeds you fetch (_which is okay_), but you're force to do this. You cannot really share individual Twts anymore really like yarnd does (_as peering_) because there is no "integrity" to the Twt identified by it's <url> <timestamp>. The identify is meaningless and is only valid as long as you can trust the location and that the location at that point hasn't changed its content.
Location-based addressing is vulnerable to the content changing. If the content changes the "location" is no longer valid. This is a problem if you build systems that rely on this.
Location-based addressing is vulnerable to the content changing. If the content changes the "location" is no longer valid. This is a problem if you build systems that rely on this.
So really your argument is just that switching to a location-based addressing "just makes sense". Why? Without concrete pros/cons of each approach this isn't really a strong argument I'm afraid. In fact I probably need to just sit down and detail the properties of both approaches and the pros/cons of both.

I also don't really buy the argument of simplicity either personally, because I don't technically see it much more difficult to take a echo -e "<url>\t<timestamp>\t<content>" | sha256sum | base64 as the Twt Subject or concatenating the <url> <timestamp> -- The "effort" is the same. If we're going to argue that SHA256 or cryptographic hashes are "too complicated" then I'm not really sure how to support that argument.
So really your argument is just that switching to a location-based addressing "just makes sense". Why? Without concrete pros/cons of each approach this isn't really a strong argument I'm afraid. In fact I probably need to just sit down and detail the properties of both approaches and the pros/cons of both.

I also don't really buy the argument of simplicity either personally, because I don't technically see it much more difficult to take a echo -e "<url>\t<timestamp>\t<content>" | sha256sum | base64 as the Twt Subject or concatenating the <url> <timestamp> -- The "effort" is the same. If we're going to argue that SHA256 or cryptographic hashes are "too complicated" then I'm not really sure how to support that argument.
So really your argument is just that switching to a location-based addressing "just makes sense". Why? Without concrete pros/cons of each approach this isn't really a strong argument I'm afraid. In fact I probably need to just sit down and detail the properties of both approaches and the pros/cons of both.

I also don't really buy the argument of simplicity either personally, because I don't technically see it much more difficult to take a echo -e "<url>\\t<timestamp>\\t<content>" | sha256sum | base64 as the Twt Subject or concatenating the <url> <timestamp> -- The "effort" is the same. If we're going to argue that SHA256 or cryptographic hashes are "too complicated" then I'm not really sure how to support that argument.
@sorenpeter Points 2 & 3 aren't really applicable here in the discussion of the threading model really I'm afraid. WebMentions is completely orthogonal to the discussion. Further, no-one that uses Twtxt really uses WebMentions, whilst yarnd supports the use of WebMentions, it's very rarely used in practise (_if ever_) -- In fact I should just drop the feature entirely.

The use of WebSub OTOH is far more useful and is used by every single yarnd pod everywhere (_no that there's that many around these days_) to subscribe to feed updates in ~near real-time _without_ having the poll constantly.~
@sorenpeter Points 2 & 3 aren't really applicable here in the discussion of the threading model really I'm afraid. WebMentions is completely orthogonal to the discussion. Further, no-one that uses Twtxt really uses WebMentions, whilst yarnd supports the use of WebMentions, it's very rarely used in practise (_if ever_) -- In fact I should just drop the feature entirely.

The use of WebSub OTOH is far more useful and is used by every single yarnd pod everywhere (_no that there's that many around these days_) to subscribe to feed updates in ~near real-time _without_ having the poll constantly.~
Some more arguments for a local-based treading model over a content-based one:

1. The format: (#<DATE URL>) or (@<DATE URL>) both makes sense: # as prefix is for a hashtag like we allredy got with the (#twthash) and @ as prefix denotes that this is mention of a specific post in a feed, and not just the feed in general. Using either can make implementation easier, since most clients already got this kind of filtering.

2. Having something like (#<DATE URL>) will also make mentions via webmetions for twtxt easier to implement, since there is no need for looking up the #twthash. This will also make it possible to make 3th part twt-mentions services.

3. Supporting twt/webmentions will also increase discoverability as a way to know about both replies and feed mentions from feeds that you don't follow.
Some more arguments for a local-based treading model over a content-based one:

1. The format: (#<DATE URL>) or (@<DATE URL>) both makes sense: # as prefix is for a hashtag like we allredy got with the (#twthash) and @ as prefix denotes that this is mention of a specific post in a feed, and not just the feed in general. Using either can make implementation easier, since most clients already got this kind of filtering.

2. Having something like (#<DATE URL>) will also make mentions via webmetions for twtxt easier to implement, since there is no need for looking up the #twthash. This will also make it possible to make 3th part twt-mentions services.

3. Supporting twt/webmentions will also increase discoverability as a way to know about both replies and feed mentions from feeds that you don't follow.
Some more arguments for a local-based treading model over a content-based one:

1. The format: (#<DATE URL>) or (@<DATE URL>) both makes sense: # as prefix is for a hashtag like we allredy got with the (#twthash) and @ as prefix denotes that this is mention of a specific post in a feed, and not just the feed in general. Using either can make implementation easier, since most clients already got this kind of filtering.

2. Having something like (#<DATE URL>) will also make mentions via webmetions for twtxt easier to implement, since there is no need for looking up the #twthash. This will also make it possible to make 3th part twt-mentions services.

3. Supporting twt/webmentions will also increase discoverability as a way to know about both replies and feed mentions from feeds that you don't follow.
Some more arguments for a local-based treading model over a content-based one:

1. The format: (#<DATE URL>) or (@<DATE URL>) both makes sense: # as prefix is for a hashtag like we allredy got with the (#twthash) and @ as prefix denotes that this is mention of a specific post in a feed, and not just the feed in general. Using either can make implementation easier, since most clients already got this kind of filtering.

2. Having something like (#<DATE URL>) will also make mentions via webmetions for twtxt easier to implement, since there is no need for looking up the #twthash. This will also make it possible to make 3th part twt-mentions services.

3. Supporting twt/webmentions will also increase discoverability as a way to know about both replies and feed mentions from feeds that you don't follow.
@doesnm Welcome back 😅
@doesnm Welcome back 😅
Finally pubnix is alive! That's im missing? Im only reading twtxt.net timeline because twtxt-v2.sh works slowly for displaying timeline...
[47°09′52″S, 126°43′28″W] Bad satellite signal -- switching to analog communication
Pinellas County Running: 4.06 miles, 00:09:11 average pace, 00:37:21 duration

#running
Pinellas County Running: 4.06 miles, 00:09:11 average pace, 00:37:21 duration

#running
Pinellas County Running: 4.06 miles, 00:09:11 average pace, 00:37:21 duration

#running
[47°09′03″S, 126°43′42″W] Storm recedes -- back to normal work
[47°09′47″S, 126°43′53″W] Wind speed: 42kph
🧮 USERS:1 FEEDS:2 TWTS:1102 ARCHIVED:79309 CACHE:2611 FOLLOWERS:17 FOLLOWING:14
Been trying to get acquainted with rsync(1) but, whenever I Tab for completion and get this:

> λ ~/ rsync --
> zsh: do you wish to see all 484 possibilities (162 lines)?

I'm like: Nope! a scp -rpCq ... or whatever option salad will do just fine. 😅 \n~
Been trying to get acquainted with rsync(1) but, whenever I Tab for completion and get this:

> λ ~/ rsync --
> zsh: do you wish to see all 484 possibilities (162 lines)?

I'm like: Nope! a scp -rpCq ... or whatever option salad will do just fine. 😅 [Insert: "Ain't nobody got time fo'that!" Meme.]~
Been trying to get acquainted with rsync(1) but, whenever I Tab for completion and get this:

> λ ~/ rsync --
> zsh: do you wish to see all 484 possibilities (162 lines)?

I'm like: Nope! a scp -rpCq ... or whatever option salad will do just fine. 😅 [Insert: "Ain't nobody got time fo'that!" Meme.]~
experimenting with litefs has been really interesting. i'm still learning about consul, so nothing distributed is happening yet. so far i have a setup that shares a virtual filesystem with a set of nixos containers running ejabberd and redka. soon some ory services for auth and security which also support sqlite will join the party, but those require higher availability that i can manage with my current deployment. the big server needs to me migrated before security can come online.
Syncthing is also as good as everyone says it is.
@movq Interesting, it's always good to know how things work under the hood. But I'm very glad, that I do not have to deal with this low-level stuff. :-)
[47°09′34″S, 126°43′02″W] Wind speed: N/A -- Cannot comunicate
@prologic violent enough to be taken away by the police. 🤭😂
@prologic @movq Luckily, we were only touched by the thunderstorm cell. Even though the sky lit up a bunch and the thunder roared, there were no close thunderbolts. But it rained cats and dogs. The air smelled lovely.
@eapl.me All the best, see you next life around. :-) On Twtxt I only meet my online friends. I'm staying in touch with some of my real life mates on IRC or e-mail. But that's fine. That's just how it goes.

Thanks, @bender. :-)
@aelaraji Hahaha, brilliant! :-D
I know what keeps me coming back to twtxt. It is the little group of people with whom I interact. I don’t need a big audience. More often than not I have nothing interesting to write, but I enjoy the small interactions: bugging prologic, reading abucci, browsing Lyse’s clicks. I enjoy movq commentaries (I imagine him as a professor of some kind, don’t ask me why).

Anyway… cheers!
#catsoftwtxt
/https://baldo.cat/media/photos/IMG_2115.jpeg) #catsoftwtxt
#catsoftwtxt
#catsoftwtxt
/https://baldo.cat/media/photos/IMG_2113.jpeg) #catsoftwtxt
#catsoftwtxt
@eapl.me are you sure X will bring joy, and value? Will you have clear conscience knowing you are contributing to such despicable platform? It is your decision to make, sure.

Joy starts at you, not the platform you use. When you get bored, disgusted, offended, and leave X, come and let us know. I will be interested to read all about your experiment then. For now, “¡hasta pronto!”
[47°09′01″S, 126°43′17″W] Weather forecast alert -- storm from NE
@movq Yes, the tools are surprisingly fast. Still, magrep takes about 20 seconds to search through my archive of 140K emails, so to speed things up I would probably combine it with an indexer like mu, mairix or notmuch.
@eapl.me Aww. Well, I gave you a Follow on Mastodon (although that appears to be moastly Spanish 🤔).
@eapl.me Aww. Well, I gave you a Follow on Mastodon (although that appears to be moastly Spanish 🤔).
@eapl.me Aww. Well, I gave you a Follow on Mastodon (although that appears to be moastly Spanish 🤔).
@eapl.me Aww. Well, I gave you a Follow on Mastodon (although that appears to be moastly Spanish 🤔).
@falsifian You had me there for a second. 😅

I have to admit, even though I knew they existed, I never had a look at Leah’s mail tools. Just gave mthread a spin and this is crazy fast. 🤯 Tempting!
@falsifian You had me there for a second. 😅

I have to admit, even though I knew they existed, I never had a look at Leah’s mail tools. Just gave mthread a spin and this is crazy fast. 🤯 Tempting!
@falsifian You had me there for a second. 😅

I have to admit, even though I knew they existed, I never had a look at Leah’s mail tools. Just gave mthread a spin and this is crazy fast. 🤯 Tempting!
@falsifian You had me there for a second. 😅

I have to admit, even though I knew they existed, I never had a look at Leah’s mail tools. Just gave mthread a spin and this is crazy fast. 🤯 Tempting!
@eapl.me Sad to see you go, disappointed in your choice of X, but respect your decision and choice. I will never cave in myself, even if it means my "circle of friends" remains low. I guess we call 'em internet friends right? 😅
@eapl.me Sad to see you go, disappointed in your choice of X, but respect your decision and choice. I will never cave in myself, even if it means my "circle of friends" remains low. I guess we call 'em internet friends right? 😅
#fzf is the new emacs: a tool with a simple purpose that has evolved to include an #email client. https://sr.ht/~rakoo/omail/

I'm being a little silly, of course. fzf doesn't actually check your email, but it appears to be basically the whole user interface for that mail program, with #mblaze wrangling the emails.

I've been thinking about how I handle my email, and am tempted to make something similar. (When I originally saw this linked the author was presenting it as an example tweaked to their own needs, encouraging people to make their own.)

This approach could surely also be combined with #jenny, taking the place of (neo)mutt. For example mblaze's mthread tool presents a threaded discussion with indentation.
@lyse Gut festhalten!
@lyse Gut festhalten!
@lyse Gut festhalten!
@lyse Gut festhalten!
Someone recommended a nice (German) talk:

https://media.ccc.de/v/ds24-394-linux-hello-world-nur-mit-einem-hex-editor

Luckily, everything™ is easier™ on DOS with .COM files. A fun little time killer to make a HELLO.COM using only a hex editor, the Intel docs and the DOS interrupt list.

That ModR/M stuff is easy in the end, but it took me quite some time to understand it. 🥴

(I’m still new to DOS on this level and didn’t know that all segment registers are initialized to the same values, apparently, so copying CS to DS was not necessary. Too lazy to update the screenshot. File size shrinks by 4 bytes.)

https://movq.de/v/0139fbaabc/doshello.png
Someone recommended a nice (German) talk:

https://media.ccc.de/v/ds24-394-linux-hello-world-nur-mit-einem-hex-editor

Luckily, everything™ is easier™ on DOS with .COM files. A fun little time killer to make a HELLO.COM using only a hex editor, the Intel docs and the DOS interrupt list.

That ModR/M stuff is easy in the end, but it took me quite some time to understand it. 🥴

(I’m still new to DOS on this level and didn’t know that all segment registers are initialized to the same values, apparently, so copying CS to DS was not necessary. Too lazy to update the screenshot. File size shrinks by 4 bytes.)

https://movq.de/v/0139fbaabc/doshello.png
Someone recommended a nice (German) talk:

https://media.ccc.de/v/ds24-394-linux-hello-world-nur-mit-einem-hex-editor

Luckily, everything™ is easier™ on DOS with .COM files. A fun little time killer to make a HELLO.COM using only a hex editor, the Intel docs and the DOS interrupt list.

That ModR/M stuff is easy in the end, but it took me quite some time to understand it. 🥴

(I’m still new to DOS on this level and didn’t know that all segment registers are initialized to the same values, apparently, so copying CS to DS was not necessary. Too lazy to update the screenshot. File size shrinks by 4 bytes.)

https://movq.de/v/0139fbaabc/doshello.png
Someone recommended a nice (German) talk:

https://media.ccc.de/v/ds24-394-linux-hello-world-nur-mit-einem-hex-editor

Luckily, everything™ is easier™ on DOS with .COM files. A fun little time killer to make a HELLO.COM using only a hex editor, the Intel docs and the DOS interrupt list.

That ModR/M stuff is easy in the end, but it took me quite some time to understand it. 🥴

(I’m still new to DOS on this level and didn’t know that all segment registers are initialized to the same values, apparently, so copying CS to DS was not necessary. Too lazy to update the screenshot. File size shrinks by 4 bytes.)

https://movq.de/v/0139fbaabc/doshello.png
@lyse How violent is the thunderstorm? 🤔
@lyse How violent is the thunderstorm? 🤔
We're now having a thunderstorm with rain, lightning and thunder and the severe weather map shows all green. I'd expect it to be violet.
@aelaraji LOl 😂
@aelaraji LOl 😂
Okay, I figured out the cause of the broken output. I also replaced the first subject = '' for the existing conversation roots with subject > ''. Somehow, my brain must have read subject <> ''. That equality check should not have been touched at all. I just updated the updated archive for anyone who is interested to follow along: https://lyse.isobeef.org/tmp/tt2cache.tar.bz2 (151.1 KiB)
LMAO 🤣 ... I've been scrolling through mutt(1) man page and found this:

> BUGS
> None. Mutts have fleas, not bugs.
LMAO 🤣 ... I've been scrolling through mutt(1) man page and found this:

> BUGS
> None. Mutts have fleas, not bugs.
LMAO 🤣 ... I've been scrolling through mutt(1) man page and found this:

> BUGS
> None. Mutts have fleas, not bugs.
A new thing LLM(s) can't do well. Write patches 🤣
A new thing LLM(s) can't do well. Write patches 🤣
@lyse Yeah I _think_ it's one of the reasons why yarnd's cache became so complicated really. I mean it's a bunch of maps and lists that is recalculated every ~5m. I don't know of any better way to do this right now, but maybe one day I'll figure out a better way to represent the same information that is displayed today that works reasonably well.~
@lyse Yeah I _think_ it's one of the reasons why yarnd's cache became so complicated really. I mean it's a bunch of maps and lists that is recalculated every ~5m. I don't know of any better way to do this right now, but maybe one day I'll figure out a better way to represent the same information that is displayed today that works reasonably well.~
@prologic Yeah, relational databases are definitely not the perfect fit for trees, but I want to give it a shot anyway. :-)

Using EXPLAIN QUERY PLAN I was able to create two indices, to avoid some table scans:

CREATE INDEX parent ON messages (hash, subject);
CREATE INDEX subject_created_at ON messages (subject, created_at);

Also, since strings are sortable, instead of str_col <> '' I now use str_col > '' to allow the use of an index.

But somehow, my output seems to be broken at the end for some reason, I just noticed. :-? Hmm.

The read status still gives me headache. I think I either have to filter in the application or create more meta data structures in the database.

I'm wondering if anyone here already used certain storages for tree data.
My point is, this is not a small trade-off to make for the sake of simplicity 😅
My point is, this is not a small trade-off to make for the sake of simplicity 😅
@movq Maybe I misspoke. It's a factor of 5 in the size of the keyspace required. The impact is significantly less for on-disk storage of raw feeds and such, around ~1-1.5x depending on how many replies there are I suppose.

I wasn't very clear; my apologies. If we update the current hash truncation length from 7 to 11. But then still decide anyway to go down this location-based twt identity and threading model then yes, we're talking about twt subjects having a ~5x increase in size on average. Going from 14 characters (11 for the has, 2 for the parens, 1 for the #) to ~63 bytes (average I've worked out of length of URL + Timestamp) + 3 byte overhead for parents and space.~
@movq Maybe I misspoke. It's a factor of 5 in the size of the keyspace required. The impact is significantly less for on-disk storage of raw feeds and such, around ~1-1.5x depending on how many replies there are I suppose.

I wasn't very clear; my apologies. If we update the current hash truncation length from 7 to 11. But then still decide anyway to go down this location-based twt identity and threading model then yes, we're talking about twt subjects having a ~5x increase in size on average. Going from 14 characters (11 for the has, 2 for the parens, 1 for the #) to ~63 bytes (average I've worked out of length of URL + Timestamp) + 3 byte overhead for parents and space.~
@prologic A factor of 5 is hard to believe, to be honest. Especially disk usage. I know nothing about the internals of yarnd, but still.

If this constitutes a hard “no” to the proposal, then I think we don’t need to discuss it further.
@prologic A factor of 5 is hard to believe, to be honest. Especially disk usage. I know nothing about the internals of yarnd, but still.

If this constitutes a hard “no” to the proposal, then I think we don’t need to discuss it further.
@prologic A factor of 5 is hard to believe, to be honest. Especially disk usage. I know nothing about the internals of yarnd, but still.

If this constitutes a hard “no” to the proposal, then I think we don’t need to discuss it further.
@prologic A factor of 5 is hard to believe, to be honest. Especially disk usage. I know nothing about the internals of yarnd, but still.

If this constitutes a hard “no” to the proposal, then I think we don’t need to discuss it further.
@lyse Yes I think so.
@lyse Yes I think so.
@prologic I see. I reckon, it makes to combine 1 and 2, because if we change the hashing anyway, we don't break it twice.
Don't forget about the upcoming Yarn.social meetup coming up this Saturday! See #jjbnvgq for details! Hope to see some/all of y'all there 💪
Don't forget about the upcoming Yarn.social meetup coming up this Saturday! See # for details! Hope to see some/all of y'all there 💪
Don't forget about the upcoming Yarn.social meetup coming up this Saturday! See #jjbnvgq for details! Hope to see some/all of y'all there 💪
@lyse And your query to construct a tree? Can you share the full query (_screenshot looks scary 🤣_) -- On another note, SQL and relational databases aren't really that conduces to tree-like structures are they? 🤣_