# I am the Watcher. I am your guide through this vast new twtiverse.
# 
# Usage:
#     https://watcher.sour.is/api/plain/users              View list of users and latest twt date.
#     https://watcher.sour.is/api/plain/twt                View all twts.
#     https://watcher.sour.is/api/plain/mentions?uri=:uri  View all mentions for uri.
#     https://watcher.sour.is/api/plain/conv/:hash         View all twts for a conversation subject.
# 
# Options:
#     uri     Filter to show a specific users twts.
#     offset  Start index for quey.
#     limit   Count of items to return (going back in time).
# 
# twt range = 1 196266
# self = https://watcher.sour.is?offset=171556
# next = https://watcher.sour.is?offset=171656
# prev = https://watcher.sour.is?offset=171456
@prologic This does not hold if the edit happened before I even got the original.
If OTOH your client doesn't store individual Twts in a cache/archive or some kind of database, then verification becomes quite hard and tedious. However I think of this as an implementation details. The spec should just call out that clients must validate/verify the edit request and the matching hash actually exists in that feed, not how the client should implement that.
If OTOH your client doesn't store individual Twts in a cache/archive or some kind of database, then verification becomes quite hard and tedious. However I think of this as an implementation details. The spec should just call out that clients must validate/verify the edit request and the matching hash actually exists in that feed, not how the client should implement that.
@lyse Yes you do. You keep both versions in your cache. They have different hashes. So you have Twt A, a client indicates Twt B is an edit of A, your client has already seen A and cached and archived it, now your client fetches B which is indicated of editing A. You cache/archive B as well, but now indicate in your display that B replaces A (_maybe display, link both_) or just display B or whatever. But essentially you now have both, but an indicator of one being an edit of the other.

The right thing to do here of course is to keep A in the "thread" but display B. Why? So the thread/chain doesn't actually break or fork (_forking is a natural consequence of editing, or is it the other way around? 🤔_)._
@lyse Yes you do. You keep both versions in your cache. They have different hashes. So you have Twt A, a client indicates Twt B is an edit of A, your client has already seen A and cached and archived it, now your client fetches B which is indicated of editing A. You cache/archive B as well, but now indicate in your display that B replaces A (_maybe display, link both_) or just display B or whatever. But essentially you now have both, but an indicator of one being an edit of the other.

The right thing to do here of course is to keep A in the "thread" but display B. Why? So the thread/chain doesn't actually break or fork (_forking is a natural consequence of editing, or is it the other way around? 🤔_)._
@lyse I'm all for dropping delete btw, Or at least not making it mandatory, as-in "clients should" rather than "clients must". But yes I agree, let's explore all the possible ways this can be exploited (_if at all_).
@lyse I'm all for dropping delete btw, Or at least not making it mandatory, as-in "clients should" rather than "clients must". But yes I agree, let's explore all the possible ways this can be exploited (_if at all_).
@movq I think not.

> What about edits of edits? Do we want to “chain” edits or does the latest edit simply win?

This gets too complicated if we start to support this kind of nonsense 🤣
@movq I think not.

> What about edits of edits? Do we want to “chain” edits or does the latest edit simply win?

This gets too complicated if we start to support this kind of nonsense 🤣
@movq Thank you! 🙏
@movq Thank you! 🙏
@lyse Walk me through this? 🤔 I get what you're saying, but I'm too stupid to be a "hacker" 🤣
@lyse Walk me through this? 🤔 I get what you're saying, but I'm too stupid to be a "hacker" 🤣
flakes arent real https://jade.fyi/blog/flakes-arent-real/
But yes, at the end of the day if the edit request is invalid or cannot be verified, it should be ignored as treated as "malicious".
But yes, at the end of the day if the edit request is invalid or cannot be verified, it should be ignored as treated as "malicious".
@lyse @movq So a client that has the idea of a cache/archive wouldn't necessarily have to re-check that the Twt being marked as "edited" belongs to that feed or not, the client would already know that for sure. At least this is how yarnd works and I'm sure jenny can make similar assertions too.
@lyse @movq So a client that has the idea of a cache/archive wouldn't necessarily have to re-check that the Twt being marked as "edited" belongs to that feed or not, the client would already know that for sure. At least this is how yarnd works and I'm sure jenny can make similar assertions too.
@lyse @falsifian Contributions to search.twtxt.net, which runs yarns (_not to be confused with yarnd_) are always welcome 🤗 -- I don't have as much "spare time" as I used to due to the nature of my job (_Staff Engineer_); but I try to make improvements every now and again 💪
@lyse @falsifian Contributions to search.twtxt.net, which runs yarns (_not to be confused with yarnd_) are always welcome 🤗 -- I don't have as much "spare time" as I used to due to the nature of my job (_Staff Engineer_); but I try to make improvements every now and again 💪
@falsifian You make good points though, I made similar arguments about this too back in the day. Twtxt v2 / Yarn.social being at least ~4 years old now 😅~
@falsifian You make good points though, I made similar arguments about this too back in the day. Twtxt v2 / Yarn.social being at least ~4 years old now 😅~
@falsifian Do you have specifics about the GRPD law about this?

> Would the GDPR would apply to a one-person client like jenny? I seriously hope not. If someone asks me to delete an email they sent me, I don’t think I have to honour that request, no matter how European they are.

I'm not sure myself now. So let's find out whether parts of the GDPR actually apply to a truly decentralised system? 🤔
@falsifian Do you have specifics about the GRPD law about this?

> Would the GDPR would apply to a one-person client like jenny? I seriously hope not. If someone asks me to delete an email they sent me, I don’t think I have to honour that request, no matter how European they are.

I'm not sure myself now. So let's find out whether parts of the GDPR actually apply to a truly decentralised system? 🤔
LOL 😂 This:

> anyone could claim that some feed contained a certain message which was then removed again by just creating the hash over the fake message in said feed and invented timestamp themselves

I'd like to see a step-by-step reproduction of this. I don't buy it 🤣

Admittedly yarnd had a few implementation security bugs, but I'm not sure this is actually possible, unless I'm missing something? 🤔
LOL 😂 This:

> anyone could claim that some feed contained a certain message which was then removed again by just creating the hash over the fake message in said feed and invented timestamp themselves

I'd like to see a step-by-step reproduction of this. I don't buy it 🤣

Admittedly yarnd had a few implementation security bugs, but I'm not sure this is actually possible, unless I'm missing something? 🤔
@david Very nice! 👍
@david Very nice! 👍
And they have arrived (well, they did around 3 hours ago, LOL). Buttery smooth, my 16 Pro (one with dark cover). It took a bit over an hour to transfer all my data.

iPhones 16, and 16 Pro
And they have arrived (well, they did around 3 hours ago, LOL). Buttery smooth, my 16 Pro (one with dark cover). It took a bit over an hour to transfer all my data.

iPhones 16, and 16 Pro
[47°09′46″S, 126°43′09″W] Not enough data -- sampling finished
Ah, and now he is "conveniently" sleeping. How, well, convenient! LOL.
Ah, and now he is "conveniently" sleeping. How, well, convenient! LOL.
@lyse yeah, tell us, @prologic, what isn't true? 🤔 You can't just go around, "that's not true, and that's not true; and that, and that!" without spelling out exactly what isn't, and why? For the love of god, why?! 😂
@lyse yeah, tell us, @prologic, what isn't true? 🤔 You can't just go around, "that's not true, and that's not true; and that, and that!" without spelling out exactly what isn't, and why? For the love of god, why?! 😂
@falsifian Something similar exists over at https://search.twtxt.net/. But a usable search engine would be actually nice (to be fair, yarns improved a bit). :-) I don't care about feed changes over time. In fact, it would even feel creepy to me. Of course, anyone could still surveil, but I'm not looking forward to these stats.
@movq We could still let the client display a warning if it cannot verify it. But yeah.
@falsifian comments on the feeds as in nick, url, follow, that kind of thing? If that, then not interested at all. I envision an archive that would allow searching, and potentially browsing threads on a nice, neat interface. You will have to think, though, on other things. Like, what to do with images? Yarn allows users to upload images, but also embed it in twtxts from other sources (hotlinking, actually).
@falsifian comments on the feeds as in nick, url, follow, that kind of thing? If that, then not interested at all. I envision an archive that would allow searching, and potentially browsing threads on a nice, neat interface. You will have to think, though, on other things. Like, what to do with images? Yarn allows users to upload images, but also embed it in twtxts from other sources (hotlinking, actually).
@david Thanks, that's good feedback to have. I wonder to what extent this already exists in registry servers and yarn pods. I haven't really tried digging into the past in either one.

How interested would you be in changes in metadata and other comments in the feeds? I'm thinking of just permanently saving every version of each twtxt file that gets pulled, not just the twts. It wouldn't be hard to do (though presenting the information in a sensible way is another matter). Compression should make storage a non-issue unless someone does something weird with their feed like shuffle the comments around every time I fetch it.
I ended up installing Headscale on my little VPS. Just in case the collide, I turned off WireGuard. Turning that one off (which ran on a container) also frees some memory. Headscale is running quite well! Indeed, I have struggled getting any web management console to work, but it really isn't needed. Everything needed to commandeer the server is available through the CLI.
I ended up installing Headscale on my little VPS. Just in case the collide, I turned off WireGuard. Turning that one off (which ran on a container) also frees some memory. Headscale is running quite well! Indeed, I have struggled getting any web management console to work, but it really isn't needed. Everything needed to commandeer the server is available through the CLI.
@movq Reminds me of this beautiful face recognition failure: https://qz.com/823820/carnegie-mellon-made-a-special-pair-of-glasses-that-lets-you-steal-a-digital-identity :-D
@falsifian "*I was actually thinking about making an Internet Archive style twtxt archiver, letting you explore past twts*" --- that's an awesome idea for a project. Something I would certainly use!
@falsifian "*I was actually thinking about making an Internet Archive style twtxt archiver, letting you explore past twts*" --- that's an awesome idea for a project. Something I would certainly use!
@prologic What exactly?
@prologic Just what @bender did. :-D If he'd additionally serve the fake message from his yarnd twt endpoint, everybody querying that hash from him (or any other yarnd that synced it in the meantime) would believe, that I didn't like Australians.

In fact, I really don't. I love'em! 8-)

We would need to sign each message in a feed, so others could verify that this was actually part of that feed and not made up. But then we end up in the crypto debate for identities again, which I'm not a big fan of. :-)

I just want to highlight, one might get a false sense of message authenticity, if one just briefly looks at the hashes.
[47°09′51″S, 126°43′39″W] Taking samples
@movq Ah, cool. :-)
@lyse I think that’s what we would *have to* enforce – otherwise we’d run into the problem you’ve outlined. 😃
@lyse I think that’s what we would *have to* enforce – otherwise we’d run into the problem you’ve outlined. 😃
@lyse I think that’s what we would *have to* enforce – otherwise we’d run into the problem you’ve outlined. 😃
@lyse I think that’s what we would *have to* enforce – otherwise we’d run into the problem you’ve outlined. 😃
@falsifian I think we’re talking about different ideas here. 🤔

Maybe it’s time to draft all this into a spec or, rather, two different specs. I might do that over the weekend.
@falsifian I think we’re talking about different ideas here. 🤔

Maybe it’s time to draft all this into a spec or, rather, two different specs. I might do that over the weekend.
@falsifian I think we’re talking about different ideas here. 🤔

Maybe it’s time to draft all this into a spec or, rather, two different specs. I might do that over the weekend.
@falsifian I think we’re talking about different ideas here. 🤔

Maybe it’s time to draft all this into a spec or, rather, two different specs. I might do that over the weekend.
It just occurs to me we're now building some kind of control structures or commands with (edit:…) and (delete:…) into feeds. It's not just a simple "add this to your cache" or "replace the cache with this set of messages" anymore. Hmm. We might need to think about the consequences of that, can this be exploited somehow, etc.
@movq Not sure if I like the idea of keeping the original message around. It goes against the spirit of an edit in my mind.

If that's what we want to enforce, forget about my other message above in the thread.
@prologic @movq I still don't understand it. If the original message has been replaced with the edited one, I cannot verify that the original was in the same feed. I don't know the original text.
@movq I don't think it has to be like that. Just make sure the new version of the twt is always appended to your current feed, and have some convention for indicating it's an edit and which twt it supersedes. Keep the original twt as-is (or delete it if you don't want new followers to see it); doesn't matter if it's archived because you aren't changing that copy.
@prologic Do you have a link to some past discussion?

Would the GDPR would apply to a one-person client like jenny? I seriously hope not. If someone asks me to delete an email they sent me, I don't think I have to honour that request, no matter how European they are.

I am really bothered by the idea that someone could force me to delete my private, personal record of my interactions with them. Would I have to delete my journal entries about them too if they asked?

Maybe a public-facing client like yarnd needs to consider this, but that also bothers me. I was actually thinking about making an Internet Archive style twtxt archiver, letting you explore past twts, including long-dead feeds, see edit histories, deleted twts, etc.
@movq Ok 😅
@movq Ok 😅
@prologic Nah, just language barrier and/or me being a big stupid. 🥴 All good. 👌
@prologic Nah, just language barrier and/or me being a big stupid. 🥴 All good. 👌
@prologic Nah, just language barrier and/or me being a big stupid. 🥴 All good. 👌
@prologic Nah, just language barrier and/or me being a big stupid. 🥴 All good. 👌
@movq Hmmm not sure what I was thinking sorry 🤦‍♂️been a long day 😂
@movq Hmmm not sure what I was thinking sorry 🤦‍♂️been a long day 😂
@movq Am I missing something? 😅
@movq Am I missing something? 😅
@prologic Okay, looks like I misunderstood/misinterpreted your previous message then. 👌
@prologic Okay, looks like I misunderstood/misinterpreted your previous message then. 👌
@prologic Okay, looks like I misunderstood/misinterpreted your previous message then. 👌
@prologic Okay, looks like I misunderstood/misinterpreted your previous message then. 👌
@movq Precisely 👌
@movq Precisely 👌
@movq Is t it? You read each Twt and compute its hash. It's a simple O(1) lookup of the hash in that feed or your cache/archive right?
@movq Is t it? You read each Twt and compute its hash. It's a simple O(1) lookup of the hash in that feed or your cache/archive right?
@prologic So, this is either me nit-picking or me not understanding the hash system after all. 😃

An edit twt would look like this:

2024-09-20T14:57:11Z (edit:#123467) foobar

So we now have to verify that #123467 actually exists in this same feed. How do we do that? We must build a list of all twts/hashes of this feed and then check if #123467 is in that list. Right?

You’re kind of implying that it would be possible to cryptographically validate that this hash belongs to this feed. That’s not possible, is it? 🤔
@prologic So, this is either me nit-picking or me not understanding the hash system after all. 😃

An edit twt would look like this:

2024-09-20T14:57:11Z (edit:#123467) foobar

So we now have to verify that #123467 actually exists in this same feed. How do we do that? We must build a list of all twts/hashes of this feed and then check if #123467 is in that list. Right?

You’re kind of implying that it would be possible to cryptographically validate that this hash belongs to this feed. That’s not possible, is it? 🤔
@prologic So, this is either me nit-picking or me not understanding the hash system after all. 😃

An edit twt would look like this:

2024-09-20T14:57:11Z (edit:#123467) foobar

So we now have to verify that #123467 actually exists in this same feed. How do we do that? We must build a list of all twts/hashes of this feed and then check if #123467 is in that list. Right?

You’re kind of implying that it would be possible to cryptographically validate that this hash belongs to this feed. That’s not possible, is it? 🤔
@prologic So, this is either me nit-picking or me not understanding the hash system after all. 😃

An edit twt would look like this:

2024-09-20T14:57:11Z (edit:#123467) foobar

So we now have to verify that #123467 actually exists in this same feed. How do we do that? We must build a list of all twts/hashes of this feed and then check if #123467 is in that list. Right?

You’re kind of implying that it would be possible to cryptographically validate that this hash belongs to this feed. That’s not possible, is it? 🤔
One distinct disadvantage of (replyto:…) over (edit:#): (replyto:…) relies on clients always processing the entire feed – otherwise they wouldn’t even notice when a twt gets updated. a) This is more expensive, b) you cannot edit twts once they get rotated into an archived feed, because there is nothing signalling clients that they have to re-fetch that archived feed.

I guess neither matters that much in practice. It’s still a disadvantage.
One distinct disadvantage of (replyto:…) over (edit:#): (replyto:…) relies on clients always processing the entire feed – otherwise they wouldn’t even notice when a twt gets updated. a) This is more expensive, b) you cannot edit twts once they get rotated into an archived feed, because there is nothing signalling clients that they have to re-fetch that archived feed.

I guess neither matters that much in practice. It’s still a disadvantage.
One distinct disadvantage of (replyto:…) over (edit:#): (replyto:…) relies on clients always processing the entire feed – otherwise they wouldn’t even notice when a twt gets updated. a) This is more expensive, b) you cannot edit twts once they get rotated into an archived feed, because there is nothing signalling clients that they have to re-fetch that archived feed.

I guess neither matters that much in practice. It’s still a disadvantage.
One distinct disadvantage of (replyto:…) over (edit:#): (replyto:…) relies on clients always processing the entire feed – otherwise they wouldn’t even notice when a twt gets updated. a) This is more expensive, b) you cannot edit twts once they get rotated into an archived feed, because there is nothing signalling clients that they have to re-fetch that archived feed.

I guess neither matters that much in practice. It’s still a disadvantage.
Held another “talk” about Git today at work. It was covering some “basics” about what’s going on in the .git directory. Last time I did that was over 11 years ago. 😅 (I often give introductions about Git, but they’re about day to day usage and very high-level.)

I’ve gotta say, Git is one of the very few pieces of software that I love using and teaching. The files on your disk follow a simple enough format/pattern and you can actually teach people how it all works and, for example, *why* things like rebasing produce a particular result. 👌
Held another “talk” about Git today at work. It was covering some “basics” about what’s going on in the .git directory. Last time I did that was over 11 years ago. 😅 (I often give introductions about Git, but they’re about day to day usage and very high-level.)

I’ve gotta say, Git is one of the very few pieces of software that I love using and teaching. The files on your disk follow a simple enough format/pattern and you can actually teach people how it all works and, for example, *why* things like rebasing produce a particular result. 👌
Held another “talk” about Git today at work. It was covering some “basics” about what’s going on in the .git directory. Last time I did that was over 11 years ago. 😅 (I often give introductions about Git, but they’re about day to day usage and very high-level.)

I’ve gotta say, Git is one of the very few pieces of software that I love using and teaching. The files on your disk follow a simple enough format/pattern and you can actually teach people how it all works and, for example, *why* things like rebasing produce a particular result. 👌
Held another “talk” about Git today at work. It was covering some “basics” about what’s going on in the .git directory. Last time I did that was over 11 years ago. 😅 (I often give introductions about Git, but they’re about day to day usage and very high-level.)

I’ve gotta say, Git is one of the very few pieces of software that I love using and teaching. The files on your disk follow a simple enough format/pattern and you can actually teach people how it all works and, for example, *why* things like rebasing produce a particular result. 👌
@lyse I’m gonna do some self-tests on face blindness. 😂
@lyse I’m gonna do some self-tests on face blindness. 😂
@lyse I’m gonna do some self-tests on face blindness. 😂
@lyse I’m gonna do some self-tests on face blindness. 😂
@lyse

> So, what would happen if there is no original message anymore in the feed and you encounter an "edit" subject?

We’d *have to* classify this as invalid and discard it. If the referenced twt is not present in the feed (or any archived feed), then it might potentially belong to some other feed, and feeds overwriting the contents of other feeds is pretty bad. 😅

As @prologic said, clients must always check that twts referenced by edit and delete are actually present in that very feed.
@lyse

> So, what would happen if there is no original message anymore in the feed and you encounter an "edit" subject?

We’d *have to* classify this as invalid and discard it. If the referenced twt is not present in the feed (or any archived feed), then it might potentially belong to some other feed, and feeds overwriting the contents of other feeds is pretty bad. 😅

As @prologic said, clients must always check that twts referenced by edit and delete are actually present in that very feed.
@lyse

> So, what would happen if there is no original message anymore in the feed and you encounter an "edit" subject?

We’d *have to* classify this as invalid and discard it. If the referenced twt is not present in the feed (or any archived feed), then it might potentially belong to some other feed, and feeds overwriting the contents of other feeds is pretty bad. 😅

As @prologic said, clients must always check that twts referenced by edit and delete are actually present in that very feed.
@lyse

> So, what would happen if there is no original message anymore in the feed and you encounter an "edit" subject?

We’d *have to* classify this as invalid and discard it. If the referenced twt is not present in the feed (or any archived feed), then it might potentially belong to some other feed, and feeds overwriting the contents of other feeds is pretty bad. 😅

As @prologic said, clients must always check that twts referenced by edit and delete are actually present in that very feed.