# I am the Watcher. I am your guide through this vast new twtiverse.
# 
# Usage:
#     https://watcher.sour.is/api/plain/users              View list of users and latest twt date.
#     https://watcher.sour.is/api/plain/twt                View all twts.
#     https://watcher.sour.is/api/plain/mentions?uri=:uri  View all mentions for uri.
#     https://watcher.sour.is/api/plain/conv/:hash         View all twts for a conversation subject.
# 
# Options:
#     uri     Filter to show a specific users twts.
#     offset  Start index for quey.
#     limit   Count of items to return (going back in time).
# 
# twt range = 1 196279
# self = https://watcher.sour.is?offset=171525
# next = https://watcher.sour.is?offset=171625
# prev = https://watcher.sour.is?offset=171425
Ah, and now he is "conveniently" sleeping. How, well, convenient! LOL.
Ah, and now he is "conveniently" sleeping. How, well, convenient! LOL.
@lyse yeah, tell us, @prologic, what isn't true? 🤔 You can't just go around, "that's not true, and that's not true; and that, and that!" without spelling out exactly what isn't, and why? For the love of god, why?! 😂
@lyse yeah, tell us, @prologic, what isn't true? 🤔 You can't just go around, "that's not true, and that's not true; and that, and that!" without spelling out exactly what isn't, and why? For the love of god, why?! 😂
@falsifian Something similar exists over at https://search.twtxt.net/. But a usable search engine would be actually nice (to be fair, yarns improved a bit). :-) I don't care about feed changes over time. In fact, it would even feel creepy to me. Of course, anyone could still surveil, but I'm not looking forward to these stats.
@movq We could still let the client display a warning if it cannot verify it. But yeah.
@falsifian comments on the feeds as in nick, url, follow, that kind of thing? If that, then not interested at all. I envision an archive that would allow searching, and potentially browsing threads on a nice, neat interface. You will have to think, though, on other things. Like, what to do with images? Yarn allows users to upload images, but also embed it in twtxts from other sources (hotlinking, actually).
@falsifian comments on the feeds as in nick, url, follow, that kind of thing? If that, then not interested at all. I envision an archive that would allow searching, and potentially browsing threads on a nice, neat interface. You will have to think, though, on other things. Like, what to do with images? Yarn allows users to upload images, but also embed it in twtxts from other sources (hotlinking, actually).
@david Thanks, that's good feedback to have. I wonder to what extent this already exists in registry servers and yarn pods. I haven't really tried digging into the past in either one.

How interested would you be in changes in metadata and other comments in the feeds? I'm thinking of just permanently saving every version of each twtxt file that gets pulled, not just the twts. It wouldn't be hard to do (though presenting the information in a sensible way is another matter). Compression should make storage a non-issue unless someone does something weird with their feed like shuffle the comments around every time I fetch it.
I ended up installing Headscale on my little VPS. Just in case the collide, I turned off WireGuard. Turning that one off (which ran on a container) also frees some memory. Headscale is running quite well! Indeed, I have struggled getting any web management console to work, but it really isn't needed. Everything needed to commandeer the server is available through the CLI.
I ended up installing Headscale on my little VPS. Just in case the collide, I turned off WireGuard. Turning that one off (which ran on a container) also frees some memory. Headscale is running quite well! Indeed, I have struggled getting any web management console to work, but it really isn't needed. Everything needed to commandeer the server is available through the CLI.
@movq Reminds me of this beautiful face recognition failure: https://qz.com/823820/carnegie-mellon-made-a-special-pair-of-glasses-that-lets-you-steal-a-digital-identity :-D
@falsifian "*I was actually thinking about making an Internet Archive style twtxt archiver, letting you explore past twts*" --- that's an awesome idea for a project. Something I would certainly use!
@falsifian "*I was actually thinking about making an Internet Archive style twtxt archiver, letting you explore past twts*" --- that's an awesome idea for a project. Something I would certainly use!
@prologic What exactly?
@prologic Just what @bender did. :-D If he'd additionally serve the fake message from his yarnd twt endpoint, everybody querying that hash from him (or any other yarnd that synced it in the meantime) would believe, that I didn't like Australians.

In fact, I really don't. I love'em! 8-)

We would need to sign each message in a feed, so others could verify that this was actually part of that feed and not made up. But then we end up in the crypto debate for identities again, which I'm not a big fan of. :-)

I just want to highlight, one might get a false sense of message authenticity, if one just briefly looks at the hashes.
[47°09′51″S, 126°43′39″W] Taking samples
@movq Ah, cool. :-)
@lyse I think that’s what we would *have to* enforce – otherwise we’d run into the problem you’ve outlined. 😃
@lyse I think that’s what we would *have to* enforce – otherwise we’d run into the problem you’ve outlined. 😃
@lyse I think that’s what we would *have to* enforce – otherwise we’d run into the problem you’ve outlined. 😃
@lyse I think that’s what we would *have to* enforce – otherwise we’d run into the problem you’ve outlined. 😃
@falsifian I think we’re talking about different ideas here. 🤔

Maybe it’s time to draft all this into a spec or, rather, two different specs. I might do that over the weekend.
@falsifian I think we’re talking about different ideas here. 🤔

Maybe it’s time to draft all this into a spec or, rather, two different specs. I might do that over the weekend.
@falsifian I think we’re talking about different ideas here. 🤔

Maybe it’s time to draft all this into a spec or, rather, two different specs. I might do that over the weekend.
@falsifian I think we’re talking about different ideas here. 🤔

Maybe it’s time to draft all this into a spec or, rather, two different specs. I might do that over the weekend.
It just occurs to me we're now building some kind of control structures or commands with (edit:…) and (delete:…) into feeds. It's not just a simple "add this to your cache" or "replace the cache with this set of messages" anymore. Hmm. We might need to think about the consequences of that, can this be exploited somehow, etc.
@movq Not sure if I like the idea of keeping the original message around. It goes against the spirit of an edit in my mind.

If that's what we want to enforce, forget about my other message above in the thread.
@prologic @movq I still don't understand it. If the original message has been replaced with the edited one, I cannot verify that the original was in the same feed. I don't know the original text.
@movq I don't think it has to be like that. Just make sure the new version of the twt is always appended to your current feed, and have some convention for indicating it's an edit and which twt it supersedes. Keep the original twt as-is (or delete it if you don't want new followers to see it); doesn't matter if it's archived because you aren't changing that copy.
@prologic Do you have a link to some past discussion?

Would the GDPR would apply to a one-person client like jenny? I seriously hope not. If someone asks me to delete an email they sent me, I don't think I have to honour that request, no matter how European they are.

I am really bothered by the idea that someone could force me to delete my private, personal record of my interactions with them. Would I have to delete my journal entries about them too if they asked?

Maybe a public-facing client like yarnd needs to consider this, but that also bothers me. I was actually thinking about making an Internet Archive style twtxt archiver, letting you explore past twts, including long-dead feeds, see edit histories, deleted twts, etc.
@movq Ok 😅
@movq Ok 😅
@prologic Nah, just language barrier and/or me being a big stupid. 🥴 All good. 👌
@prologic Nah, just language barrier and/or me being a big stupid. 🥴 All good. 👌
@prologic Nah, just language barrier and/or me being a big stupid. 🥴 All good. 👌
@prologic Nah, just language barrier and/or me being a big stupid. 🥴 All good. 👌
@movq Hmmm not sure what I was thinking sorry 🤦‍♂️been a long day 😂
@movq Hmmm not sure what I was thinking sorry 🤦‍♂️been a long day 😂
@movq Am I missing something? 😅
@movq Am I missing something? 😅
@prologic Okay, looks like I misunderstood/misinterpreted your previous message then. 👌
@prologic Okay, looks like I misunderstood/misinterpreted your previous message then. 👌
@prologic Okay, looks like I misunderstood/misinterpreted your previous message then. 👌
@prologic Okay, looks like I misunderstood/misinterpreted your previous message then. 👌
@movq Precisely 👌
@movq Precisely 👌
@movq Is t it? You read each Twt and compute its hash. It's a simple O(1) lookup of the hash in that feed or your cache/archive right?
@movq Is t it? You read each Twt and compute its hash. It's a simple O(1) lookup of the hash in that feed or your cache/archive right?
@prologic So, this is either me nit-picking or me not understanding the hash system after all. 😃

An edit twt would look like this:

2024-09-20T14:57:11Z (edit:#123467) foobar

So we now have to verify that #123467 actually exists in this same feed. How do we do that? We must build a list of all twts/hashes of this feed and then check if #123467 is in that list. Right?

You’re kind of implying that it would be possible to cryptographically validate that this hash belongs to this feed. That’s not possible, is it? 🤔
@prologic So, this is either me nit-picking or me not understanding the hash system after all. 😃

An edit twt would look like this:

2024-09-20T14:57:11Z (edit:#123467) foobar

So we now have to verify that #123467 actually exists in this same feed. How do we do that? We must build a list of all twts/hashes of this feed and then check if #123467 is in that list. Right?

You’re kind of implying that it would be possible to cryptographically validate that this hash belongs to this feed. That’s not possible, is it? 🤔
@prologic So, this is either me nit-picking or me not understanding the hash system after all. 😃

An edit twt would look like this:

2024-09-20T14:57:11Z (edit:#123467) foobar

So we now have to verify that #123467 actually exists in this same feed. How do we do that? We must build a list of all twts/hashes of this feed and then check if #123467 is in that list. Right?

You’re kind of implying that it would be possible to cryptographically validate that this hash belongs to this feed. That’s not possible, is it? 🤔
@prologic So, this is either me nit-picking or me not understanding the hash system after all. 😃

An edit twt would look like this:

2024-09-20T14:57:11Z (edit:#123467) foobar

So we now have to verify that #123467 actually exists in this same feed. How do we do that? We must build a list of all twts/hashes of this feed and then check if #123467 is in that list. Right?

You’re kind of implying that it would be possible to cryptographically validate that this hash belongs to this feed. That’s not possible, is it? 🤔
One distinct disadvantage of (replyto:…) over (edit:#): (replyto:…) relies on clients always processing the entire feed – otherwise they wouldn’t even notice when a twt gets updated. a) This is more expensive, b) you cannot edit twts once they get rotated into an archived feed, because there is nothing signalling clients that they have to re-fetch that archived feed.

I guess neither matters that much in practice. It’s still a disadvantage.
One distinct disadvantage of (replyto:…) over (edit:#): (replyto:…) relies on clients always processing the entire feed – otherwise they wouldn’t even notice when a twt gets updated. a) This is more expensive, b) you cannot edit twts once they get rotated into an archived feed, because there is nothing signalling clients that they have to re-fetch that archived feed.

I guess neither matters that much in practice. It’s still a disadvantage.
One distinct disadvantage of (replyto:…) over (edit:#): (replyto:…) relies on clients always processing the entire feed – otherwise they wouldn’t even notice when a twt gets updated. a) This is more expensive, b) you cannot edit twts once they get rotated into an archived feed, because there is nothing signalling clients that they have to re-fetch that archived feed.

I guess neither matters that much in practice. It’s still a disadvantage.
One distinct disadvantage of (replyto:…) over (edit:#): (replyto:…) relies on clients always processing the entire feed – otherwise they wouldn’t even notice when a twt gets updated. a) This is more expensive, b) you cannot edit twts once they get rotated into an archived feed, because there is nothing signalling clients that they have to re-fetch that archived feed.

I guess neither matters that much in practice. It’s still a disadvantage.
Held another “talk” about Git today at work. It was covering some “basics” about what’s going on in the .git directory. Last time I did that was over 11 years ago. 😅 (I often give introductions about Git, but they’re about day to day usage and very high-level.)

I’ve gotta say, Git is one of the very few pieces of software that I love using and teaching. The files on your disk follow a simple enough format/pattern and you can actually teach people how it all works and, for example, *why* things like rebasing produce a particular result. 👌
Held another “talk” about Git today at work. It was covering some “basics” about what’s going on in the .git directory. Last time I did that was over 11 years ago. 😅 (I often give introductions about Git, but they’re about day to day usage and very high-level.)

I’ve gotta say, Git is one of the very few pieces of software that I love using and teaching. The files on your disk follow a simple enough format/pattern and you can actually teach people how it all works and, for example, *why* things like rebasing produce a particular result. 👌
Held another “talk” about Git today at work. It was covering some “basics” about what’s going on in the .git directory. Last time I did that was over 11 years ago. 😅 (I often give introductions about Git, but they’re about day to day usage and very high-level.)

I’ve gotta say, Git is one of the very few pieces of software that I love using and teaching. The files on your disk follow a simple enough format/pattern and you can actually teach people how it all works and, for example, *why* things like rebasing produce a particular result. 👌
Held another “talk” about Git today at work. It was covering some “basics” about what’s going on in the .git directory. Last time I did that was over 11 years ago. 😅 (I often give introductions about Git, but they’re about day to day usage and very high-level.)

I’ve gotta say, Git is one of the very few pieces of software that I love using and teaching. The files on your disk follow a simple enough format/pattern and you can actually teach people how it all works and, for example, *why* things like rebasing produce a particular result. 👌
@lyse I’m gonna do some self-tests on face blindness. 😂
@lyse I’m gonna do some self-tests on face blindness. 😂
@lyse I’m gonna do some self-tests on face blindness. 😂
@lyse I’m gonna do some self-tests on face blindness. 😂
@lyse

> So, what would happen if there is no original message anymore in the feed and you encounter an "edit" subject?

We’d *have to* classify this as invalid and discard it. If the referenced twt is not present in the feed (or any archived feed), then it might potentially belong to some other feed, and feeds overwriting the contents of other feeds is pretty bad. 😅

As @prologic said, clients must always check that twts referenced by edit and delete are actually present in that very feed.
@lyse

> So, what would happen if there is no original message anymore in the feed and you encounter an "edit" subject?

We’d *have to* classify this as invalid and discard it. If the referenced twt is not present in the feed (or any archived feed), then it might potentially belong to some other feed, and feeds overwriting the contents of other feeds is pretty bad. 😅

As @prologic said, clients must always check that twts referenced by edit and delete are actually present in that very feed.
@lyse

> So, what would happen if there is no original message anymore in the feed and you encounter an "edit" subject?

We’d *have to* classify this as invalid and discard it. If the referenced twt is not present in the feed (or any archived feed), then it might potentially belong to some other feed, and feeds overwriting the contents of other feeds is pretty bad. 😅

As @prologic said, clients must always check that twts referenced by edit and delete are actually present in that very feed.
@lyse

> So, what would happen if there is no original message anymore in the feed and you encounter an "edit" subject?

We’d *have to* classify this as invalid and discard it. If the referenced twt is not present in the feed (or any archived feed), then it might potentially belong to some other feed, and feeds overwriting the contents of other feeds is pretty bad. 😅

As @prologic said, clients must always check that twts referenced by edit and delete are actually present in that very feed.
@prologic what time in UTC?
@prologic what time in UTC?
@prologic what time in UTC?
@prologic what time in UTC?
What about edits of edits? Do we want to “chain” edits or does the latest edit simply win?

Chained edits:

[#abcd111] [2024-09-20T12:00:00Z] [Hello!]
[#abcd222] [2024-09-20T12:10:00Z] [(edit:#abcd111) Hello World!]
[#abcd333] [2024-09-20T12:20:00Z] [(edit:#abcd222) Hello Birds!]

Latest edit wins:

[#abcd111] [2024-09-20T12:00:00Z] [Hello!]
[#abcd222] [2024-09-20T12:10:00Z] [(edit:#abcd111) Hello World!]
[#abcd333] [2024-09-20T12:20:00Z] [(edit:#abcd111) Hello Birds!]

Does the first version have any benefits? I don’t think so … ?
What about edits of edits? Do we want to “chain” edits or does the latest edit simply win?

Chained edits:

[#abcd111] [2024-09-20T12:00:00Z] [Hello!]
[#abcd222] [2024-09-20T12:10:00Z] [(edit:#abcd111) Hello World!]
[#abcd333] [2024-09-20T12:20:00Z] [(edit:#abcd222) Hello Birds!]

Latest edit wins:

[#abcd111] [2024-09-20T12:00:00Z] [Hello!]
[#abcd222] [2024-09-20T12:10:00Z] [(edit:#abcd111) Hello World!]
[#abcd333] [2024-09-20T12:20:00Z] [(edit:#abcd111) Hello Birds!]

Does the first version have any benefits? I don’t think so … ?
What about edits of edits? Do we want to “chain” edits or does the latest edit simply win?

Chained edits:

\n \n \n
\n \n [(edit:#abcd111) Hello World!]
\n \n [(edit:#abcd222) Hello Birds!]

Latest edit wins:

\n \n \n
\n \n [(edit:#abcd111) Hello World!]
\n \n [(edit:#abcd111) Hello Birds!]

Does the first version have any benefits? I don’t think so … ?
What about edits of edits? Do we want to “chain” edits or does the latest edit simply win?

Chained edits:

[#abcd111] [2024-09-20T12:00:00Z] [Hello!]
[#abcd222] [2024-09-20T12:10:00Z] [(edit:#abcd111) Hello World!]
[#abcd333] [2024-09-20T12:20:00Z] [(edit:#abcd222) Hello Birds!]

Latest edit wins:

[#abcd111] [2024-09-20T12:00:00Z] [Hello!]
[#abcd222] [2024-09-20T12:10:00Z] [(edit:#abcd111) Hello World!]
[#abcd333] [2024-09-20T12:20:00Z] [(edit:#abcd111) Hello Birds!]

Does the first version have any benefits? I don’t think so … ?
What about edits of edits? Do we want to “chain” edits or does the latest edit simply win?

Chained edits:

[#abcd111] [2024-09-20T12:00:00Z] [Hello!]
[#abcd222] [2024-09-20T12:10:00Z] [(edit:#abcd111) Hello World!]
[#abcd333] [2024-09-20T12:20:00Z] [(edit:#abcd222) Hello Birds!]

Latest edit wins:

[#abcd111] [2024-09-20T12:00:00Z] [Hello!]
[#abcd222] [2024-09-20T12:10:00Z] [(edit:#abcd111) Hello World!]
[#abcd333] [2024-09-20T12:20:00Z] [(edit:#abcd111) Hello Birds!]

Does the first version have any benefits? I don’t think so … ?
@prologic Yeah, you’re right. That’s an implementation detail of jenny. Right now, the order of twts doesn’t matter at all, because it’s only relevant at display time – and that’s the job of mutt. 😅
@prologic Yeah, you’re right. That’s an implementation detail of jenny. Right now, the order of twts doesn’t matter at all, because it’s only relevant at display time – and that’s the job of mutt. 😅
@prologic Yeah, you’re right. That’s an implementation detail of jenny. Right now, the order of twts doesn’t matter at all, because it’s only relevant at display time – and that’s the job of mutt. 😅
@prologic Yeah, you’re right. That’s an implementation detail of jenny. Right now, the order of twts doesn’t matter at all, because it’s only relevant at display time – and that’s the job of mutt. 😅
@falsifian Oof, yeah, I haven’t even started thinking about supporting two schemes at the same time. 😅 I’d be hoping for *not* having to use something like an sqlite database, if it can’t be avoided.

By the way: Since we have so few modern twtxt/Yarn clients, forking jenny might not be the worst idea. *If* you wanted to take it into a very different direction, then by all means, go for it. 👍
@falsifian Oof, yeah, I haven’t even started thinking about supporting two schemes at the same time. 😅 I’d be hoping for *not* having to use something like an sqlite database, if it can’t be avoided.

By the way: Since we have so few modern twtxt/Yarn clients, forking jenny might not be the worst idea. *If* you wanted to take it into a very different direction, then by all means, go for it. 👍
@falsifian Oof, yeah, I haven’t even started thinking about supporting two schemes at the same time. 😅 I’d be hoping for *not* having to use something like an sqlite database, if it can’t be avoided.

By the way: Since we have so few modern twtxt/Yarn clients, forking jenny might not be the worst idea. *If* you wanted to take it into a very different direction, then by all means, go for it. 👍
@falsifian Oof, yeah, I haven’t even started thinking about supporting two schemes at the same time. 😅 I’d be hoping for *not* having to use something like an sqlite database, if it can’t be avoided.

By the way: Since we have so few modern twtxt/Yarn clients, forking jenny might not be the worst idea. *If* you wanted to take it into a very different direction, then by all means, go for it. 👍
@lyse When it asks a Yarn pod, you mean? Yeah, it does so implicitly. It builds a tiny dummy feed from the JSON response and then looks for the specified twt hash in that feed.
@lyse When it asks a Yarn pod, you mean? Yeah, it does so implicitly. It builds a tiny dummy feed from the JSON response and then looks for the specified twt hash in that feed.
@lyse When it asks a Yarn pod, you mean? Yeah, it does so implicitly. It builds a tiny dummy feed from the JSON response and then looks for the specified twt hash in that feed.
@lyse When it asks a Yarn pod, you mean? Yeah, it does so implicitly. It builds a tiny dummy feed from the JSON response and then looks for the specified twt hash in that feed.
@prologic Wouldn’t work in what way? Could you elaborate? 🤔

Do you consider crawling archived feeds a problem/failure? 🤔
@prologic Wouldn’t work in what way? Could you elaborate? 🤔

Do you consider crawling archived feeds a problem/failure? 🤔
@prologic Wouldn’t work in what way? Could you elaborate? 🤔

Do you consider crawling archived feeds a problem/failure? 🤔
@prologic Wouldn’t work in what way? Could you elaborate? 🤔

Do you consider crawling archived feeds a problem/failure? 🤔
@prologic cool, I will be there! Are you going to post the regular banner notice? It will serve as a reminder, at least for me.
@prologic cool, I will be there! Are you going to post the regular banner notice? It will serve as a reminder, at least for me.
[47°09′46″S, 126°43′19″W] Raw reading: 0x66ED55B1, offset +/-5
I'm looking forward to a sunny weekend with mild temperatures.
AWESOME COOL ☻
👋 Reminder that next Saturday 28th September will be out monthly online meetup! Hope to see some/all of you there 👌