# I am the Watcher. I am your guide through this vast new twtiverse.
# 
# Usage:
#     https://watcher.sour.is/api/plain/users              View list of users and latest twt date.
#     https://watcher.sour.is/api/plain/twt                View all twts.
#     https://watcher.sour.is/api/plain/mentions?uri=:uri  View all mentions for uri.
#     https://watcher.sour.is/api/plain/conv/:hash         View all twts for a conversation subject.
# 
# Options:
#     uri     Filter to show a specific users twts.
#     offset  Start index for quey.
#     limit   Count of items to return (going back in time).
# 
# twt range = 1 196314
# self = https://watcher.sour.is?offset=171199
# next = https://watcher.sour.is?offset=171299
# prev = https://watcher.sour.is?offset=171099
@xuu I don't recall where that discussion ended up being though?
@bender wut da fuq?! 🤣
@bender wut da fuq?! 🤣
@xuu you mean my original idea of basically just automatically detecting Twt edits from the client side?
@xuu you mean my original idea of basically just automatically detecting Twt edits from the client side?
@xuu this is where you would need to prove that the editor delete request actually came from that feed author. Hence why integrity is much more important here.
@xuu this is where you would need to prove that the editor delete request actually came from that feed author. Hence why integrity is much more important here.
@falsifian without supporting dudes properly though you're running into GDP issues and the right to forget. 🤣 we've had pretty lengthy discussions about this in the past years ago as well, but we never came to a conclusion. We're all happy with.
@falsifian without supporting dudes properly though you're running into GDP issues and the right to forget. 🤣 we've had pretty lengthy discussions about this in the past years ago as well, but we never came to a conclusion. We're all happy with.
🧮 USERS:1 FEEDS:2 TWTS:1097 ARCHIVED:79000 CACHE:2495 FOLLOWERS:17 FOLLOWING:14
@movq it would work, you are right, however, it has drawbacks, and I think in the long term would create a new set of problems that we would also then have to solve.
@movq it would work, you are right, however, it has drawbacks, and I think in the long term would create a new set of problems that we would also then have to solve.
@david Hah 🤣
@david Hah 🤣
@prologic :-D Thanks! Things can come in cycles, right? This is simply another one. Another cycle, more personal than the other "alter egos".
@prologic :-D Thanks! Things can come in cycles, right? This is simply another one. Another cycle, more personal than the other "alter egos".
@david We'll get there soon™ 🔜
@david We'll get there soon™ 🔜
@david Hah Welcome back! 😅
@david Hah Welcome back! 😅
@aelaraji hey, hey! You are my very first reply! 👋🏻 Cheers!
@aelaraji hey, hey! You are my very first reply! 👋🏻 Cheers!
@david "Hello back" from the other corner of the world! 🫡
@david "Hello back" from the other corner of the world! 🫡
@david "Hello back" from the other corner of the world! 🫡
Incredibly upset---more than you could imagine---because I already made the first mistake, and corrected it (but twtxt.net got it on it's cache, ugh!) :'-( . Can't wait for editing to become a reality!
Incredibly upset---more than you could imagine---because I already made the first mistake, and corrected it (but twtxt.net got it on it's cache, ugh!) :'-( . Can't wait for editing to become a reality!
Alright. My first mentions---which were picked not so randomly, LOL---are @prologic, @lyse, and @movq. I am also posting my first image too, which you see below. That's my neighbourhood, in a "winter" day. Hopefully @prologic will add my domain to his allowed list, so that the image (and any other further) renders.

David's neighbourhood showing a stone sky.
Alright. My first mentions---which were picked not so randomly, LOL---are @prologic, @lyse, and @movq. I am also posting my first image too, which you see below. That's my neighbourhood, in a "winter" day. Hopefully @prologic will add my domain to his allowed list, so that the image (and any other further) renders.

David's neighbourhood showing a stone sky.
Alright, announce_me set to true. Now, who do I pick to be my first mention? Decisions, decisions. Next twtxt will have my first mention(s). :-)
Alright, announce_me set to true. Now, who do I pick to be my first mention? Decisions, decisions. Next twtxt will have my first mention(s). :-)
I have configured my twtxt.txt as simple as possible. I have setup a publish_command on jenny. Hopefully all works fine, and I am good to go. Next will be setting the announce_me to true. Here we go!
I have configured my twtxt.txt as simple as possible. I have setup a publish_command on jenny. Hopefully all works fine, and I am good to go. Next will be setting the announce_me to true. Here we go!
Everything starts at a "hello world". At least around these parts; the nerdy parts.
Everything starts at a "hello world". At least around these parts; the nerdy parts.
@sorenpeter hmm, how does your client handles "a little editing"? I am sure threads would break just as well. 😉
@sorenpeter hmm, how does your client handles "a little editing"? I am sure threads would break just as well. 😉
@prologic, there is a parser bug on parent. Specifically on this portion:


"*If twtxt/Yarn was to grow bigger, then this would become a concern again. *But even Mastodon allows editing*, so how
+much of a problem can it really be? 😅*"
@prologic, there is a parser bug on parent. Specifically on this portion:


"*If twtxt/Yarn was to grow bigger, then this would become a concern again. *But even Mastodon allows editing*, so how
+much of a problem can it really be? 😅*"
@movq going a little sideways on this, "*If twtxt/Yarn was to grow bigger, then this would become a concern again. *But even Mastodon allows editing*, so how much of a problem can it really be? 😅*", wouldn't it preparing for a potential (even if very, very, veeeeery remote) growth be a good thing? Mastodon signs all messages, keeps a history of edits, and it doesn't break threads. It isn't a problem there.😉 It is here.

I think keeping hashes is a must. If anything for that "feels good" feeling.*
@movq going a little sideways on this, "*If twtxt/Yarn was to grow bigger, then this would become a concern again. *But even Mastodon allows editing*, so how much of a problem can it really be? 😅*", wouldn't it preparing for a potential (even if very, very, veeeeery remote) growth be a good thing? Mastodon signs all messages, keeps a history of edits, and it doesn't break threads. It isn't a problem there.😉 It is here.

I think keeping hashes is a must. If anything for that "feels good" feeling.*
@movq Agreed that hashes have a benefit. I came up with a similar example where when I twted about an 11-character hash collision. Perhaps hashes could be made optional somehow. Like, you could use the "replyto" idea and then additionally put a hash somewhere if you want to lock in which version of the twt you are replying to.
There is nothing wrong with how we currently run a diff to see what has been removed. if i build a merkle tree off all the twt hashes in a feed i can use that to verify a twt should be in a feed or not. and gossip that to my peers.
There is nothing wrong with how we currently run a diff to see what has been removed. if i build a merkle tree off all the twt hashes in a feed i can use that to verify a twt should be in a feed or not. and gossip that to my peers.
(Or maybe I’m talking nonsense. That’s known to happen. I’ll go to bed. 😂)
(Or maybe I’m talking nonsense. That’s known to happen. I’ll go to bed. 😂)
(Or maybe I’m talking nonsense. That’s known to happen. I’ll go to bed. 😂)
(Or maybe I’m talking nonsense. That’s known to happen. I’ll go to bed. 😂)
So.. basically a rehash of the email "unsend" requests? What if i was to make a (delete: 5vbi2ea) .. would it delete someone elses twt?
So.. basically a rehash of the email "unsend" requests? What if i was to make a (delete: 5vbi2ea) .. would it delete someone elses twt?
> Brisbane is coming onboard. Roosters are "singing" all around @prologic, and the dog is begging for the morning poo/pee walk. @prologic throws a slipper at the dog, as he turns around, and hides under his comforter.

😂😂😂
@quark Printing a version? I’ll think about it. 🤔

It would be easy to do for releases, but it’s a little hard to do for all the commits in between – jenny has no build process, so there’s no easy way to incorporate the output of git describe, for example.
@quark Printing a version? I’ll think about it. 🤔

It would be easy to do for releases, but it’s a little hard to do for all the commits in between – jenny has no build process, so there’s no easy way to incorporate the output of git describe, for example.
@quark Printing a version? I’ll think about it. 🤔

It would be easy to do for releases, but it’s a little hard to do for all the commits in between – jenny has no build process, so there’s no easy way to incorporate the output of git describe, for example.
@quark Printing a version? I’ll think about it. 🤔

It would be easy to do for releases, but it’s a little hard to do for all the commits in between – jenny has no build process, so there’s no easy way to incorporate the output of git describe, for example.
isn't the benefit of blake2b that it is a more efficient algo than sha1 and has the same or similar entropy to sha3? i thought we had partially solved this with some type of expanding hash size? additionally we could increase bit density by using base36 or base64/url-safe...
isn't the benefit of blake2b that it is a more efficient algo than sha1 and has the same or similar entropy to sha3? i thought we had partially solved this with some type of expanding hash size? additionally we could increase bit density by using base36 or base64/url-safe...
I’m not advocating in either direction, btw. I haven’t made up my mind yet. 😅 Just braindumping here.

The (replyto:…) proposal is definitely more in the spirit of twtxt, I’d say. It’s much simpler, anyone can use it even with the simplest tools, no need for any client code. That is certainly a great property, if you ask me, and it’s things like that that brought me to twtxt in the first place.

I’d also say that in our tiny little community, message integrity simply doesn’t matter. Signed feeds don’t matter. I signed my feed for a while using GPG, someone else did the same, but in the end, nobody cares. The community is so tiny, there’s enough “implicit trust” or whatever you want to call it.

If twtxt/Yarn was to grow bigger, then this would become a concern again. *But even Mastodon allows editing*, so how much of a problem can it really be? 😅

I do have to “admit”, though, that hashes *feel* better. It feels good to know that we can clearly identify a certain twt. It feels more correct and stable.

Hm.

I *suspect* that the (replyto:…) proposal would work just as well in practice.
I’m not advocating in either direction, btw. I haven’t made up my mind yet. 😅 Just braindumping here.

The (replyto:…) proposal is definitely more in the spirit of twtxt, I’d say. It’s much simpler, anyone can use it even with the simplest tools, no need for any client code. That is certainly a great property, if you ask me, and it’s things like that that brought me to twtxt in the first place.

I’d also say that in our tiny little community, message integrity simply doesn’t matter. Signed feeds don’t matter. I signed my feed for a while using GPG, someone else did the same, but in the end, nobody cares. The community is so tiny, there’s enough “implicit trust” or whatever you want to call it.

If twtxt/Yarn was to grow bigger, then this would become a concern again. *But even Mastodon allows editing*, so how much of a problem can it really be? 😅

I do have to “admit”, though, that hashes *feel* better. It feels good to know that we can clearly identify a certain twt. It feels more correct and stable.

Hm.

I *suspect* that the (replyto:…) proposal would work just as well in practice.
I’m not advocating in either direction, btw. I haven’t made up my mind yet. 😅 Just braindumping here.

The (replyto:…) proposal is definitely more in the spirit of twtxt, I’d say. It’s much simpler, anyone can use it even with the simplest tools, no need for any client code. That is certainly a great property, if you ask me, and it’s things like that that brought me to twtxt in the first place.

I’d also say that in our tiny little community, message integrity simply doesn’t matter. Signed feeds don’t matter. I signed my feed for a while using GPG, someone else did the same, but in the end, nobody cares. The community is so tiny, there’s enough “implicit trust” or whatever you want to call it.

If twtxt/Yarn was to grow bigger, then this would become a concern again. *But even Mastodon allows editing*, so how much of a problem can it really be? 😅

I do have to “admit”, though, that hashes *feel* better. It feels good to know that we can clearly identify a certain twt. It feels more correct and stable.

Hm.

I *suspect* that the (replyto:…) proposal would work just as well in practice.
I’m not advocating in either direction, btw. I haven’t made up my mind yet. 😅 Just braindumping here.

The (replyto:…) proposal is definitely more in the spirit of twtxt, I’d say. It’s much simpler, anyone can use it even with the simplest tools, no need for any client code. That is certainly a great property, if you ask me, and it’s things like that that brought me to twtxt in the first place.

I’d also say that in our tiny little community, message integrity simply doesn’t matter. Signed feeds don’t matter. I signed my feed for a while using GPG, someone else did the same, but in the end, nobody cares. The community is so tiny, there’s enough “implicit trust” or whatever you want to call it.

If twtxt/Yarn was to grow bigger, then this would become a concern again. *But even Mastodon allows editing*, so how much of a problem can it really be? 😅

I do have to “admit”, though, that hashes *feel* better. It feels good to know that we can clearly identify a certain twt. It feels more correct and stable.

Hm.

I *suspect* that the (replyto:…) proposal would work just as well in practice.
Hey, @movq, a tiny thing to add to jenny, a -v switch. That way when you twtxt "*That’s an older format that was used before jenny version v23.04*", I can go and run jenny -v, and "duh!" myself on the way to a git pull. :-D
Hey, @movq, a tiny thing to add to jenny, a -v switch. That way when you twtxt "*That’s an older format that was used before jenny version v23.04*", I can go and run jenny -v, and "duh!" myself on the way to a git pull. :-D
@movq ooooh, nice! commit 62a2b7735749f2ff3c9306dd984ad28f853595c5:

> Crawl archived feeds in --fetch-context

Like, very much! :-)
@movq ooooh, nice! commit 62a2b7735749f2ff3c9306dd984ad28f853595c5:

> Crawl archived feeds in --fetch-context

Like, very much! :-)
@falsifian @prologic @lyse

> - editing, if you don't care about message integrity

So that’s the big question, because that’s the only real difference between hashes and the (replyto:…) proposal.

Do we care about message integrity?

With (replyto:…), someone could write a twt, then I reply to it, like “you’re absolutely right!”, and then that person could change their twt to something malicious like “the earth is flat!” And then it would look like I’m a nutcase agreeing with that person. 😅

Hashes (in their current form) prevent that. The thread is broken and my reply clearly refers to something else. That’s good, right?

But now take into account that we want to allow editing anyway. Is there even a point to using hashes anymore? Isn’t message integrity ignored anyway now, at least in practice?

There’s no difference (in practice) between someone writing

2024-09-18T12:34Z Brds are great!

and then editing it to either

2024-09-18T12:34Z (original:#12379) Birds are great! (Whoops, fixed a typo.)

or

2024-09-18T12:34Z (original:#12379) The earth is flat!

The actual original message is (potentially) gone. The only thing that we can be sure of now is that the twt was edited in *some* way. *Essentially*, the actual twt message is no longer part of the hash, is it? What does #12379 refer to? The edited message or the original one? We *want* it to refer to the edited one, because we don’t want to break threads, so … what’s the point of using a hash?
@falsifian @prologic @lyse

> - editing, if you don't care about message integrity

So that’s the big question, because that’s the only real difference between hashes and the (replyto:…) proposal.

Do we care about message integrity?

With (replyto:…), someone could write a twt, then I reply to it, like “you’re absolutely right!”, and then that person could change their twt to something malicious like “the earth is flat!” And then it would look like I’m a nutcase agreeing with that person. 😅

Hashes (in their current form) prevent that. The thread is broken and my reply clearly refers to something else. That’s good, right?

But now take into account that we want to allow editing anyway. Is there even a point to using hashes anymore? Isn’t message integrity ignored anyway now, at least in practice?

There’s no difference (in practice) between someone writing

2024-09-18T12:34Z Brds are great!

and then editing it to either

2024-09-18T12:34Z (original:#12379) Birds are great! (Whoops, fixed a typo.)

or

2024-09-18T12:34Z (original:#12379) The earth is flat!

The actual original message is (potentially) gone. The only thing that we can be sure of now is that the twt was edited in *some* way. *Essentially*, the actual twt message is no longer part of the hash, is it? What does #12379 refer to? The edited message or the original one? We *want* it to refer to the edited one, because we don’t want to break threads, so … what’s the point of using a hash?
@falsifian @prologic @lyse

> - editing, if you don't care about message integrity

So that’s the big question, because that’s the only real difference between hashes and the (replyto:…) proposal.

Do we care about message integrity?

With (replyto:…), someone could write a twt, then I reply to it, like “you’re absolutely right!”, and then that person could change their twt to something malicious like “the earth is flat!” And then it would look like I’m a nutcase agreeing with that person. 😅

Hashes (in their current form) prevent that. The thread is broken and my reply clearly refers to something else. That’s good, right?

But now take into account that we want to allow editing anyway. Is there even a point to using hashes anymore? Isn’t message integrity ignored anyway now, at least in practice?

There’s no difference (in practice) between someone writing

2024-09-18T12:34Z Brds are great!

and then editing it to either

2024-09-18T12:34Z (original:#12379) Birds are great! (Whoops, fixed a typo.)

or

2024-09-18T12:34Z (original:#12379) The earth is flat!

The actual original message is (potentially) gone. The only thing that we can be sure of now is that the twt was edited in *some* way. *Essentially*, the actual twt message is no longer part of the hash, is it? What does #12379 refer to? The edited message or the original one? We *want* it to refer to the edited one, because we don’t want to break threads, so … what’s the point of using a hash?
@falsifian @prologic @lyse

> - editing, if you don't care about message integrity

So that’s the big question, because that’s the only real difference between hashes and the (replyto:…) proposal.

Do we care about message integrity?

With (replyto:…), someone could write a twt, then I reply to it, like “you’re absolutely right!”, and then that person could change their twt to something malicious like “the earth is flat!” And then it would look like I’m a nutcase agreeing with that person. 😅

Hashes (in their current form) prevent that. The thread is broken and my reply clearly refers to something else. That’s good, right?

But now take into account that we want to allow editing anyway. Is there even a point to using hashes anymore? Isn’t message integrity ignored anyway now, at least in practice?

There’s no difference (in practice) between someone writing

2024-09-18T12:34Z Brds are great!

and then editing it to either

2024-09-18T12:34Z (original:#12379) Birds are great! (Whoops, fixed a typo.)

or

2024-09-18T12:34Z (original:#12379) The earth is flat!

The actual original message is (potentially) gone. The only thing that we can be sure of now is that the twt was edited in *some* way. *Essentially*, the actual twt message is no longer part of the hash, is it? What does #12379 refer to? The edited message or the original one? We *want* it to refer to the edited one, because we don’t want to break threads, so … what’s the point of using a hash?
@movq to paraphrase US Presidents speech on each State of the Union, "the State of the Jenny is strong!" :-D As for the potential upcoming changes, there has to be a knowledgeable head honcho that will agglomerate and coalesce, and guide onto the direction that will be taken. All that with the strong input from the developers that will be implementing the changes, and a lesser (but not less valuable) input from users.
@movq to paraphrase US Presidents speech on each State of the Union, "the State of the Jenny is strong!" :-D As for the potential upcoming changes, there has to be a knowledgeable head honcho that will agglomerate and coalesce, and guide onto the direction that will be taken. All that with the strong input from the developers that will be implementing the changes, and a lesser (but not less valuable) input from users.
[47°09′45″S, 126°43′35″W] Transfer completed
Better is better than best. Wisdom droplet from the Google developer documentation style guide.
Regarding jenny development: There have been enough changes in the last few weeks, imo. I want to let things settle for a while (potential bugfixes aside) and then I’m going to cut a new release.

And I guess the release after that is going to include all the threading/hashing stuff – if we can decide on one of the proposals. 😂
Regarding jenny development: There have been enough changes in the last few weeks, imo. I want to let things settle for a while (potential bugfixes aside) and then I’m going to cut a new release.

And I guess the release after that is going to include all the threading/hashing stuff – if we can decide on one of the proposals. 😂
Regarding jenny development: There have been enough changes in the last few weeks, imo. I want to let things settle for a while (potential bugfixes aside) and then I’m going to cut a new release.

And I guess the release after that is going to include all the threading/hashing stuff – if we can decide on one of the proposals. 😂
Regarding jenny development: There have been enough changes in the last few weeks, imo. I want to let things settle for a while (potential bugfixes aside) and then I’m going to cut a new release.

And I guess the release after that is going to include all the threading/hashing stuff – if we can decide on one of the proposals. 😂
@lyse I call upon the services of the @yarn_police to further investigate this oddness!
@lyse I call upon the services of the @yarn_police to further investigate this oddness!
@quark Oh, sure, it would be nice if edits didn't break threads. I was just pondering the circumstances under which I get annoyed about data being irrecoverably deleted or otherwise lost.
@falsifian Yeah, delete requests feel very odd.
@falsifian "*I don't really mind if the twt gets edited before I even fetch it.*", right, that's never the problem. Editing a twtxt before anyone fetches it isn't even editing, right? :-P The problem we are trying to fix is the havoc is causes editing twtxts that have already been replied to, often ad nauseam. That's the real problem.
@falsifian "*I don't really mind if the twt gets edited before I even fetch it.*", right, that's never the problem. Editing a twtxt before anyone fetches it isn't even editing, right? :-P The problem we are trying to fix is the havoc is causes editing twtxts that have already been replied to, often ad nauseam. That's the real problem.
@quark I don't really mind if the twt gets edited before I even fetch it. I think it's the idea of my computer discarding old versions it's fetched, especially if it's shown them to me, that bugs me.

But I do like @movq's suggestion on this thread that feeds could contain both the original and the edited twt. I guess it would be up to the author.
@lyse now, how am I not surprised at that reply?! Hahahahaha!
@lyse now, how am I not surprised at that reply?! Hahahahaha!
@prologic I wish that was true! But I reckon there is still heaps of old stuff out there, that was created on a Windows machine. :-D And I wouldn't be surprised if even today in that environment a new file does not make use of UTF-8.
@falsifian that would be problematic to do on a fully decentralised system. I am not disagreeing, though. That's the reason I have stopped editing twtxts. I strive to own mistakes, as minor as they might be. Now, if trail editing can be accomplished, I am all for it!
@falsifian that would be problematic to do on a fully decentralised system. I am not disagreeing, though. That's the reason I have stopped editing twtxts. I strive to own mistakes, as minor as they might be. Now, if trail editing can be accomplished, I am all for it!
@quark I'm not convinced. :-D
@quark None. I like being able to see edit history for the same reason.
@quark @movq Yep, they're all RFC3339. Obviously, +02:00 and +01:00 are best, because I use them! :-P In all seriousness, Z might be the best timezone, as it is shortest. And regarding privacy, it leaks the least information about the user's rough location. But of course, one can just look at the activity and narrow down plausible regions, so that's a weak argument.
@movq You're right! switching from zsh to bash gave me the same result zq4fgq Thanks!
@movq You're right! switching from zsh to bash gave me the same result zq4fgq Thanks!
@movq You're right! switching from zsh to bash gave me the same result zq4fgq Thanks!
@falsifian what would the difference be between an edit the changes everything on the original twtxt, and a delete?
@falsifian what would the difference be between an edit the changes everything on the original twtxt, and a delete?
@prologic Why sha1 in particular? There are known attacks on it. sha256 seems pretty widely supported if you're worried about support.
@prologic I wouldn't want my client to honour delete requests. I like my computer's memory to be better than mine, not worse, so it would bug me if I remember seeing something and my computer can't find it.
@prologic

There's a simple reason all the current hashes end in a or q: the hash is 256 bits, the base32 encoding chops that into groups of 5 bits, and 256 isn't divisible by 5. The last character of the base32 encoding just has that left-over single bit (256 mod 5 = 1).

So I agree with #3 below, but do you have a source for #1, #2 or #4? I would expect any lack of variability in any part of a hash function's output would make it more vulnerable to attacks, so designers of hash functions would want to make the whole output vary as much as possible.

Other than the divisible-by-5 thing, my current intuition is it doesn't matter what part you take.

> 1. Hash Structure: Hashes are typically designed so that their outputs have specific statistical properties. The first few characters often have more entropy or variability, meaning they are less likely to have patterns. The last characters may not maintain this randomness, especially if the encoding method has a tendency to produce less varied endings.
>
> 2. Collision Resistance: When using hashes, the goal is to minimize the risk of collisions (different inputs producing the same output). By using the first few characters, you leverage the full distribution of the hash. The last characters may not distribute in the same way, potentially increasing the likelihood of collisions.
>
> 3. Encoding Characteristics: Base32 encoding has a specific structure and padding that might influence the last characters more than the first. If the data being hashed is similar, the last characters may be more similar across different hashes.
>
> 4. Use Cases: In many applications (like generating unique identifiers), the beginning of the hash is often the most informative and varied. Relying on the end might reduce the uniqueness of generated identifiers, especially if a prefix has a specific context or meaning.=