# I am the Watcher. I am your guide through this vast new twtiverse.
#
# Usage:
# https://watcher.sour.is/api/plain/users View list of users and latest twt date.
# https://watcher.sour.is/api/plain/twt View all twts.
# https://watcher.sour.is/api/plain/mentions?uri=:uri View all mentions for uri.
# https://watcher.sour.is/api/plain/conv/:hash View all twts for a conversation subject.
#
# Options:
# uri Filter to show a specific users twts.
# offset Start index for quey.
# limit Count of items to return (going back in time).
#
# twt range = 1 17
# self = https://watcher.sour.is/conv/tbcptaa
I finally decided to do a few experiments with yarnd to see how many things would break and how many assumptions there are around the idea of "Content Addressing"; here's where I'm at so far:
- What breaks
Basically I'm at a point where spending time on this is going to provide very little value, there are assumptions made in the lextwt parser, assumptions made in yarnd, assumptions in the way storage is done and the way threading works and things are looked up. There are far reaching implications to changing the way Twts are identified here to be "location addressed" that I'm quite worried about the amount of effort would be required to change yarnd here.
I finally decided to do a few experiments with yarnd to see how many things would break and how many assumptions there are around the idea of "Content Addressing"; here's where I'm at so far:
- What breaks
Basically I'm at a point where spending time on this is going to provide very little value, there are assumptions made in the lextwt parser, assumptions made in yarnd, assumptions in the way storage is done and the way threading works and things are looked up. There are far reaching implications to changing the way Twts are identified here to be "location addressed" that I'm quite worried about the amount of effort would be required to change yarnd here.
The three things we briefly talk about tonight (your morning), so that I don’t forget:
1. Add the ability to allow feed address changes.
2. Increase hash from 7 to 11, and/or change the hashing algorithm to something else, better.
3. Implement movq (I simply can’t mention while on mobile) second option (the one you like, which maintains content addressing).
@prologic Yeah. I guess no matter what we do, either one of us has to do a lot of work on his respective software. 😅 The (absolute) impact on Yarn is probably bigger than that on jenny, simply because Yarn is a much larger piece of software to begin with.
I’ll begin experimenting in jenny with Update Commands and see how bad it gets. I’ve already determined that my storage model wouldn’t work anymore. It’s all *doable*, but, like you, I’m not that happy with all the consequences. 😂
@prologic Yeah. I guess no matter what we do, either one of us has to do a lot of work on his respective software. 😅 The (absolute) impact on Yarn is probably bigger than that on jenny, simply because Yarn is a much larger piece of software to begin with.
I’ll begin experimenting in jenny with Update Commands and see how bad it gets. I’ve already determined that my storage model wouldn’t work anymore. It’s all *doable*, but, like you, I’m not that happy with all the consequences. 😂
@prologic Yeah. I guess no matter what we do, either one of us has to do a lot of work on his respective software. 😅 The (absolute) impact on Yarn is probably bigger than that on jenny, simply because Yarn is a much larger piece of software to begin with.
I’ll begin experimenting in jenny with Update Commands and see how bad it gets. I’ve already determined that my storage model wouldn’t work anymore. It’s all *doable*, but, like you, I’m not that happy with all the consequences. 😂
@prologic Yeah. I guess no matter what we do, either one of us has to do a lot of work on his respective software. 😅 The (absolute) impact on Yarn is probably bigger than that on jenny, simply because Yarn is a much larger piece of software to begin with.
I’ll begin experimenting in jenny with Update Commands and see how bad it gets. I’ve already determined that my storage model wouldn’t work anymore. It’s all *doable*, but, like you, I’m not that happy with all the consequences. 😂
@movq Yeah I think what I'm proposing here is a more pragmatic approach to improvements that will last much longer than our first interaction (~4 years and going strong, but running into minor issues with edit/identify and some collssions_). This scope of changes is much easier to implement for yarnd and I suspect jenny too. and as indicated in here quite easy to have a reference implementation written in Bash with standard UNIX tools.~_
@movq Yeah I think what I'm proposing here is a more pragmatic approach to improvements that will last much longer than our first interaction (~4 years and going strong, but running into minor issues with edit/identify and some collssions_). This scope of changes is much easier to implement for yarnd and I suspect jenny too. and as indicated in here quite easy to have a reference implementation written in Bash with standard UNIX tools.~_
@prologic
> This scope of changes is much easier to implement for yarnd and *I suspect jenny too.*
No, (edit:) is a lot of work for jenny and also adds a lot of overhead.
Right now, jenny itself has no idea which twt hashes are present on which feed, because it doesn’t need to. This information only exists in the mail files. This means I can’t check if an (edit:) operation is legal. jenny will have to get an sqlite database (or read/parse/write 1-2 MB of JSON on every invocation, which isn’t great either).
I’ve already spent several hours this morning rewriting the feed fetching/parsing code in an effort to pave the way to even be *able* to support any of this. I’ll spare you the details. Until now, twts were individual items in a feed, they could be processed in any order and they didn’t reference each other *from jennys point of view*. A lot of the heavy lifting happens in the mail client.
Honestly, the database thing bugs me the most. The whole concept of “just create some mail files” doesn’t really work anymore. I now have to duplicate state between the mail files and an internal database. This is a big “meh”.
Of course, if we were to switch to location-based addressing, then *you* would have to do a lot of work. There’s no easy way out.
Maybe I could say jenny does not support (edit:) for now. That’s the good thing about this proposal: I don’t have to implement it right away. Users will see spurious twts (or I could hide them as a workaround) and they won’t see twt updates, but nothing will break.
@prologic
> This scope of changes is much easier to implement for yarnd and *I suspect jenny too.*
No, (edit:) is a lot of work for jenny and also adds a lot of overhead.
Right now, jenny itself has no idea which twt hashes are present on which feed, because it doesn’t need to. This information only exists in the mail files. This means I can’t check if an (edit:) operation is legal. jenny will have to get an sqlite database (or read/parse/write 1-2 MB of JSON on every invocation, which isn’t great either).
I’ve already spent several hours this morning rewriting the feed fetching/parsing code in an effort to pave the way to even be *able* to support any of this. I’ll spare you the details. Until now, twts were individual items in a feed, they could be processed in any order and they didn’t reference each other *from jennys point of view*. A lot of the heavy lifting happens in the mail client.
Honestly, the database thing bugs me the most. The whole concept of “just create some mail files” doesn’t really work anymore. I now have to duplicate state between the mail files and an internal database. This is a big “meh”.
Of course, if we were to switch to location-based addressing, then *you* would have to do a lot of work. There’s no easy way out.
Maybe I could say jenny does not support (edit:) for now. That’s the good thing about this proposal: I don’t have to implement it right away. Users will see spurious twts (or I could hide them as a workaround) and they won’t see twt updates, but nothing will break.
@prologic
> This scope of changes is much easier to implement for yarnd and *I suspect jenny too.*
No, (edit:) is a lot of work for jenny and also adds a lot of overhead.
Right now, jenny itself has no idea which twt hashes are present on which feed, because it doesn’t need to. This information only exists in the mail files. This means I can’t check if an (edit:) operation is legal. jenny will have to get an sqlite database (or read/parse/write 1-2 MB of JSON on every invocation, which isn’t great either).
I’ve already spent several hours this morning rewriting the feed fetching/parsing code in an effort to pave the way to even be *able* to support any of this. I’ll spare you the details. Until now, twts were individual items in a feed, they could be processed in any order and they didn’t reference each other *from jennys point of view*. A lot of the heavy lifting happens in the mail client.
Honestly, the database thing bugs me the most. The whole concept of “just create some mail files” doesn’t really work anymore. I now have to duplicate state between the mail files and an internal database. This is a big “meh”.
Of course, if we were to switch to location-based addressing, then *you* would have to do a lot of work. There’s no easy way out.
Maybe I could say jenny does not support (edit:) for now. That’s the good thing about this proposal: I don’t have to implement it right away. Users will see spurious twts (or I could hide them as a workaround) and they won’t see twt updates, but nothing will break.
@prologic
> This scope of changes is much easier to implement for yarnd and *I suspect jenny too.*
No, (edit:) is a lot of work for jenny and also adds a lot of overhead.
Right now, jenny itself has no idea which twt hashes are present on which feed, because it doesn’t need to. This information only exists in the mail files. This means I can’t check if an (edit:) operation is legal. jenny will have to get an sqlite database (or read/parse/write 1-2 MB of JSON on every invocation, which isn’t great either).
I’ve already spent several hours this morning rewriting the feed fetching/parsing code in an effort to pave the way to even be *able* to support any of this. I’ll spare you the details. Until now, twts were individual items in a feed, they could be processed in any order and they didn’t reference each other *from jennys point of view*. A lot of the heavy lifting happens in the mail client.
Honestly, the database thing bugs me the most. The whole concept of “just create some mail files” doesn’t really work anymore. I now have to duplicate state between the mail files and an internal database. This is a big “meh”.
Of course, if we were to switch to location-based addressing, then *you* would have to do a lot of work. There’s no easy way out.
Maybe I could say jenny does not support (edit:) for now. That’s the good thing about this proposal: I don’t have to implement it right away. Users will see spurious twts (or I could hide them as a workaround) and they won’t see twt updates, but nothing will break.
@movq Makes sense 👌 I think it's fair to implement any spec changes incrementaly for sure 👌
And yea since yarnd has a store it's a bit easier to support edit / delete actions 😅
@movq Makes sense 👌 I think it's fair to implement any spec changes incrementaly for sure 👌
And yea since yarnd has a store it's a bit easier to support edit / delete actions 😅