# I am the Watcher. I am your guide through this vast new twtiverse.
# 
# Usage:
#     https://watcher.sour.is/api/plain/users              View list of users and latest twt date.
#     https://watcher.sour.is/api/plain/twt                View all twts.
#     https://watcher.sour.is/api/plain/mentions?uri=:uri  View all mentions for uri.
#     https://watcher.sour.is/api/plain/conv/:hash         View all twts for a conversation subject.
# 
# Options:
#     uri     Filter to show a specific users twts.
#     offset  Start index for quey.
#     limit   Count of items to return (going back in time).
# 
# twt range = 1 196267
# self = https://watcher.sour.is?offset=170756
# next = https://watcher.sour.is?offset=170856
# prev = https://watcher.sour.is?offset=170656
@quark I think I may have lost my original feed because it was written with the old software before Yarn social became a thing.
Irvine Running: 8.20 miles, 00:09:17 average pace, 01:16:06 duration
oh man, this was great! little to no humidity and 63F outside with hills. felt stupid easy which i think being able to breathe easily really was the main reason.
#running
Irvine Running: 8.20 miles, 00:09:17 average pace, 01:16:06 duration
oh man, this was great! little to no humidity and 63F outside with hills. felt stupid easy which i think being able to breathe easily really was the main reason.
#running
Irvine Running: 8.20 miles, 00:09:17 average pace, 01:16:06 duration
oh man, this was great! little to no humidity and 63F outside with hills. felt stupid easy which i think being able to breathe easily really was the main reason.
#running
@sorenpeter not really you're really forming a cryptographic chain of twts, that are cryptographically provable by anyone, at least in one direction ). It's called content addressing. Your propose scheme while simple doesn't do this.
@sorenpeter not really you're really forming a cryptographic chain of twts, that are cryptographically provable by anyone, at least in one direction ). It's called content addressing. Your propose scheme while simple doesn't do this.
After re-fetching feeds, the earliest twtxt I have from you is n7gavoa.
After re-fetching feeds, the earliest twtxt I have from you is n7gavoa.
@prologic, your first twtxt ever was o6dsrga, but I can't find the source for it (the raw file). I reset everything, and re-fetched fresh feeds (allegedly including archives). Where is it?
@prologic, your first twtxt ever was o6dsrga, but I can't find the source for it (the raw file). I reset everything, and re-fetched fresh feeds (allegedly including archives). Where is it?
@movq I figured it will be something like this, yet, you were able to reply just fine, and I wasn't. Looking at your twtxt.txt I see this line:


2024-09-16T17:37:14+00:00\t(#o6dsrga) @<prologic https://twtxt.net/user/prologic/twtxt.txt>

@<quark https://ferengi.one/twtxt.txt> This is what I get. 🤔


Which is using the right hash. Mine, on the other hand, when I replied to the original, old style message (Message-Id: <o6dsrga>), looks like this:


2024-09-16T16:42:27+00:00\t(#o) @<prologic https://twtxt.net/user/prologic/twtxt.txt> this was your first twtxt. Cool! :-P


What did you do to make yours work? I simply went to the oldest @prologic's entry on my Maildir, and replied to it (jenny set the reply-to hash to #o, even though the Message-Id is o6dsrga). Since jenny can't fetch archived twtxts, how could I go to re-fetch everything? And, most importantly, would re-fetching fix the Message-Id:?
@movq I figured it will be something like this, yet, you were able to reply just fine, and I wasn't. Looking at your twtxt.txt I see this line:


2024-09-16T17:37:14+00:00	(#o6dsrga) @<prologic https://twtxt.net/user/prologic/twtxt.txt>

@<quark https://ferengi.one/twtxt.txt> This is what I get. 🤔


Which is using the right hash. Mine, on the other hand, when I replied to the original, old style message (Message-Id: <o6dsrga>), looks like this:


2024-09-16T16:42:27+00:00	(#o) @<prologic https://twtxt.net/user/prologic/twtxt.txt> this was your first twtxt. Cool! :-P


What did you do to make yours work? I simply went to the oldest @prologic's entry on my Maildir, and replied to it (jenny set the reply-to hash to #o, even though the Message-Id is o6dsrga). Since jenny can't fetch archived twtxts, how could I go to re-fetch everything? And, most importantly, would re-fetching fix the Message-Id:?
[47°09′10″S, 126°43′48″W] Dosimeter overflow
****
¿Quien dijo que con las dietas se come menos? ⌘ Read more****
@quark Ahh, I see:

> Message-Id: <o6dsrga>

That’s an older format that was used before jenny version v23.04. It should look like this nowadays:

> Message-Id: <o6dsrga@twtxt>

Changelog entry from back then:


v23.04  2023-04-19
  [Changed]
  - The format of the "Message-Id" and "In-Reply-To" headers has
    changed. They now need an "@twtxt" suffix to be more compliant with
    RFC(2)822. This fixes issues when using aerc
    (https://aerc-mail.org/) as a frontend instead of mutt.

    If you want to retain compatibility with existing files in your
    maildir, you must manually add this suffix to these headers. (Or go
    ahead and re-sync everything.)


I guess I could have added backwards compatibility to the code. Maybe I’ll fix that later. 🤔
@quark Ahh, I see:

> Message-Id: <o6dsrga>

That’s an older format that was used before jenny version v23.04. It should look like this nowadays:

> Message-Id: <o6dsrga@twtxt>

Changelog entry from back then:


v23.04  2023-04-19
  [Changed]
  - The format of the "Message-Id" and "In-Reply-To" headers has
    changed. They now need an "@twtxt" suffix to be more compliant with
    RFC(2)822. This fixes issues when using aerc
    (https://aerc-mail.org/) as a frontend instead of mutt.

    If you want to retain compatibility with existing files in your
    maildir, you must manually add this suffix to these headers. (Or go
    ahead and re-sync everything.)


I guess I could have added backwards compatibility to the code. Maybe I’ll fix that later. 🤔
@quark Ahh, I see:

> Message-Id: <o6dsrga>

That’s an older format that was used before jenny version v23.04. It should look like this nowadays:

> Message-Id: <o6dsrga@twtxt>

Changelog entry from back then:


v23.04  2023-04-19
  [Changed]
  - The format of the "Message-Id" and "In-Reply-To" headers has
    changed. They now need an "@twtxt" suffix to be more compliant with
    RFC(2)822. This fixes issues when using aerc
    (https://aerc-mail.org/) as a frontend instead of mutt.

    If you want to retain compatibility with existing files in your
    maildir, you must manually add this suffix to these headers. (Or go
    ahead and re-sync everything.)


I guess I could have added backwards compatibility to the code. Maybe I’ll fix that later. 🤔
@quark Ahh, I see:

> Message-Id: <o6dsrga>

That’s an older format that was used before jenny version v23.04. It should look like this nowadays:

> Message-Id: <o6dsrga@twtxt>

Changelog entry from back then:


v23.04  2023-04-19
  [Changed]
  - The format of the "Message-Id" and "In-Reply-To" headers has
    changed. They now need an "@twtxt" suffix to be more compliant with
    RFC(2)822. This fixes issues when using aerc
    (https://aerc-mail.org/) as a frontend instead of mutt.

    If you want to retain compatibility with existing files in your
    maildir, you must manually add this suffix to these headers. (Or go
    ahead and re-sync everything.)


I guess I could have added backwards compatibility to the code. Maybe I’ll fix that later. 🤔
@prologic you will always be replying to OP - that is what the twthash is a shorthand for, it it not?!
@prologic you will always be replying to OP - that is what the twthash is a shorthand for, it it not?!
@prologic you will always be replying to OP - that is what the twthash is a shorthand for, it it not?!
@prologic you will always be replying to OP - that is what the twthash is a shorthand for, it it not?!
no my fault your client can't handle a little editing ;)
no my fault your client can't handle a little editing ;)
no my fault your client can't handle a little editing ;)
no my fault your client can't handle a little editing ;)
@movq I'm glad you like it. A mention (@<movq https://www.uninformativ.de/twtxt.txt> ) is also long, but we live with it anyway. In a way a replyto: is just a mention of a twt instead of a feed/person.
@movq I'm glad you like it. A mention (@<movq https://www.uninformativ.de/twtxt.txt> ) is also long, but we live with it anyway. In a way a replyto: is just a mention of a twt instead of a feed/person. Maybe we chould even model the syntax for replies on mentions:

(#<2024-09-17T08:39:18Z https://www.eksempel.dk/twtxt.txt>) ?!
@movq I'm glad you like it. A mention (@<movq https://www.uninformativ.de/twtxt.txt> ) is also long, but we live with it anyway. In a way a replyto: is just a mention of a twt instead of a feed/person.
@movq I'm glad you like it. A mention (@<movq https://www.uninformativ.de/twtxt.txt> ) is also long, but we live with it anyway. In a way a replyto: is just a mention of a twt instead of a feed/person. Maybe we chould even model the syntax for replies on mentions: (#<2024-09-17T08:39:18Z https://www.eksempel.dk/twtxt.txt>) ?!
@movq I'm glad you like it. A mention (@<movq https://www.uninformativ.de/twtxt.txt> ) is also long, but we live with it anyway. In a way a replyto: is just a mention of a twt instead of a feed/person. Maybe we chould even model the syntax for replies on mentions: (#<2024-09-17T08:39:18Z https://www.eksempel.dk/twtxt.txt>) ?!
@movq I'm glad you like it. A mention (@<movq https://www.uninformativ.de/twtxt.txt> ) is also long, but we live with it anyway. In a way a replyto: is just a mention of a twt instead of a feed/person. Maybe we chould even model the syntax for replies on mentions: (#<2024-09-17T08:39:18Z https://www.eksempel.dk/twtxt.txt>) ?!
@movq I'm glad you like it. A mention (@<movq https://www.uninformativ.de/twtxt.txt> ) is also long, but we live with it anyway. In a way a replyto: is just a mention of a twt instead of a feed/person. Maybe we chould even model the syntax for replies on mentions: (#<2024-09-17T08:39:18Z https://www.eksempel.dk/twtxt.txt>) ?!
@lyse and @movq and possibly @aelaraji and even @cuaxolotl -- I'm very curious to understand and hear thoughts, pros and cons or other feelings about introducing the notation of a feed's identify using cryptography? If we were to keep things simple, and use what's commonly available, for example SSH ED25519 keys? using the ssh-keygen -Y sign or ssh-keygen -Y verify tools already available? Maybe in combination with @xuu 's idea of generating a random unique ID for your feed, say # id = and signing it with your ED25519 key? 🔑
@lyse and @movq and possibly @aelaraji and even @cuaxolotl -- I'm very curious to understand and hear thoughts, pros and cons or other feelings about introducing the notation of a feed's identify using cryptography? If we were to keep things simple, and use what's commonly available, for example SSH ED25519 keys? using the ssh-keygen -Y sign or ssh-keygen -Y verify tools already available? Maybe in combination with @xuu 's idea of generating a random unique ID for your feed, say # id = and signing it with your ED25519 key? 🔑
@falsifian You mean the idea of being able to inline # url = changes in your feed?

> Whatever gets used, it would be nice to be able to rotate identities. I like @lyse’s idea for that.
@falsifian You mean the idea of being able to inline # url = changes in your feed?

> Whatever gets used, it would be nice to be able to rotate identities. I like @lyse’s idea for that.
[47°09′25″S, 126°43′13″W] Raw reading: 0x66E928F1, offset +/-5
[47°09′11″S, 126°43′57″W] Reading: 0.63000 PPM
@mckinley Yes, changing domains is be a problem if you tie your identity to an https url. But I also worry about being stuck with a key I can't rotate. Whatever gets used, it would be nice to be able to rotate identities. I like @lyse's idea for that.
This scheme also only support threading off a specific Twt of someone's feed. What if you're not replying to anyone in particular?
This scheme also only support threading off a specific Twt of someone's feed. What if you're not replying to anyone in particular?
Noting that this scheme cannot support disjoint threads that should be merged together once either party discovers each other 😅
Noting that this scheme cannot support disjoint threads that should be merged together once either party discovers each other 😅
@movq It does now as of several weeks ago or so 👌
@movq It does now as of several weeks ago or so 👌
@lyse Yea I think your idea of inclining url changes in the feed works perfectly as long as folks remember to do so I guess? 🤔
@lyse Yea I think your idea of inclining url changes in the feed works perfectly as long as folks remember to do so I guess? 🤔
@bender Seems to have used the hash correctly here 🤣
@bender Seems to have used the hash correctly here 🤣
Holy shot! what an old thread 🤣
Holy shot! what an old thread 🤣
@cuaxolotl Gitea is plenty fast for me 👌 Even with a SQLite database 🤣
@cuaxolotl Gitea is plenty fast for me 👌 Even with a SQLite database 🤣
I dunno whether any of this is actually true 🤷‍♂️ The LLM 🤖could have made (hallucinated) this shit up 🤣
I dunno whether any of this is actually true 🤷‍♂️ The LLM 🤖could have made (hallucinated) this shit up 🤣
WiscKey's approach to handling key-value pairs in SSDs offers several advantages:

1. It minimizes I/O amplification by separating keys and values, allowing for more efficient storage and retrieval.
2. The design leverages the SSD's performance characteristics, utilizing sequential writes and parallel random reads to enhance throughput and reduce latency.
3. WiscKey maintains excellent insert and lookup performance while improving SSD lifespan by reducing the number of erase cycles required.
WiscKey's approach to handling key-value pairs in SSDs offers several advantages:

1. It minimizes I/O amplification by separating keys and values, allowing for more efficient storage and retrieval.
2. The design leverages the SSD's performance characteristics, utilizing sequential writes and parallel random reads to enhance throughput and reduce latency.
3. WiscKey maintains excellent insert and lookup performance while improving SSD lifespan by reducing the number of erase cycles required.
WiscKey minimizes CPU usage compared to LevelDB by eliminating the log file, which reduces the CPU cost associated with encoding and writing key-value pairs. While LevelDB has higher CPU usage due to its single writer protocol and background compaction using one thread, WiscKey's architecture allows for garbage collection to be decoupled and performed later, minimizing its impact on foreground performance. Consequently, WiscKey generally exhibits lower CPU usage during workloads, except when utilizing multiple background threads for prefetching.
WiscKey minimizes CPU usage compared to LevelDB by eliminating the log file, which reduces the CPU cost associated with encoding and writing key-value pairs. While LevelDB has higher CPU usage due to its single writer protocol and background compaction using one thread, WiscKey's architecture allows for garbage collection to be decoupled and performed later, minimizing its impact on foreground performance. Consequently, WiscKey generally exhibits lower CPU usage during workloads, except when utilizing multiple background threads for prefetching.
The four critical ideas in the design of WiscKey are:

1. Separation of keys from values, with only keys stored in the LSM-tree and values in a separate log file.
2. Utilization of the parallel random-read characteristic of SSD devices to handle unsorted values during range queries.
3. Implementation of unique crash-consistency techniques to manage the value log efficiently.
4. Optimization of performance by removing the LSM-tree log without sacrificing consistency, reducing system-call overhead from small writes.
The four critical ideas in the design of WiscKey are:

1. Separation of keys from values, with only keys stored in the LSM-tree and values in a separate log file.
2. Utilization of the parallel random-read characteristic of SSD devices to handle unsorted values during range queries.
3. Implementation of unique crash-consistency techniques to manage the value log efficiently.
4. Optimization of performance by removing the LSM-tree log without sacrificing consistency, reducing system-call overhead from small writes.
# Summary of WiscKey: Separating Keys from Values in SSD-Conscious Storage

- Authors: Lanyue Lu, Thanumalayan Sankaranarayana Pillai, Andrea C. Arpaci-Dusseau, Remzi H. Arpaci-Dusseau
- Conference: 14th USENIX Conference on File and Storage Technologies (FAST '16)
- Key Concept: WiscKey is a key-value store that separates keys from values to minimize I/O amplification, optimizing performance for SSDs.
- Performance: WiscKey outperforms LevelDB and RocksDB in various workloads, achieving up to 111× faster loading and improved random lookup speeds.
- Design Goals: Focus on low write/read amplification, SSD optimization, and support for modern features like range queries.
# Summary of WiscKey: Separating Keys from Values in SSD-Conscious Storage

- Authors: Lanyue Lu, Thanumalayan Sankaranarayana Pillai, Andrea C. Arpaci-Dusseau, Remzi H. Arpaci-Dusseau
- Conference: 14th USENIX Conference on File and Storage Technologies (FAST '16)
- Key Concept: WiscKey is a key-value store that separates keys from values to minimize I/O amplification, optimizing performance for SSDs.
- Performance: WiscKey outperforms LevelDB and RocksDB in various workloads, achieving up to 111× faster loading and improved random lookup speeds.
- Design Goals: Focus on low write/read amplification, SSD optimization, and support for modern features like range queries.
🧮 USERS:1 FEEDS:2 TWTS:1095 ARCHIVED:78843 CACHE:2434 FOLLOWERS:17 FOLLOWING:14
i have been using grapheneos for a long time (maybe 7 years now?) and i would fully reccomend it to anyone who is OK with buying a new pixel device to run it on. the install instructions are really easy to follow and can be executed on any device that supports WebUSB https://grapheneos.org/install/web
afaik nobody has done this, but i really need some numbers that can indicate the relative performance of various git servers (cgit, gitea, gitlab) on comparable hardware. cgit claims to be hyperfast, but what does that mean in practice?
the first version of the minibase flake cdn is online as of today. more details on how this is populated and funded will be coming along soon. currently minibase includes a very small selection of packages, but you can install them using the experimental flakes feature today! https://src.cyb.red/
@movq the thing is, the twtxt is in Maildir. When I reply to it, it doesn’t use the existing hash.
so this path *has been* trod and its bad lmao https://web.archive.org/web/20230926192451/https://gemini.spam.works/~emery/sigil-report.gmi
so it looks like genode has taken some inspiration from nix.. that's a rabbit-hole for another time https://genode.org/documentation/developer-resources/package_management
@aelaraji 🥳
@aelaraji 🥳
@aelaraji 🥳
@aelaraji 🥳
tw.txt is fun because its a hackers distributed system. simple, with lots of sharp edges to keep things interesting. i hope we don't go chasing mass-appeal because it'll make writing my own crappy tw.txt services un-fun and pointless since i can barely keep up with the current somewhat glacial pace of development.
it would be interesting to build a new nixpkgs based on a totally different base system. like, genode (sel4+) that can take the interesting security properties of nix and apply them to a non-posix secure-by-construction type of OS
@bender Ah, haha, --fetch-context doesn’t go back into archived feeds … 🤦
@bender Ah, haha, --fetch-context doesn’t go back into archived feeds … 🤦
@bender Ah, haha, --fetch-context doesn’t go back into archived feeds … 🤦
@bender Ah, haha, --fetch-context doesn’t go back into archived feeds … 🤦
[47°09′36″S, 126°43′20″W] Transponder still failing -- switching to analog communication
Thank you @movq Things are working again!! 🙏
Thank you @movq Things are working again!! 🙏
Thank you @movq Things are working again!! 🙏
Trying to fetch the original (highlighting yours) with jenny renders this:

@movq the puzzle thickens:

@prologic

@quark This is what I get. 🤔
@prologic

@quark This is what I get. 🤔
@prologic

@quark This is what I get. 🤔
@prologic

@quark This is what I get. 🤔
@lyse Argh. Yeah. Well. 🤦
@lyse Argh. Yeah. Well. 🤦
@lyse Argh. Yeah. Well. 🤦
@lyse Argh. Yeah. Well. 🤦
@movq I'd guess the same goes for all twtxt.social feeds... I can't see bender's archived twts either, didn't check for the others.
@movq I'd guess the same goes for all twtxt.social feeds... I can't see bender's archived twts either, didn't check for the others.
@movq I'd guess the same goes for all twtxt.social feeds... I can't see bender's archived twts either, didn't check for the others.
Found it: https://github.com/buckket/twtxt/issues/157
@prologic Yeah, but I reckon we can kill both birds with one stone. If we change it to support edits, it should be fairly easy to also tweak it to support feed URL changes. Like outlined in my first reply: https://twtxt.net/twt/n4omfvq The URL part sounds way easier to me. :-)