# I am the Watcher. I am your guide through this vast new twtiverse.
# 
# Usage:
#     https://watcher.sour.is/api/plain/users              View list of users and latest twt date.
#     https://watcher.sour.is/api/plain/twt                View all twts.
#     https://watcher.sour.is/api/plain/mentions?uri=:uri  View all mentions for uri.
#     https://watcher.sour.is/api/plain/conv/:hash         View all twts for a conversation subject.
# 
# Options:
#     uri     Filter to show a specific users twts.
#     offset  Start index for quey.
#     limit   Count of items to return (going back in time).
# 
# twt range = 1 196266
# self = https://watcher.sour.is?offset=172156
# next = https://watcher.sour.is?offset=172256
# prev = https://watcher.sour.is?offset=172056
[47°09′18″S, 126°43′07″W] Reading: 1.32000 PPM
****
El mundo se acaba:

https://es.euronews.com/business/2024/09/24/el-precio-del-cafe-sube-mientras-las-catastrofes-naturales-diezman-las-cosechasRead more****
@anth Thank you I'll have a read 👌
@anth Thank you I'll have a read 👌
This is only first draft quality, but I made some notes on the #twtxt v2 proposal. http://a.9srv.net/b/2024-09-25
This is only first draft quality, but I made some notes on the #twtxt v2 proposal. http://a.9srv.net/b/2024-09-25
[47°09′32″S, 126°43′12″W] Reading: 0.44000 PPM
@sorenpeter i'm just saying that your argument, better support better clients and worrying less about the actual underlying raw Twtxt feed. so the simplicity argument is a bit weaker here.
@sorenpeter i'm just saying that your argument, better support better clients and worrying less about the actual underlying raw Twtxt feed. so the simplicity argument is a bit weaker here.
[47°09′46″S, 126°43′45″W] --white noise--
https://jacobin.com/2024/09/rashida-tlaib-antisemitism-palestinians-media
Based in Barcelona rigth now! :P
Based in Barcelona rigth now! :P
why can we both have a format that you can write by hand and better clients?
why can we both have a format that you can write by hand and better clients?
why can we both have a format that you can write by hand and better clients?
why can we both have a format that you can write by hand and better clients?
@sorenpeter This is an argument for better clients really and less worry about the "transport" -- the raw Twtxt feed file.
@sorenpeter This is an argument for better clients really and less worry about the "transport" -- the raw Twtxt feed file.
@sorenpeter CPU cost of calculating hashes are negligible
@sorenpeter CPU cost of calculating hashes are negligible
🧮 USERS:1 FEEDS:2 TWTS:1103 ARCHIVED:79345 CACHE:2623 FOLLOWERS:17 FOLLOWING:14
Actually, I stand corrected. It shows a reply to a twtxt both having the same date/timestamp.
No, json is overhead. I love twtxt for simplicity where blog is just text file and not several json files where fields are repeated...
yes that works
yes that works
yes that works
yes that works
@sorenpeter not even this: https://twtxt.net/media/AzUmzTN5YEJdt4VPeeprjB.png?full=1
@sorenpeter this will show broken, because you are hellbent on editing twtxts, aren't you? :-D
@sorenpeter this will show broken, because you are hellbent on editing twtxts, aren't you? :-D
@bender doesn't work if you are not logged in at twtxt.net. Even if I use the image URL from the raw twtxt it's still a blur.
@sorenpeter right click, open on new tab, zoom if needed. If shows how the location addressing increases the size on each twtxt.
[47°09′26″S, 126°43′08″W] Transfer aborted
What does this screenshot show? The resolution it too low for reading the text...
(#2024-09-24T12:53:35Z) What does this screenshot show? The resolution it too low for reading the text...
(#2024-09-24T12:53:35Z) What does this screenshot show? The resolution it too low for reading the text...
(#2024-09-24T12:53:35Z) What does this screenshot show? The resolution it too low for reading the text...
(#2024-09-24T12:53:35Z) What does this screenshot show? The resolution it too low for reading the text...
(#2024-09-24T12:45:54Z) @prologic I'm not really buying this one about readability. It's easy to recognize that this is a URL and a date, so you skim over it like you would we mentions and markdown links and images. If you are not suppose to read the raw file, then we might a well jam everything into JSON like mastodon
(#2024-09-24T12:45:54Z) @prologic I'm not really buying this one about readability. It's easy to recognize that this is a URL and a date, so you skim over it like you would we mentions and markdown links and images. If you are not suppose to read the raw file, then we might a well jam everything into JSON like mastodon
(#2024-09-24T12:45:54Z) @prologic I'm not really buying this one about readability. It's easy to recognize that this is a URL and a date, so you skim over it like you would we mentions and markdown links and images. If you are not suppose to read the raw file, then we might a well jam everything into JSON like mastodon
(#2024-09-24T12:45:54Z) @prologic I'm not really buying this one about readability. It's easy to recognize that this is a URL and a date, so you skim over it like you would we mentions and markdown links and images. If you are not suppose to read the raw file, then we might a well jam everything into JSON like mastodon
@prologic I'm not really buying this one about readability. It's easy to recognize that this is a URL and a date, so you skim over it like you would we mentions and markdown links and images. If you are not suppose to read the raw file, then we might a well jam everything into JSON like mastodon
(#2024-09-24T12:44:35Z) There is a increase in space/memory for sure. But calculating the hashes also takes up CPU. I'm not good with that kind of math, but it's a tradeoff either way.
(#2024-09-24T12:44:35Z)\tThere is a increase in space/memory for sure. But calculating the hashes also takes up CPU. I'm not good with that kind of math, but it's a tradeoff either way.
(#2024-09-24T12:44:35Z) There is a increase in space/memory for sure. But calculating the hashes also takes up CPU. I'm not good with that kind of math, but it's a tradeoff either way.
There is a increase in space/memory for sure. But calculating the hashes also takes up CPU. I'm not good with that kind of math, but it's a tradeoff either way.
(#2024-09-24T12:44:35Z) There is a increase in space/memory for sure. But calculating the hashes also takes up CPU. I'm not good with that kind of math, but it's a tradeoff either way.
(#2024-09-24T12:44:35Z) There is a increase in space/memory for sure. But calculating the hashes also takes up CPU. I'm not good with that kind of math, but it's a tradeoff either way.
(#2024-09-24T12:39:32Z) @prologic It might be simple for you to run echo -e "\\t\\t" | sha256sum | base64, but for people who are not comfortable in a terminal and got their dev env set up, then that is magic, compared to the simplicity of just copy/pasting what you see in a textfile into another textfile -- Basically what @movq also said. I'm also on team extreme minimalism, otherwise we could just use mastodon etc. Replacing line-breaks with a tab would also make it easier to handwrite your twtxt. You don't have to hardwrite it, but at least you should have the option to. Just as i do with all my HTML and CSS.
(#2024-09-24T12:39:32Z) @prologic It might be simple for you to run echo -e "\t\t" | sha256sum | base64, but for people who are not comfortable in a terminal and got their dev env set up, then that is magic, compared to the simplicity of just copy/pasting what you see in a textfile into another textfile -- Basically what @movq also said. I'm also on team extreme minimalism, otherwise we could just use mastodon etc. Replacing line-breaks with a tab would also make it easier to handwrite your twtxt. You don't have to hardwrite it, but at least you should have the option to. Just as i do with all my HTML and CSS.
(#2024-09-24T12:39:32Z) @prologic) @prologic It might be simple for you to run echo -e "\\t\\t" | sha256sum | base64, but for people who are not comfortable in a terminal and got their dev env set up, then that is magic, compared to the simplicity of just copy/pasting what you see in a textfile into another textfile -- Basically what @movq also said. I'm also on team extreme minimalism, otherwise we could just use mastodon etc. Replacing line-breaks with a tab would also make it easier to handwrite your twtxt. You don't have to hardwrite it, but at least you should have the option to. Just as i do with all my HTML and CSS.
(#2024-09-24T12:39:32Z) @prologic It might be simple for you to run echo -e "\t\t" | sha256sum | base64, but for people who are not comfortable in a terminal and got their dev env set up, then that is magic, compared to the simplicity of just copy/pasting what you see in a textfile into another textfile -- Basically what @movq also said. I'm also on team extreme minimalism, otherwise we could just use mastodon etc. Replacing line-breaks with a tab would also make it easier to handwrite your twtxt. You don't have to hardwrite it, but at least you should have the option to. Just as i do with all my HTML and CSS.
(#2024-09-24T12:39:32Z) @prologic It might be simple for you to run echo -e "\t\t" | sha256sum | base64, but for people who are not comfortable in a terminal and got their dev env set up, then that is magic, compared to the simplicity of just copy/pasting what you see in a textfile into another textfile -- Basically what @movq also said. I'm also on team extreme minimalism, otherwise we could just use mastodon etc. Replacing line-breaks with a tab would also make it easier to handwrite your twtxt. You don't have to hardwrite it, but at least you should have the option to. Just as i do with all my HTML and CSS.
(#2024-09-24T12:39:32Z) @prologic It might be simple for you to run echo -e "\t\t" | sha256sum | base64, but for people who are not comfortable in a terminal and got their dev env set up, then that is magic, compared to the simplicity of just copy/pasting what you see in a textfile into another textfile -- Basically what @movq also said. I'm also on team extreme minimalism, otherwise we could just use mastodon etc. Replacing line-breaks with a tab would also make it easier to handwrite your twtxt. You don't have to hardwrite it, but at least you should have the option to. Just as i do with all my HTML and CSS.
@prologic It might be simple for you to run echo -e "\\t\\t" | sha256sum | base64, but for people who are not comfortable in a terminal and got their dev env set up, then that is magic, compared to the simplicity of just copy/pasting what you see in a textfile into another textfile -- Basically what @movq also said. I'm also on team extreme minimalism, otherwise we could just use mastodon etc. Replacing line-breaks with a tab would also make it easier to handwrite your twtxt. You don't have to hardwrite it, but at least you should have the option to. Just as i do with all my HTML and CSS.
(#2024-09-24T12:34:31Z) WebMentions does would work if we agreed to implement it correctly. I never figured out how yarnd's WebMentions work, so I decide to make my own, which I'm the only one using...

I had a look at WebSub, witch looks way more complex than WebMentions, and seem to need a lot more overhead. We don't need near realtime. We just need a way to notify someone that someone they don't know about mentioned or replied to their post.
(#2024-09-24T12:34:31Z) WebMentions does would work if we agreed to implement it correctly. I never figured out how yarnd's WebMentions work, so I decide to make my own, which I'm the only one using...

I had a look at WebSub, witch looks way more complex than WebMentions, and seem to need a lot more overhead. We don't need near realtime. We just need a way to notify someone that someone they don't know about mentioned or replied to their post.
WebMentions does would work if we decince to implement it correctly. I never figured out how yarnd's WebMentions work, so I decide to make my own, with I'm the only one using...

I had a look at WebSub, witch is way more complex than WebMentions, and seem to need a lot more overhead. We don't need near realtime. We just need a way to notify someone that someone they don't know about mentioned or replied to their post.
WebMentions does would work if we agreed to implement it correctly. I never figured out how yarnd's WebMentions work, so I decide to make my own, which I'm the only one using...

I had a look at WebSub, witch looks way more complex than WebMentions, and seem to need a lot more overhead. We don't need near realtime. We just need a way to notify someone that someone they don't know about mentioned or replied to their post.
WebMentions does would work if we agreed to implement it correctly. I never figured out how yarnd's WebMentions work, so I decide to make my own, with I'm the only one using...

I had a look at WebSub, witch is way more complex than WebMentions, and seem to need a lot more overhead. We don't need near realtime. We just need a way to notify someone that someone they don't know about mentioned or replied to their post.
(#2024-09-24T12:34:31Z) WebMentions does would work if we agreed to implement it correctly. I never figured out how yarnd's WebMentions work, so I decide to make my own, which I'm the only one using...

I had a look at WebSub, witch looks way more complex than WebMentions, and seem to need a lot more overhead. We don't need near realtime. We just need a way to notify someone that someone they don't know about mentioned or replied to their post.
(#2024-09-24T12:34:31Z) WebMentions does would work if we agreed to implement it correctly. I never figured out how yarnd's WebMentions work, so I decide to make my own, which I'm the only one using...

I had a look at WebSub, witch looks way more complex than WebMentions, and seem to need a lot more overhead. We don't need near realtime. We just need a way to notify someone that someone they don't know about mentioned or replied to their post.
Finally! After hours I figured out my problems.

1. The clever Go code to filter out completely read conversations got in the way with the filtering now moved into SQL. Yeah, I also did not think that this could ever conflict. But it did. Initializing the completeConversationRead flag to true got now in my way, this caused a conversation to be removed. Simply deleting all the code around that flag solved it.

2. Generation of missing conversation roots in SQL simply used the oldest (smallest) timestamp from any direct reply in the tree. To find the missing roots I grouped by subject and then aggregated using min(created_at). Now that I optimized this to only take unread messages into consideration in the first place, I do not necessarily see the smallest child anymore (when it's already read), so the timestamp is then moved forward to the next oldest unread reply. As I do not care too much about an accurate timestamp for something made up, I just adjusted my test case accordingly. Good enough for me. :-)

It's an interesting experiment with SQLite so far. I certainly did learn a few things along the way. Mission accomplished.
@lyse aha! Just like Bash would do. I figure -- is way too broad to start an autocomplete. Got to feed it a bit more! :-D
@lyse aha! Just like Bash would do. I figure -- is way too broad to start an autocomplete. Got to feed it a bit more! :-D
@lyse Haha 😝
@lyse Haha 😝
@prologic Ta! Somehow, my unit tests break, though. Running the same query manually looks like it's producing a plausible looking result, though. I do not understand it.
@david As far as I understand it, auto-completion *is* working, that's the issue. :-D Instead of spamming the terminal with bucketloads of possibilities, zsh's auto-complete is nice enough to ask whether to proceed or not.
@david Weird, I always thought that rsync automatically resumes the up- or download when aborted. But the manual indicates otherwise with --partial (-P is --partial --progress).
[47°09′05″S, 126°43′16″W] Carrier too weak
@prologic I reckon, I could just hash the subject internally to get a shorter version.
lol, this flags looks like russian name
@lyse that -P is a life saver when running rsync over spotty connections. In my very illiterate opinion, it should always be a default.
@lyse that -P is a life saver when running rsync over spotty connections. In my very illiterate opinion, it should always be a default.
@lyse Now increase the indexes on the Twt Subject form 7 bytes to 64 bytes 😈
@lyse Now increase the indexes on the Twt Subject form 7 bytes to 64 bytes 😈
@lyse Congrats 🙌
@lyse Congrats 🙌
Hmm this question has a leading "Yes" in favor of so far with 13 votes:

> Should we formally support edit and deletion requests?


Thanks y'all for voting (_it's all anonymous so I have no idea who's voted for what!_)

If you haven't already had your say, please do so here: http://polljunkie.com/poll/xdgjib/twtxt-v2 -- This is my feeble attempt at trying to ascertain the voice of the greater community with ideas of a Twtxt v2 specification (_which I'm hoping will just be an improved specification of what we largely have already built to date with some small but important improvements 🤞_)_
Hmm this question has a leading "Yes" in favor of so far with 13 votes:

> Should we formally support edit and deletion requests?


Thanks y'all for voting (_it's all anonymous so I have no idea who's voted for what!_)

If you haven't already had your say, please do so here: http://polljunkie.com/poll/xdgjib/twtxt-v2 -- This is my feeble attempt at trying to ascertain the voice of the greater community with ideas of a Twtxt v2 specification (_which I'm hoping will just be an improved specification of what we largely have already built to date with some small but important improvements 🤞_)_
Three feeds (prologic, movq and mine) and my database is already 1.3 MiB in size. Hmm. I actually got the read filter working. More on that later after polishing it.
Starting a couple of new projects (_geez where do I find the time?!_):

HomeTunnel:
> HomeTunnel is a self-hosted solution that combines secure tunneling, proxying, and automation to create your own private cloud. Utilizing Wireguard for VPN, Caddy for reverse proxying, and Traefik for service routing, HomeTunnel allows you to securely expose your home network services (such as Gitea, Poste.io, etc.) to the Internet. With seamless automation and on-demand TLS, HomeTunnel gives you the power to manage your own cloud-like environment with the control and privacy of self-hosting.

CraneOps:
> craneops is an open-source operator framework, written in Go, that allows self-hosters to automate the deployment and management of infrastructure and applications. Inspired by Kubernetes operators, CraneOps uses declarative YAML Custom Resource Definitions (CRDs) to manage Docker Swarm deployments on Proxmox VE clusters.
Starting a couple of new projects (_geez where do I find the time?!_):

HomeTunnel:
> HomeTunnel is a self-hosted solution that combines secure tunneling, proxying, and automation to create your own private cloud. Utilizing Wireguard for VPN, Caddy for reverse proxying, and Traefik for service routing, HomeTunnel allows you to securely expose your home network services (such as Gitea, Poste.io, etc.) to the Internet. With seamless automation and on-demand TLS, HomeTunnel gives you the power to manage your own cloud-like environment with the control and privacy of self-hosting.

CraneOps:
> craneops is an open-source operator framework, written in Go, that allows self-hosters to automate the deployment and management of infrastructure and applications. Inspired by Kubernetes operators, CraneOps uses declarative YAML Custom Resource Definitions (CRDs) to manage Docker Swarm deployments on Proxmox VE clusters.
@aelaraji I think all replies are missing the fact that your auto-completion isn't working. LOL. Or did I misunderstood?
@aelaraji I think all replies are missing the fact that your auto-completion isn't working. LOL. Or did I misunderstood?
@aelaraji @mckinley rsync -avzr with an optional --progress is what I always use. Ah, I could use the shorter -P, thanks @movq.
I think that's one of the worst aspects of the proposed idea of location-based addressing or identity. The fact that Alice reads Twt A and Bob reads Twt A at the same location, but Alice and Bob _could_ have in fact read very different content entirely. It is no longer possible to have consistency in a decentralised way that works properly.

One could argue this is fine, because we're so small and nothing matters, but it's a properly I rely on fairly heavily in yarnd, a properly that if lost would have significant impact on how yarnd works I think. 🤔
I think that's one of the worst aspects of the proposed idea of location-based addressing or identity. The fact that Alice reads Twt A and Bob reads Twt A at the same location, but Alice and Bob _could_ have in fact read very different content entirely. It is no longer possible to have consistency in a decentralised way that works properly.

One could argue this is fine, because we're so small and nothing matters, but it's a properly I rely on fairly heavily in yarnd, a properly that if lost would have significant impact on how yarnd works I think. 🤔
Unless I"m missing something here 🤔 But a <url> <timestamp> does not for me identify an individual Twt, it only identifies its location, which may or may not have changed since I last saw a version of it hmmm 🧐
Unless I"m missing something here 🤔 But a <url> <timestamp> does not for me identify an individual Twt, it only identifies its location, which may or may not have changed since I last saw a version of it hmmm 🧐
Also I'm not even sure I can validly cache, let alone index feeds anymore if we do this, because if the structure of a Twt is cuh that I can no longer trust that an individual Twt's content hasn't been changed at the source, what's the point of caching or indexing individual twts at all? This makes the implementations of yarnd and yarns (_the search engine, crawlers and indexer_) kind of hard to reason about.
Also I'm not even sure I can validly cache, let alone index feeds anymore if we do this, because if the structure of a Twt is cuh that I can no longer trust that an individual Twt's content hasn't been changed at the source, what's the point of caching or indexing individual twts at all? This makes the implementations of yarnd and yarns (_the search engine, crawlers and indexer_) kind of hard to reason about.
Also you're right I guess. But still that also requires the author not to change the timestamp too. Hmmm
Also you're right I guess. But still that also requires the author not to change the timestamp too. Hmmm
@movq I don't think there's any misunderstand at all. I just treat every lines in a feed as an individual entity. These are stored on their own.
@movq I don't think there's any misunderstand at all. I just treat every lines in a feed as an individual entity. These are stored on their own.
@movq So I obviously happen to agree with you as well. However in so saying, one of my goals was also to bring the simplicity of Twtxt to the Web and for the general "lay person" (_of sorts_). So I eventually found myself building yarnd. Has it been successful, well sort of, somewhat (_but that doesn't matter, I like that it's small and niche anyway_).

I agree that the goal of simplicity is a good goal to strive for, which is why I'm actually suggesting we change the Twt identifiers to be a simple SHA256 hash, something that everyone understand and has readily available tools for. I really don't think we should be doing any of this by hand to be honest. But part of the beauty of Twt Subject and Twt Hash(es) in the first place is replying by hand is much much easier because you only have a short 7 or 11 character thing to copy/paste in your reply. Switching to something like <url> <timestamp> with a space in it is going to become a lot harder to copy/paste, because you can't "double click" (_or is it triple click for some?_) to copy/paste to your clipboard/buffer now 🤣

Anyway I digress... On the whole edit thing, I'm actually find if we don't support it at all and don't build a protocol around that. I have zero issues with dropping that as an idea. Why? Because I actually think that clients should be auto-detecting edits anyway. They already can, I've PoC'd this myself, I _think_ it can be done. I haven't (yet), and one of the reasons I've not spent much effort in it is it isn't something that comes up frequently anyway.

Who cares if a thread breaks every now 'n again anyway?_