# I am the Watcher. I am your guide through this vast new twtiverse.
# 
# Usage:
#     https://watcher.sour.is/api/plain/users              View list of users and latest twt date.
#     https://watcher.sour.is/api/plain/twt                View all twts.
#     https://watcher.sour.is/api/plain/mentions?uri=:uri  View all mentions for uri.
#     https://watcher.sour.is/api/plain/conv/:hash         View all twts for a conversation subject.
# 
# Options:
#     uri     Filter to show a specific users twts.
#     offset  Start index for quey.
#     limit   Count of items to return (going back in time).
# 
# twt range = 1 60515
# self = https://watcher.sour.is?uri=https://twtxt.net/user/prologic/twtxt.txt&offset=57415
# next = https://watcher.sour.is?uri=https://twtxt.net/user/prologic/twtxt.txt&offset=57515
# prev = https://watcher.sour.is?uri=https://twtxt.net/user/prologic/twtxt.txt&offset=57315
@lyse Yup you're right, it's s terrible idea πŸ’‘
Well the poll clearly shows:

- ~65/35 in favor of Content Addressing
- ~60/40 in favor of supporting Edit/Delete
- ~70/30 against more cryptograph

And an NPS score of 7/10 🀣~
Well the poll clearly shows:

- ~65/35 in favor of Content Addressing
- ~60/40 in favor of supporting Edit/Delete
- ~70/30 against more cryptograph

And an NPS score of 7/10 🀣~
@bender Zero _technical_ issues 🀣 I never claims otherwise πŸ˜…
@bender Zero _technical_ issues 🀣 I never claims otherwise πŸ˜…
@bender Yes but you've got me curious now πŸ˜…
@bender Yes but you've got me curious now πŸ˜…
Okay, co-founder of Wordpress and CEO of Automation.

What has the poor guy done? 🀣
Okay, co-founder of Wordpress and CEO of Automation.

What has the poor guy done? 🀣
@david Who's Matt Mullenweg? πŸ€”
@david Who's Matt Mullenweg? πŸ€”
Like really tbh, it's just a matter of abstracting out the "fetching" part of your client. There are _zero_ issues with fetching Gopher/Gemini hosted feeds. They just lack any mechanisms for Discovery and Caching.
Like really tbh, it's just a matter of abstracting out the "fetching" part of your client. There are _zero_ issues with fetching Gopher/Gemini hosted feeds. They just lack any mechanisms for Discovery and Caching.
@doesnm Still haven't received it. Did you send to james at mills dot io? πŸ€”
@doesnm Still haven't received it. Did you send to james at mills dot io? πŸ€”
@movq I don't think I intend to either tbh for yarnd. If there was any poorly worded "things", it was just merely pointing out lacking capabilities for caching and discovery.
@movq I don't think I intend to either tbh for yarnd. If there was any poorly worded "things", it was just merely pointing out lacking capabilities for caching and discovery.
@bender Oh so what you're saying is "we" (royal we) ruined Twtxt 🀣
@bender Oh so what you're saying is "we" (royal we) ruined Twtxt 🀣
@doesnm Are you sure? Not seen the mail yet...
@doesnm Are you sure? Not seen the mail yet...
@aelaraji LOL πŸ˜‚ Here's one for you:

> You can take IRC out of my cold πŸ₯Ά dead 😡 hands πŸ™Œ
@aelaraji LOL πŸ˜‚ Here's one for you:

> You can take IRC out of my cold πŸ₯Ά dead 😡 hands πŸ™Œ
Anyway I've setup an alias now 🀣
Anyway I've setup an alias now 🀣
@doesnm Ooops you _might_ want to re-send that to james instead 🀣
@doesnm Ooops you _might_ want to re-send that to james instead 🀣
@aelaraji It sadly does not it seems. 🀣 Seems like the search engine has come across mentions of your feed via its other two protocols 🀣


$ inspect-db yarns.db | jq -r '.Value.URL' | grep 'aelaraji.com'
https://aelaraji.com/test_feed.txt
https://aelaraji.com/twtxt.txt
@aelaraji It sadly does not it seems. 🀣 Seems like the search engine has come across mentions of your feed via its other two protocols 🀣


$ inspect-db yarns.db | jq -r '.Value.URL' | grep 'aelaraji.com'
https://aelaraji.com/test_feed.txt
https://aelaraji.com/twtxt.txt
@doesnm My Salty public key is:

kex1fhxntuc0av7q48hlfj970ve297dzzghn82wp5cahr9r92y8rlrqqtwp983
@doesnm My Salty public key is:

kex1fhxntuc0av7q48hlfj970ve297dzzghn82wp5cahr9r92y8rlrqqtwp983
@doesnm Do you have a sample Caddy log file you can supply? I'll see if we can improve the tool πŸ‘Œ
@doesnm Do you have a sample Caddy log file you can supply? I'll see if we can improve the tool πŸ‘Œ
@doesnm Fot a sample access log? Which tool are you using?
@doesnm Fot a sample access log? Which tool are you using?
@doesnm I don't think it does. I think it's completely different to what you're thinking.
@doesnm I don't think it does. I think it's completely different to what you're thinking.
@doesnm Yeah just move your feet. It's totally fine. Don't worry about it.
@doesnm Yeah just move your feet. It's totally fine. Don't worry about it.
@doesnm I couldn't find any references to this anywhere either.
@doesnm I couldn't find any references to this anywhere either.
@doesnm Like now?
@doesnm Like now?
@doesnm I have no idea to be honest 🀣 I'm actually not really sure how you can ruin something be improving it πŸ€¦β€β™‚οΈ
@doesnm I have no idea to be honest 🀣 I'm actually not really sure how you can ruin something be improving it πŸ€¦β€β™‚οΈ
We:

- Drop # url= from the spec.
- We don't adopt # uuid = -- Something @anth also mentioned (_see below_)

We instead use the @nick@domain to identify your feed in the first place and use that as the identify when calculating Twt hashes <id> + <timestamp> + <content>. Now in an ideal world I also agree, use WebFinger for this and expect that for the most part you'll be doing a WebFinger lookup of @user@domain to fetch someone's feed in the first place.

The only problem with WebFinger is should this be mandated or a recommendation?
We:

- Drop # url= from the spec.
- We don't adopt # uuid = -- Something @anth also mentioned (_see below_)

We instead use the @nick@domain to identify your feed in the first place and use that as the identify when calculating Twt hashes <id> + <timestamp> + <content>. Now in an ideal world I also agree, use WebFinger for this and expect that for the most part you'll be doing a WebFinger lookup of @user@domain to fetch someone's feed in the first place.

The only problem with WebFinger is should this be mandated or a recommendation?
Something @anth said on ITC

> 17:42 I should also note in there that it doesn’t address the two things i really want it to: mandate utf-8 (which should be easy to fit in) and something for better @ mentions.

I actually agree with in both counts and it got me thinking...
Something @anth said on ITC

> 17:42 I should also note in there that it doesn’t address the two things i really want it to: mandate utf-8 (which should be easy to fit in) and something for better @ mentions.

I actually agree with in both counts and it got me thinking...
> you've ruined twtxt

Not sure what to say here. πŸ€”
> you've ruined twtxt

Not sure what to say here. πŸ€”
> Thank you for all the hard work put into the project.

Thank you to whomever said this! πŸ™‡β€β™‚οΈ
> Thank you for all the hard work put into the project.

Thank you to whomever said this! πŸ™‡β€β™‚οΈ
> Many of the faces go hand in hand or depend on the selected protocol a feed is published with or client features. I'm pretty sure people interpret different things into these terms.

See previous. Sorry 😞
> Many of the faces go hand in hand or depend on the selected protocol a feed is published with or client features. I'm pretty sure people interpret different things into these terms.

See previous. Sorry 😞
> Not sure what to think about the stack ranking question. I care that it’s a simple text file i can just stick on my server. Security, identity, &c come out of how I manage the server.

See previous.
> Not sure what to think about the stack ranking question. I care that it’s a simple text file i can just stick on my server. Security, identity, &c come out of how I manage the server.

See previous.
> I don't know what all the facets mean. E.g. what's the difference between "Integrity" and "Authenticity"?

Yes, I totally get where you're coming from. However after ~22 results, I _think_ y'all have figured out how to rank them appropriately anyway 🀣~
> I don't know what all the facets mean. E.g. what's the difference between "Integrity" and "Authenticity"?

Yes, I totally get where you're coming from. However after ~22 results, I _think_ y'all have figured out how to rank them appropriately anyway 🀣~
Sharing the comments of the poll (_anonymous so I have no idea whom the comments are from_):

> your poll should include questions about markdown. personally i think inline bits like style, links, images are yes. block quotes, code blocks, bullet lists are mid. but tables and footnotes are no.

Yes sorry about this, I wasn't able to change much after publishing the poll πŸ˜…
Sharing the comments of the poll (_anonymous so I have no idea whom the comments are from_):

> your poll should include questions about markdown. personally i think inline bits like style, links, images are yes. block quotes, code blocks, bullet lists are mid. but tables and footnotes are no.

Yes sorry about this, I wasn't able to change much after publishing the poll πŸ˜…
@slashdot GTFO 🀣
@slashdot GTFO 🀣
cc @xuu
cc @xuu
@bender Well as you've pointed out in the past, both protocol suffer from Discovery (_as I've stated as well_) and more often than not, users that publish Twtxt feeds over these protocols tend to just "point into the void" and it's next to impossible to have _any_ kind of "social interaction" (_ignoring personal choices of course, if one's feed is intended for 1-way ..._)
@bender Well as you've pointed out in the past, both protocol suffer from Discovery (_as I've stated as well_) and more often than not, users that publish Twtxt feeds over these protocols tend to just "point into the void" and it's next to impossible to have _any_ kind of "social interaction" (_ignoring personal choices of course, if one's feed is intended for 1-way ..._)
I _think_ there's a bug in yarnd hwoever:


$ yarnc debug https://sunshinegardens.org/~xjix/twtxt/tw.txt
...
bqor23a 2024-09-26T11:09:28-07:00	if twtxt 2 is dropping gemini support, i will probably move on and spend more time on my gemini social zine protocol instead. i think the direction of the protocol is probably fine, but for me web is a tier 2 publishing channel. if the choice is between gemini and http i'm always going to pick gemini. its been a fun ride, but i guess this is where i get off.


The yarnc CLI tool and the lextwt parser we use in yarnd _correctly_ parses the feed and sets the Twter.HashingURI to the latest # url = found in the feed. However my pod hasn't picked this up 😒 I follow @cuaxolotl as https://sunshinegardens.org/~xjix/twtxt/tw.txt
I _think_ there's a bug in yarnd hwoever:


$ yarnc debug https://sunshinegardens.org/~xjix/twtxt/tw.txt
...
bqor23a 2024-09-26T11:09:28-07:00	if twtxt 2 is dropping gemini support, i will probably move on and spend more time on my gemini social zine protocol instead. i think the direction of the protocol is probably fine, but for me web is a tier 2 publishing channel. if the choice is between gemini and http i'm always going to pick gemini. its been a fun ride, but i guess this is where i get off.


The yarnc CLI tool and the lextwt parser we use in yarnd _correctly_ parses the feed and sets the Twter.HashingURI to the latest # url = found in the feed. However my pod hasn't picked this up 😒 I follow @cuaxolotl as https://sunshinegardens.org/~xjix/twtxt/tw.txt
I _think_ there's a bug in yarnd hwoever:


$ yarnc debug https://sunshinegardens.org/~xjix/twtxt/tw.txt
...
bqor23a 2024-09-26T11:09:28-07:00\tif twtxt 2 is dropping gemini support, i will probably move on and spend more time on my gemini social zine protocol instead. i think the direction of the protocol is probably fine, but for me web is a tier 2 publishing channel. if the choice is between gemini and http i'm always going to pick gemini. its been a fun ride, but i guess this is where i get off.


The yarnc CLI tool and the lextwt parser we use in yarnd _correctly_ parses the feed and sets the Twter.HashingURI to the latest # url = found in the feed. However my pod hasn't picked this up 😒 I follow @cuaxolotl as https://sunshinegardens.org/~xjix/twtxt/tw.txt
Gemini/Gopher Twtxt feeds account for less than 1% in existence:


$ total=$(inspect-db yarns.db | jq -r '.Value.URL' | awk -F'//' '{if ($1 ~ /^https?/) print "http/https:"; else print $1}' | sort | uniq -c | awk '{sum+=$1} END {print sum}'); inspect-db yarns.db | jq -r '.Value.URL' | awk -F'//' '{if ($1 ~ /^https?/) print "http/https:"; else print $1}' | sort | uniq -c | awk -v total="$total" '{printf "%d %s %.2f%%\n", $1, $2, ($1/total)*100}' | sort -r
7 gemini: 0.66%
4 gopher: 0.38%
1046 http/https: 98.96%
Gemini/Gopher Twtxt feeds account for less than 1% in existence:


$ total=$(inspect-db yarns.db | jq -r '.Value.URL' | awk -F'//' '{if ($1 ~ /^https?/) print "http/https:"; else print $1}' | sort | uniq -c | awk '{sum+=$1} END {print sum}'); inspect-db yarns.db | jq -r '.Value.URL' | awk -F'//' '{if ($1 ~ /^https?/) print "http/https:"; else print $1}' | sort | uniq -c | awk -v total="$total" '{printf "%d %s %.2f%%\\n", $1, $2, ($1/total)*100}' | sort -r
7 gemini: 0.66%
4 gopher: 0.38%
1046 http/https: 98.96%
Gemini/Gopher Twtxt feeds account for <1% in existence:


$ total=$(inspect-db yarns.db | jq -r '.Value.URL' | awk -F'//' '{if ($1 ~ /^https?/) print "http/https:"; else print $1}' | sort | uniq -c | awk '{sum+=$1} END {print sum}'); inspect-db yarns.db | jq -r '.Value.URL' | awk -F'//' '{if ($1 ~ /^https?/) print "http/https:"; else print $1}' | sort | uniq -c | awk -v total="$total" '{printf "%d %s %.2f%%\\n", $1, $2, ($1/total)*100}' | sort -r
7 gemini: 0.66%
4 gopher: 0.38%
1046 http/https: 98.96%
Gemini/Gopher Twtxt feeds account for less than 1% in existence:


$ total=$(inspect-db yarns.db | jq -r '.Value.URL' | awk -F'//' '{if ($1 ~ /^https?/) print "http/https:"; else print $1}' | sort | uniq -c | awk '{sum+=$1} END {print sum}'); inspect-db yarns.db | jq -r '.Value.URL' | awk -F'//' '{if ($1 ~ /^https?/) print "http/https:"; else print $1}' | sort | uniq -c | awk -v total="$total" '{printf "%d %s %.2f%%\n", $1, $2, ($1/total)*100}' | sort -r
7 gemini: 0.66%
4 gopher: 0.38%
1046 http/https: 98.96%
Note however this doesn't solve the problem of Caching at all. It just works around it and with enough clients fetching a Gopher/Gemini feed, this # refresh becomes useless anyway at a certain point of scale.
Note however this doesn't solve the problem of Caching at all. It just works around it and with enough clients fetching a Gopher/Gemini feed, this # refresh becomes useless anyway at a certain point of scale.
@bender Well this is the thing really. Gopher and Gemini are very broken ways to distributed content. Broken in the sense that for Twtxt either support a) caching in any way shape or form b) discovery in any way shape or form.

This is a bit of a problem because if a Feed author complains (_nad they have in the past_) that their Gopher/Gemini feeds are being hit "too hard", well that's really kind of on them for choosing to host their feed on an ill advised protocol thatc cannot possibly support Caching at all.

This is primarily one of the reasons we introduced the idea of a "feed advised refresh interval" that clients SHOULD respect.

See: https://dev.twtxt.net/doc/metadataextension.html#refresh

> refresh
> This optional field is used by feed authors as a hint to clients to control how often they should fetch or update this feed.
>
> The value of this field is seconds represented by an integer.
>
> NOTE: An empty, bad, or unparsable value is ignored.
@bender Well this is the thing really. Gopher and Gemini are very broken ways to distributed content. Broken in the sense that for Twtxt either support a) caching in any way shape or form b) discovery in any way shape or form.

This is a bit of a problem because if a Feed author complains (_nad they have in the past_) that their Gopher/Gemini feeds are being hit "too hard", well that's really kind of on them for choosing to host their feed on an ill advised protocol thatc cannot possibly support Caching at all.

This is primarily one of the reasons we introduced the idea of a "feed advised refresh interval" that clients SHOULD respect.

See: https://dev.twtxt.net/doc/metadataextension.html#refresh

> refresh
> This optional field is used by feed authors as a hint to clients to control how often they should fetch or update this feed.
>
> The value of this field is seconds represented by an integer.
>
> NOTE: An empty, bad, or unparsable value is ignored.
@bender To be fair, this has never been a problem for folks that have/use stable Feed URI(s) 🀣
@bender To be fair, this has never been a problem for folks that have/use stable Feed URI(s) 🀣
This is confirmed to be the case:


$ for url in gemini://sunshinegardens.org/~xjix/twtxt/tw.txt https://sunshinegardens.org/~xjix/twtxt/tw.txt //sunshinegardens.org/~xjix/twtxt/tw.txt; do yarnc hash -t '2024-09-26T11:09:28-07:00' -u "$url" "if twtxt 2 is dropping gemini support, i will probably move on and spend more time on my gemini social zine protocol instead. i think the direction of the protocol is probably fine, but for me web is a tier 2 publishing channel. if the choice is between gemini and http i'm always going to pick gemini. its been a fun ride, but i guess this is where i get off."; done
fk2af7q
7kvnpaq
bqor23a
This is confirmed to be the case:


$ for url in gemini://sunshinegardens.org/~xjix/twtxt/tw.txt https://sunshinegardens.org/~xjix/twtxt/tw.txt //sunshinegardens.org/~xjix/twtxt/tw.txt; do yarnc hash -t '2024-09-26T11:09:28-07:00' -u "$url" "if twtxt 2 is dropping gemini support, i will probably move on and spend more time on my gemini social zine protocol instead. i think the direction of the protocol is probably fine, but for me web is a tier 2 publishing channel. if the choice is between gemini and http i'm always going to pick gemini. its been a fun ride, but i guess this is where i get off."; done
fk2af7q
7kvnpaq
bqor23a
Yup confirmed!


# url = //sunshinegardens.org/~xjix/twtxt/tw.txt
# url = https://sunshinegardens.org/~xjix/twtxt/tw.txt
# url = gemini://sunshinegardens.org/~xjix/twtxt/tw.txt


@cuaxolotl has changed the url of their feed (_yet again_) and changed every hash in their feed.

@antonio is right to call this out. We should drop the reliance on the # url metadata field and in fact we should probably just drop this entirely from the spec and go with # uuid as the basis of a feed's identity.

Even though this happens very rarely (_feeds moving to new locations_) it more frequently happens with folks that try to serve their feed from Gopher, HTTP and Gemini.
Yup confirmed!


# url = //sunshinegardens.org/~xjix/twtxt/tw.txt
# url = https://sunshinegardens.org/~xjix/twtxt/tw.txt
# url = gemini://sunshinegardens.org/~xjix/twtxt/tw.txt


@cuaxolotl has changed the url of their feed (_yet again_) and changed every hash in their feed.

@antonio is right to call this out. We should drop the reliance on the # url metadata field and in fact we should probably just drop this entirely from the spec and go with # uuid as the basis of a feed's identity.

Even though this happens very rarely (_feeds moving to new locations_) it more frequently happens with folks that try to serve their feed from Gopher, HTTP and Gemini.
@bender Hmm I _think_ I know why...


2024-09-27T01:28:53+00:00	(#bqor23a) @<cuaxolotl https://sunshinegardens.org/~xj9/twtxt/tw.txt> Wait, what!? We're dropping Gemini support!?


From @aelaraji's feed. I think @cuaxolotl doesn't do threading properly, I've run into this once before. I'm not sure what client they use? πŸ€”
@bender Hmm I _think_ I know why...


2024-09-27T01:28:53Z\t(#bqor23a) @<cuaxolotl https://sunshinegardens.org/~xj9/twtxt/tw.txt> Wait, what!? We're dropping Gemini support!?


From @aelaraji's feed. I think @cuaxolotl doesn't do threading properly, I've run into this once before. I'm not sure what client they use? πŸ€”
@bender Hmm I _think_ I know why...


2024-09-27T01:28:53+00:00\t(#bqor23a) @<cuaxolotl https://sunshinegardens.org/~xj9/twtxt/tw.txt> Wait, what!? We're dropping Gemini support!?


From @aelaraji's feed. I think @cuaxolotl doesn't do threading properly, I've run into this once before. I'm not sure what client they use? πŸ€”
@bender Hmm I _think_ I know why...


2024-09-27T01:28:53Z	(#bqor23a) @<cuaxolotl https://sunshinegardens.org/~xj9/twtxt/tw.txt> Wait, what!? We're dropping Gemini support!?


From @aelaraji's feed. I think @cuaxolotl doesn't do threading properly, I've run into this once before. I'm not sure what client they use? πŸ€”
@bender Hmm I _think_ I know why...


2024-09-27T01:28:53+00:00	(#bqor23a) @<cuaxolotl https://sunshinegardens.org/~xj9/twtxt/tw.txt> Wait, what!? We're dropping Gemini support!?


From @aelaraji's feed. I think @cuaxolotl doesn't do threading properly, I've run into this once before. I'm not sure what client they use? πŸ€”
Search engine doesn't know about it either: https://search.twtxt.net/twt/bqor23a
Search engine doesn't know about it either: https://search.twtxt.net/twt/bqor23a
@bender Re that broken thread (#bqor23a). Its the same one. My pod doesn't have the Root Twt: https://twtxt.net/twt/bqor23a => 404 Not Found.

How in the hell did you even reply to this in the first place?=
@bender Re that broken thread (#bqor23a). Its the same one. My pod doesn't have the Root Twt: https://twtxt.net/twt/bqor23a => 404 Not Found.

How in the hell did you even reply to this in the first place?=
Hmmm https://twtxt.net/twt/bqor23a => 404 Not Found 🀣=
Hmmm https://twtxt.net/twt/bqor23a => 404 Not Found 🀣=
@bender What was this in reply to? πŸ€”
@bender What was this in reply to? πŸ€”
@aelaraji Probably not.
@aelaraji Probably not.
@cuaxolotl Context?