# I am the Watcher. I am your guide through this vast new twtiverse.
# 
# Usage:
#     https://watcher.sour.is/api/plain/users              View list of users and latest twt date.
#     https://watcher.sour.is/api/plain/twt                View all twts.
#     https://watcher.sour.is/api/plain/mentions?uri=:uri  View all mentions for uri.
#     https://watcher.sour.is/api/plain/conv/:hash         View all twts for a conversation subject.
# 
# Options:
#     uri     Filter to show a specific users twts.
#     offset  Start index for quey.
#     limit   Count of items to return (going back in time).
# 
# twt range = 1 60441
# self = https://watcher.sour.is?uri=https://twtxt.net/user/prologic/twtxt.txt&offset=57391
# next = https://watcher.sour.is?uri=https://twtxt.net/user/prologic/twtxt.txt&offset=57491
# prev = https://watcher.sour.is?uri=https://twtxt.net/user/prologic/twtxt.txt&offset=57291
Anyway I've setup an alias now 🤣
@doesnm Ooops you _might_ want to re-send that to james instead 🤣
@doesnm Ooops you _might_ want to re-send that to james instead 🤣
@aelaraji It sadly does not it seems. 🤣 Seems like the search engine has come across mentions of your feed via its other two protocols 🤣


$ inspect-db yarns.db | jq -r '.Value.URL' | grep 'aelaraji.com'
https://aelaraji.com/test_feed.txt
https://aelaraji.com/twtxt.txt
@aelaraji It sadly does not it seems. 🤣 Seems like the search engine has come across mentions of your feed via its other two protocols 🤣


$ inspect-db yarns.db | jq -r '.Value.URL' | grep 'aelaraji.com'
https://aelaraji.com/test_feed.txt
https://aelaraji.com/twtxt.txt
@doesnm My Salty public key is:

kex1fhxntuc0av7q48hlfj970ve297dzzghn82wp5cahr9r92y8rlrqqtwp983
@doesnm My Salty public key is:

kex1fhxntuc0av7q48hlfj970ve297dzzghn82wp5cahr9r92y8rlrqqtwp983
@doesnm Do you have a sample Caddy log file you can supply? I'll see if we can improve the tool 👌
@doesnm Do you have a sample Caddy log file you can supply? I'll see if we can improve the tool 👌
@doesnm Fot a sample access log? Which tool are you using?
@doesnm Fot a sample access log? Which tool are you using?
@doesnm I don't think it does. I think it's completely different to what you're thinking.
@doesnm I don't think it does. I think it's completely different to what you're thinking.
@doesnm Yeah just move your feet. It's totally fine. Don't worry about it.
@doesnm Yeah just move your feet. It's totally fine. Don't worry about it.
@doesnm I couldn't find any references to this anywhere either.
@doesnm I couldn't find any references to this anywhere either.
@doesnm Like now?
@doesnm Like now?
@doesnm I have no idea to be honest 🤣 I'm actually not really sure how you can ruin something be improving it 🤦‍♂️
@doesnm I have no idea to be honest 🤣 I'm actually not really sure how you can ruin something be improving it 🤦‍♂️
We:

- Drop # url= from the spec.
- We don't adopt # uuid = -- Something @anth also mentioned (_see below_)

We instead use the @nick@domain to identify your feed in the first place and use that as the identify when calculating Twt hashes <id> + <timestamp> + <content>. Now in an ideal world I also agree, use WebFinger for this and expect that for the most part you'll be doing a WebFinger lookup of @user@domain to fetch someone's feed in the first place.

The only problem with WebFinger is should this be mandated or a recommendation?
We:

- Drop # url= from the spec.
- We don't adopt # uuid = -- Something @anth also mentioned (_see below_)

We instead use the @nick@domain to identify your feed in the first place and use that as the identify when calculating Twt hashes <id> + <timestamp> + <content>. Now in an ideal world I also agree, use WebFinger for this and expect that for the most part you'll be doing a WebFinger lookup of @user@domain to fetch someone's feed in the first place.

The only problem with WebFinger is should this be mandated or a recommendation?
Something @anth said on ITC

> 17:42 I should also note in there that it doesn’t address the two things i really want it to: mandate utf-8 (which should be easy to fit in) and something for better @ mentions.

I actually agree with in both counts and it got me thinking...
Something @anth said on ITC

> 17:42 I should also note in there that it doesn’t address the two things i really want it to: mandate utf-8 (which should be easy to fit in) and something for better @ mentions.

I actually agree with in both counts and it got me thinking...
> you've ruined twtxt

Not sure what to say here. 🤔
> you've ruined twtxt

Not sure what to say here. 🤔
> Thank you for all the hard work put into the project.

Thank you to whomever said this! 🙇‍♂️
> Thank you for all the hard work put into the project.

Thank you to whomever said this! 🙇‍♂️
> Many of the faces go hand in hand or depend on the selected protocol a feed is published with or client features. I'm pretty sure people interpret different things into these terms.

See previous. Sorry 😞
> Many of the faces go hand in hand or depend on the selected protocol a feed is published with or client features. I'm pretty sure people interpret different things into these terms.

See previous. Sorry 😞
> Not sure what to think about the stack ranking question. I care that it’s a simple text file i can just stick on my server. Security, identity, &c come out of how I manage the server.

See previous.
> Not sure what to think about the stack ranking question. I care that it’s a simple text file i can just stick on my server. Security, identity, &c come out of how I manage the server.

See previous.
> I don't know what all the facets mean. E.g. what's the difference between "Integrity" and "Authenticity"?

Yes, I totally get where you're coming from. However after ~22 results, I _think_ y'all have figured out how to rank them appropriately anyway 🤣~
> I don't know what all the facets mean. E.g. what's the difference between "Integrity" and "Authenticity"?

Yes, I totally get where you're coming from. However after ~22 results, I _think_ y'all have figured out how to rank them appropriately anyway 🤣~
Sharing the comments of the poll (_anonymous so I have no idea whom the comments are from_):

> your poll should include questions about markdown. personally i think inline bits like style, links, images are yes. block quotes, code blocks, bullet lists are mid. but tables and footnotes are no.

Yes sorry about this, I wasn't able to change much after publishing the poll 😅
Sharing the comments of the poll (_anonymous so I have no idea whom the comments are from_):

> your poll should include questions about markdown. personally i think inline bits like style, links, images are yes. block quotes, code blocks, bullet lists are mid. but tables and footnotes are no.

Yes sorry about this, I wasn't able to change much after publishing the poll 😅
@slashdot GTFO 🤣
@slashdot GTFO 🤣
cc @xuu
cc @xuu
@bender Well as you've pointed out in the past, both protocol suffer from Discovery (_as I've stated as well_) and more often than not, users that publish Twtxt feeds over these protocols tend to just "point into the void" and it's next to impossible to have _any_ kind of "social interaction" (_ignoring personal choices of course, if one's feed is intended for 1-way ..._)
@bender Well as you've pointed out in the past, both protocol suffer from Discovery (_as I've stated as well_) and more often than not, users that publish Twtxt feeds over these protocols tend to just "point into the void" and it's next to impossible to have _any_ kind of "social interaction" (_ignoring personal choices of course, if one's feed is intended for 1-way ..._)
I _think_ there's a bug in yarnd hwoever:


$ yarnc debug https://sunshinegardens.org/~xjix/twtxt/tw.txt
...
bqor23a 2024-09-26T11:09:28-07:00	if twtxt 2 is dropping gemini support, i will probably move on and spend more time on my gemini social zine protocol instead. i think the direction of the protocol is probably fine, but for me web is a tier 2 publishing channel. if the choice is between gemini and http i'm always going to pick gemini. its been a fun ride, but i guess this is where i get off.


The yarnc CLI tool and the lextwt parser we use in yarnd _correctly_ parses the feed and sets the Twter.HashingURI to the latest # url = found in the feed. However my pod hasn't picked this up 😢 I follow @cuaxolotl as https://sunshinegardens.org/~xjix/twtxt/tw.txt
I _think_ there's a bug in yarnd hwoever:


$ yarnc debug https://sunshinegardens.org/~xjix/twtxt/tw.txt
...
bqor23a 2024-09-26T11:09:28-07:00	if twtxt 2 is dropping gemini support, i will probably move on and spend more time on my gemini social zine protocol instead. i think the direction of the protocol is probably fine, but for me web is a tier 2 publishing channel. if the choice is between gemini and http i'm always going to pick gemini. its been a fun ride, but i guess this is where i get off.


The yarnc CLI tool and the lextwt parser we use in yarnd _correctly_ parses the feed and sets the Twter.HashingURI to the latest # url = found in the feed. However my pod hasn't picked this up 😢 I follow @cuaxolotl as https://sunshinegardens.org/~xjix/twtxt/tw.txt
I _think_ there's a bug in yarnd hwoever:


$ yarnc debug https://sunshinegardens.org/~xjix/twtxt/tw.txt
...
bqor23a 2024-09-26T11:09:28-07:00\tif twtxt 2 is dropping gemini support, i will probably move on and spend more time on my gemini social zine protocol instead. i think the direction of the protocol is probably fine, but for me web is a tier 2 publishing channel. if the choice is between gemini and http i'm always going to pick gemini. its been a fun ride, but i guess this is where i get off.


The yarnc CLI tool and the lextwt parser we use in yarnd _correctly_ parses the feed and sets the Twter.HashingURI to the latest # url = found in the feed. However my pod hasn't picked this up 😢 I follow @cuaxolotl as https://sunshinegardens.org/~xjix/twtxt/tw.txt
Gemini/Gopher Twtxt feeds account for less than 1% in existence:


$ total=$(inspect-db yarns.db | jq -r '.Value.URL' | awk -F'//' '{if ($1 ~ /^https?/) print "http/https:"; else print $1}' | sort | uniq -c | awk '{sum+=$1} END {print sum}'); inspect-db yarns.db | jq -r '.Value.URL' | awk -F'//' '{if ($1 ~ /^https?/) print "http/https:"; else print $1}' | sort | uniq -c | awk -v total="$total" '{printf "%d %s %.2f%%\n", $1, $2, ($1/total)*100}' | sort -r
7 gemini: 0.66%
4 gopher: 0.38%
1046 http/https: 98.96%
Gemini/Gopher Twtxt feeds account for less than 1% in existence:


$ total=$(inspect-db yarns.db | jq -r '.Value.URL' | awk -F'//' '{if ($1 ~ /^https?/) print "http/https:"; else print $1}' | sort | uniq -c | awk '{sum+=$1} END {print sum}'); inspect-db yarns.db | jq -r '.Value.URL' | awk -F'//' '{if ($1 ~ /^https?/) print "http/https:"; else print $1}' | sort | uniq -c | awk -v total="$total" '{printf "%d %s %.2f%%\\n", $1, $2, ($1/total)*100}' | sort -r
7 gemini: 0.66%
4 gopher: 0.38%
1046 http/https: 98.96%
Gemini/Gopher Twtxt feeds account for <1% in existence:


$ total=$(inspect-db yarns.db | jq -r '.Value.URL' | awk -F'//' '{if ($1 ~ /^https?/) print "http/https:"; else print $1}' | sort | uniq -c | awk '{sum+=$1} END {print sum}'); inspect-db yarns.db | jq -r '.Value.URL' | awk -F'//' '{if ($1 ~ /^https?/) print "http/https:"; else print $1}' | sort | uniq -c | awk -v total="$total" '{printf "%d %s %.2f%%\\n", $1, $2, ($1/total)*100}' | sort -r
7 gemini: 0.66%
4 gopher: 0.38%
1046 http/https: 98.96%
Gemini/Gopher Twtxt feeds account for less than 1% in existence:


$ total=$(inspect-db yarns.db | jq -r '.Value.URL' | awk -F'//' '{if ($1 ~ /^https?/) print "http/https:"; else print $1}' | sort | uniq -c | awk '{sum+=$1} END {print sum}'); inspect-db yarns.db | jq -r '.Value.URL' | awk -F'//' '{if ($1 ~ /^https?/) print "http/https:"; else print $1}' | sort | uniq -c | awk -v total="$total" '{printf "%d %s %.2f%%\n", $1, $2, ($1/total)*100}' | sort -r
7 gemini: 0.66%
4 gopher: 0.38%
1046 http/https: 98.96%
Note however this doesn't solve the problem of Caching at all. It just works around it and with enough clients fetching a Gopher/Gemini feed, this # refresh becomes useless anyway at a certain point of scale.
Note however this doesn't solve the problem of Caching at all. It just works around it and with enough clients fetching a Gopher/Gemini feed, this # refresh becomes useless anyway at a certain point of scale.
@bender Well this is the thing really. Gopher and Gemini are very broken ways to distributed content. Broken in the sense that for Twtxt either support a) caching in any way shape or form b) discovery in any way shape or form.

This is a bit of a problem because if a Feed author complains (_nad they have in the past_) that their Gopher/Gemini feeds are being hit "too hard", well that's really kind of on them for choosing to host their feed on an ill advised protocol thatc cannot possibly support Caching at all.

This is primarily one of the reasons we introduced the idea of a "feed advised refresh interval" that clients SHOULD respect.

See: https://dev.twtxt.net/doc/metadataextension.html#refresh

> refresh
> This optional field is used by feed authors as a hint to clients to control how often they should fetch or update this feed.
>
> The value of this field is seconds represented by an integer.
>
> NOTE: An empty, bad, or unparsable value is ignored.
@bender Well this is the thing really. Gopher and Gemini are very broken ways to distributed content. Broken in the sense that for Twtxt either support a) caching in any way shape or form b) discovery in any way shape or form.

This is a bit of a problem because if a Feed author complains (_nad they have in the past_) that their Gopher/Gemini feeds are being hit "too hard", well that's really kind of on them for choosing to host their feed on an ill advised protocol thatc cannot possibly support Caching at all.

This is primarily one of the reasons we introduced the idea of a "feed advised refresh interval" that clients SHOULD respect.

See: https://dev.twtxt.net/doc/metadataextension.html#refresh

> refresh
> This optional field is used by feed authors as a hint to clients to control how often they should fetch or update this feed.
>
> The value of this field is seconds represented by an integer.
>
> NOTE: An empty, bad, or unparsable value is ignored.
@bender To be fair, this has never been a problem for folks that have/use stable Feed URI(s) 🤣
@bender To be fair, this has never been a problem for folks that have/use stable Feed URI(s) 🤣
This is confirmed to be the case:


$ for url in gemini://sunshinegardens.org/~xjix/twtxt/tw.txt https://sunshinegardens.org/~xjix/twtxt/tw.txt //sunshinegardens.org/~xjix/twtxt/tw.txt; do yarnc hash -t '2024-09-26T11:09:28-07:00' -u "$url" "if twtxt 2 is dropping gemini support, i will probably move on and spend more time on my gemini social zine protocol instead. i think the direction of the protocol is probably fine, but for me web is a tier 2 publishing channel. if the choice is between gemini and http i'm always going to pick gemini. its been a fun ride, but i guess this is where i get off."; done
fk2af7q
7kvnpaq
bqor23a
This is confirmed to be the case:


$ for url in gemini://sunshinegardens.org/~xjix/twtxt/tw.txt https://sunshinegardens.org/~xjix/twtxt/tw.txt //sunshinegardens.org/~xjix/twtxt/tw.txt; do yarnc hash -t '2024-09-26T11:09:28-07:00' -u "$url" "if twtxt 2 is dropping gemini support, i will probably move on and spend more time on my gemini social zine protocol instead. i think the direction of the protocol is probably fine, but for me web is a tier 2 publishing channel. if the choice is between gemini and http i'm always going to pick gemini. its been a fun ride, but i guess this is where i get off."; done
fk2af7q
7kvnpaq
bqor23a
Yup confirmed!


# url = //sunshinegardens.org/~xjix/twtxt/tw.txt
# url = https://sunshinegardens.org/~xjix/twtxt/tw.txt
# url = gemini://sunshinegardens.org/~xjix/twtxt/tw.txt


@cuaxolotl has changed the url of their feed (_yet again_) and changed every hash in their feed.

@antonio is right to call this out. We should drop the reliance on the # url metadata field and in fact we should probably just drop this entirely from the spec and go with # uuid as the basis of a feed's identity.

Even though this happens very rarely (_feeds moving to new locations_) it more frequently happens with folks that try to serve their feed from Gopher, HTTP and Gemini.
Yup confirmed!


# url = //sunshinegardens.org/~xjix/twtxt/tw.txt
# url = https://sunshinegardens.org/~xjix/twtxt/tw.txt
# url = gemini://sunshinegardens.org/~xjix/twtxt/tw.txt


@cuaxolotl has changed the url of their feed (_yet again_) and changed every hash in their feed.

@antonio is right to call this out. We should drop the reliance on the # url metadata field and in fact we should probably just drop this entirely from the spec and go with # uuid as the basis of a feed's identity.

Even though this happens very rarely (_feeds moving to new locations_) it more frequently happens with folks that try to serve their feed from Gopher, HTTP and Gemini.
@bender Hmm I _think_ I know why...


2024-09-27T01:28:53+00:00	(#bqor23a) @<cuaxolotl https://sunshinegardens.org/~xj9/twtxt/tw.txt> Wait, what!? We're dropping Gemini support!?


From @aelaraji's feed. I think @cuaxolotl doesn't do threading properly, I've run into this once before. I'm not sure what client they use? 🤔
@bender Hmm I _think_ I know why...


2024-09-27T01:28:53Z\t(#bqor23a) @<cuaxolotl https://sunshinegardens.org/~xj9/twtxt/tw.txt> Wait, what!? We're dropping Gemini support!?


From @aelaraji's feed. I think @cuaxolotl doesn't do threading properly, I've run into this once before. I'm not sure what client they use? 🤔
@bender Hmm I _think_ I know why...


2024-09-27T01:28:53+00:00\t(#bqor23a) @<cuaxolotl https://sunshinegardens.org/~xj9/twtxt/tw.txt> Wait, what!? We're dropping Gemini support!?


From @aelaraji's feed. I think @cuaxolotl doesn't do threading properly, I've run into this once before. I'm not sure what client they use? 🤔
@bender Hmm I _think_ I know why...


2024-09-27T01:28:53Z	(#bqor23a) @<cuaxolotl https://sunshinegardens.org/~xj9/twtxt/tw.txt> Wait, what!? We're dropping Gemini support!?


From @aelaraji's feed. I think @cuaxolotl doesn't do threading properly, I've run into this once before. I'm not sure what client they use? 🤔
@bender Hmm I _think_ I know why...


2024-09-27T01:28:53+00:00	(#bqor23a) @<cuaxolotl https://sunshinegardens.org/~xj9/twtxt/tw.txt> Wait, what!? We're dropping Gemini support!?


From @aelaraji's feed. I think @cuaxolotl doesn't do threading properly, I've run into this once before. I'm not sure what client they use? 🤔
Search engine doesn't know about it either: https://search.twtxt.net/twt/bqor23a
Search engine doesn't know about it either: https://search.twtxt.net/twt/bqor23a
@bender Re that broken thread (#bqor23a). Its the same one. My pod doesn't have the Root Twt: https://twtxt.net/twt/bqor23a => 404 Not Found.

How in the hell did you even reply to this in the first place?=
@bender Re that broken thread (#bqor23a). Its the same one. My pod doesn't have the Root Twt: https://twtxt.net/twt/bqor23a => 404 Not Found.

How in the hell did you even reply to this in the first place?=
Hmmm https://twtxt.net/twt/bqor23a => 404 Not Found 🤣=
Hmmm https://twtxt.net/twt/bqor23a => 404 Not Found 🤣=
@bender What was this in reply to? 🤔
@bender What was this in reply to? 🤔
@aelaraji Probably not.
@aelaraji Probably not.
@cuaxolotl Context?
@cuaxolotl Context?
(#bqor23a) @aelaraji
@cuaxolotl We probably won’t in fairness. i only called it out because discovery is made much harder with Gopher and Gemini. Caching is also impossible too.
@aelaraji
@cuaxolotl We probably won’t in fairness. U only called it out because discovery is made much harder with Gopher and Gemini. Caching is also impossible too.
(#bqor23a) @aelaraji
@cuaxolotl We probably won’t in fairness. i only called it out because discovery is made much harder with Gopher and Gemini. Caching is also impossible too.
@cuaxolotl We probably won't in fairness. U only called it out because discovery is made much harder with Gopher and Gemini. Caching is also impossible too.
@cuaxolotl We probably won't in fairness. I only called it out because discovery is made much harder with Gopher and Gemini. Caching is also impossible too.
@cuaxolotl We probably won't in fairness. I only called it out because discovery is made much harder with Gopher and Gemini. Caching is also impossible too.
Last chance to have your say before tomorrow's meetup:

http://polljunkie.com/poll/xdgjib/twtxt-v2
Last chance to have your say before tomorrow's meetup:

http://polljunkie.com/poll/xdgjib/twtxt-v2
No reason I haven't switched. I trust Gitea (for now).
No reason I haven't switched. I trust Gitea (for now).
@david SQLite
@david SQLite
@david Yup! 🤞
@david Yup! 🤞
@david Staying private until I've matured them a bit more 😅
@david Staying private until I've matured them a bit more 😅
> “For every complex problem, there is a solution that is clear, simple, and wrong.”

-- H.L. Mencken
> “For every complex problem, there is a solution that is clear, simple, and wrong.”

-- H.L. Mencken
Also, I'm not editing the original post. 😅
Also, I'm not editing the original post. 😅
@bender ha ha yes ideally one day I would love it if Twt hashes referenced at least any yarnd clients were automatically linked. 🤣
@bender ha ha yes ideally one day I would love it if Twt hashes referenced at least any yarnd clients were automatically linked. 🤣
> "Everything should be made as simple as possible, but not simpler."
– *Albert Einstein*

> The beauty of simplicity lies in not losing the essence.

#simplicity #Einstein #wisdom