# I am the Watcher. I am your guide through this vast new twtiverse.
# 
# Usage:
#     https://watcher.sour.is/api/plain/users              View list of users and latest twt date.
#     https://watcher.sour.is/api/plain/twt                View all twts.
#     https://watcher.sour.is/api/plain/mentions?uri=:uri  View all mentions for uri.
#     https://watcher.sour.is/api/plain/conv/:hash         View all twts for a conversation subject.
# 
# Options:
#     uri     Filter to show a specific users twts.
#     offset  Start index for quey.
#     limit   Count of items to return (going back in time).
# 
# twt range = 1 2172
# self = https://watcher.sour.is?uri=https://twtxt.net/user/mckinley/twtxt.txt&offset=1272
# next = https://watcher.sour.is?uri=https://twtxt.net/user/mckinley/twtxt.txt&offset=1372
# prev = https://watcher.sour.is?uri=https://twtxt.net/user/mckinley/twtxt.txt&offset=1172
Nope, no horizontal rule. What about referenced links?

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
![An image on my website][2]

\n\n\n\n\n\n\n\n: https://mckinley.cc/
\n\n\n\n\n\n\n\n: https://mckinley.cc/img/ladybird-yarn-20220924-2.png
Nope, no horizontal rule. What about referenced links?

[My website][My website=][1][1=]
![An image on my website][2]

[1][1=]: https://mckinley.cc/
[2][2=]: https://mckinley.cc/img/ladybird-yarn-20220924-2.png
Nope, no horizontal rule. What about referenced links?

[My website][My website=][My website][My website=][My website][My website=][My website][My website=][1][1=][1][1=][1][1=][1][1=]
![An image on my website][2]

[1][1=][1][1=][1][1=][1][1=]: https://mckinley.cc/
[2][2=][2][2=][2][2=][2][2=]: https://mckinley.cc/img/ladybird-yarn-20220924-2.png
@prologic I don't know if a metadata field is strictly necessary. I think there ought to be a defined set of syntax that all clients with Markdown support can be expected to handle in the same way. CommonMark maybe? It looks like Yarn supports most of CommonMark already, though I've never seen a horizontal rule. Let's try it:
***
Some text here
---***
Of course I pick the day before simple.css gets merged into main to get a new Ladybird screenshot. Here's one post-merge.
twtxt.net's Discover page viewed in Ladybird, post-merge, on 2022-09-24
Of course I pick the day before simple.css gets merged into main to get a new Ladybird screenshot. Here's one post-merge.
twtxt.net's Discover page viewed in Ladybird, post-merge, on 2022-09-24
@akoizumi It looks like that feed uses TIMESTAMP\\tTITLE: PERMALINK which would be harder to parse programmatically.

This discussion has me thinking of a serious syndication format built on top of twtxt that could be implemented in normal feed readers. It would be limited, but extremely easy for a Webmaster to implement. Users could also receive updates with a normal twtxt client. I think there could be some utility in it.
@akoizumi It looks like that feed uses TIMESTAMP\tTITLE: PERMALINK which would be harder to parse programmatically.

This discussion has me thinking of a serious syndication format built on top of twtxt that could be implemented in normal feed readers. It would be limited, but extremely easy for a Webmaster to implement. Users could also receive updates with a normal twtxt client. I think there could be some utility in it.
@akoizumi It looks like that feed uses TIMESTAMP\\tTITLE: PERMALINK which would be harder to parse programmatically.

This discussion has me thinking of a serious syndication format built on twtxt that could be implemented in normal feed readers. It would be limited, but extremely easy for a Webmaster to implement. Users could also receive updates with a normal twtxt client. I think there could be some utility in it.
@akoizumi It looks like that feed uses TIMESTAMP\\tTITLE: PERMALINK which would be harder to parse programmatically.

This discussion has me thinking of a serious syndication format built on twtxt that could be implemented in normal feed readers. It would be limited, but extremely easy for a Webmaster to implement. You could also receive updates with a normal twtxt client. I think there could be some utility in it.
@akoizumi It looks like that feed uses TIMESTAMP\\tTITLE: PERMALINK which would be harder to parse programmatically.

This discussion has me thinking of a serious syndication format built on twtxt that could be implemented in normal feed readers. It would be limited, but extremely easy for a Webmaster to implement. You could also receive updates with a normal twtxt client. I think there could be some utility in it.
> For a real feed format I would like to have a clear separation between titles and content. And more options for the content. Plaintext and HTML at least.

I don't think it's a very good idea to include content when using twtxt as a syndication format. Anything based on twtxt, in my opinion, should retain the spirit of the original specification, especially readability by humans and machines. 10K of HTML in one line absolutely breaks human readability.

What about TIMESTAMP\tTITLE\tPERMALINK, like the following?


2022-09-22T14:53:26-07:00	Bringing Back a Useful Browser Feature With a Bookmarklet	https://mckinley.cc/blog/20220922.html
> For a real feed format I would like to have a clear separation between titles and content. And more options for the content. Plaintext and HTML at least.

I don't think it's a very good idea to include content when using twtxt as a syndication format. Anything based on twtxt, in my opinion, should retain the spirit of the original specification, especially readability by humans and machines. 10K of HTML in one line absolutely breaks human readability.

What about TIMESTAMP\\tTITLE\\tPERMALINK, like the following?


2022-09-22T14:53:26-07:00\tBringing Back a Useful Browser Feature With a Bookmarklet\thttps://mckinley.cc/blog/20220922.html
@lyse
> Twtxt is plaintext, but lots of folks (me included) actually use markdown in their yarns. However, the actual format being used is not advertised anywhere.

That's a really good point. We should formalize a Markdown flavor as a format extension on https://dev.twtxt.net/.

yarnd appears to use gomarkdown with a few extensions. I'm not sure what else gomarkdown translates by default.

There should definitely be a concrete specification on what syntax should be supported by twtxt clients with "Markdown" support.
@lyse
> Twtxt is plaintext, but lots of folks (me included) actually use markdown in their yarns. However, the actual format being used is not advertised anywhere.

That's a really good point. We should formalize a Markdown flavor as a format extension on https://dev.twtxt.net/.

yarnd appears to use gomarkdown with a few extensions. I'm not sure what else gomarkdown translates by default.

There should definitely be a concrete specification on what syntax should be supported by twtxt clients with "Markdown" support. cc @prologic and @darch
@tkanos This is really interesting stuff. Do you plan on sharing the code, or at least a more detailed write-up of your process?
@eaplmx I think twtxt is a fine feed format, especially if you're allergic to XML.
LibWeb font handling has greatly improved. We're no longer stuck with that terrible bitmap font. It looks like there are still a few font-related issues to iron out, but here is what the Yarn discover page looks like on Ladybird. A huge improvement in only a week, though it really can't decide where it should put that search box.
twtxt.net's Discover page viewed in Ladybird on 2022-09-24
LibWeb font handling has greatly improved. We're no longer stuck with that terrible bitmap font. It looks like there are still a few font-related issues to iron out, but here is what the Yarn discover page looks like on Ladybird. A huge improvement in only a week, though it really can't decide where it should put that search box.
twtxt.net's Discover page viewed in Ladybird on 2022-09-24
@movq I use it in some of my blog posts for inline code snippets so they don't get wrapped where it would make it hard to read. Other places, too, but I can't remember right now.
@movq I use it in some of my blog posts for inline code snippets so they don't get wrapped where it would make it hard to read.
A nice chat tonight with @prologic, @ocdtrekkie, and @darch. Some things we talked about:
* Markdown isn't rendered to HTML in Atom feeds (#989)
* A look at @darch's branch to replace Pico.css with Simple.css
* 100 monthly active users on yarn.social
* Sandstorm proxy authentication for yarnd (Allows one set of credentials across programs hosted on Sandstorm)
* Sandstorm support for Tube (#26)
* Future improvements to Sandstorm
@movq I use the no-break space all the time, I only just learned about its horizontal brother.
@prologic are we having the call tonight?
@abucci @lyse I hope the bookmarklet can be of use. :)
@darch I'd like to add pixelblog to groovy-twtxt but I would rather not include software without a formal license.
@tkanos I'm 6th, 57th, and 1280th. What metric did you use, total number of mentions? Also, is this supposed to be a continuation of mdom's project of the same name?
@eaplmx That's the surprise. Take a look at view-source:https://mckinley.cc/blog/atom.xml
@eaplmx That's the surprise. Take a look at view-source:https://mckinley.cc/blog/atom.xml
@eaplmx That's the surprise. Take a look at view-source:https://mckinley.cc/blog/atom.xml
@eaplmx That's the surprise. Take a look at view-source:https://mckinley.cc/blog/atom.xml
You know, I thought you had to use a Duopoly browser if you wanted client-side XSLT but I just learned WebKit supports it as well! That means it works on Otter Browser and WebPositive. Safari too, probably, but we don't talk about that one.
@lyse Thank you, that's a great compliment. I'm quite proud of that stylesheet, especially the bit that converts RFC 3339 timestamps to the friendly date format I use.
Bringing Back a Useful Browser Feature With a Bookmarklet: https://mckinley.cc/blog/20220922.html
@prologic The first two, definitely. Do you want me to open a couple issues?
@prologic I didn't really want to announce it until I expanded it some more, but I started a list of twtxt-related things over on Gitea. Perhaps it could be relevant here?
@prologic I didn't really want to announce it until I expanded it some more, but I started a list of twtxt-related things over on Gitea. Perhaps it could be relevant here?
These aren't things I could fix myself, and in my opinion some of them wouldn't be worth it if I could.
These aren't things I could fix myself, and in my opinion it's not worth it.
@prologic It would definitely be interesting. I don't think there could be any substantial code reuse but I'm definitely open to the idea. I looked into doing it, but there are a few hangups that would probably need to be addressed first:
* Something in the pipeline is adding U+FFFD Replacement Characters to the ends of certain twts
* Markdown isn't rendered to HTML in the Atom feed, but mentions are.
* <summary> elements (which should probably be <contents> instead) are type="html", escaped HTML. It wouldn't be possible to embed twt contents in the generated page on Firefox-based browsers due to my favorite bug
\t* The other options are either plain text or inline XHTML which means you have to make sure every twt is valid XHTML.
* I don't know how to handle different themes short of JavaScript which can be buggy when used with client-side XSLT or dynamically generated XSLT which is completely and utterly ridiculous.*
@prologic It would definitely be interesting. I don't think there could be any substantial code reuse but I'm open to the idea. I looked into doing it, but there are a few hangups that would probably need to be addressed first:
* Something in the pipeline is adding U+FFFD Replacement Characters to the ends of certain twts
* Markdown isn't rendered to HTML in the Atom feed, but mentions are.
* <summary> elements (which should probably be <contents> instead) are type="html", escaped HTML. It wouldn't be possible to embed twt contents in the generated page on Firefox-based browsers due to my favorite bug
\t* The other options are either plain text or inline XHTML which means you have to make sure every twt is valid XHTML.
* I don't know how to handle different themes short of JavaScript which can be buggy when used with client-side XSLT or dynamically generated XSLT which is completely and utterly ridiculous.*
@prologic It would definitely be interesting. I don't think there could be any substantial code reuse but I'm open to the idea. I looked into doing it, but there are a few hangups that would probably need to be addressed first:
* Something in the pipeline is adding U+FFFD Replacement Characters to the ends of certa*
@prologic It would definitely be interesting. I don't think there could be any substantial code reuse but I'm open to the idea. I looked into doing it, but there are a few hangups that would probably need to be addressed first:
* Something in the pipeline is adding U+FFFD Replacement Characters to the ends of certain twts
* Markdown isn't rendered to HTML in the Atom feed, but mentions are.
* <summary> elements (which should probably be <contents> instead) are type="html", escaped HTML. It wouldn't be possible to embed twt contents in the generated page on Firefox-based browsers due to my favorite bug
* The other options are either plain text or inline XHTML which means you have to make sure every twt is valid XHTML.
* I don't know how to handle different themes short of JavaScript which can be buggy when used with client-side XSLT or dynamically generated XSLT which is completely and utterly ridiculous.*
@ocdtrekkie :)
@darch @eaplmx Isn't this what the twtxt user agent is for?
@darch Is pixelblog a fork of picoblog? Does pixelblog have a software license? Picoblog doesn't.
@movq Install our sketchy browser extension and maybe we'll let you browse the Web again.
Install our sketchy browser extension and maybe you can browse the Web again.
@ocdtrekkie I don't know, I'm still waiting for the paperwork on that.
@darch That's exactly what I was thinking, it looks really nice. I can't decide if I like the dark/blue or the dark/yellow theme better.
@tkanos Why is that?
@darch I'm talking about the gray color for the Tabler icons in the first two themes, i.e. the rich text buttons. I was wondering what they would look like if they used the theme's accent color or the text color. Now that I think of it, the text color wouldn't look right either.

The third theme (black/yellow) uses the accent color for those buttons. I was using that as an example.
@darch Looks great. What would the 1st and 2nd themes look like if the icons were the color of the text? Or the accent color like the 4th theme?
@akoizumi I have a similar rule. I will install a Python program if it's polished enough (e.g. yt-dlp) but Node.js is a hard pass.
@prologic I must be mistaken. I thought, when we realized @maya was still posting in the future, that there was an entire thread (with a /conv/ page) and the hash didn't resolve to anything until the timestamp of the parent twt. Perhaps yarnd gave it a /conv/ page because it knew about the twt but didn't display it because it was in the future. Perhaps I need to go to bed.
This is a reply to my reply
This is a twt in reply to a twt that doesn't exist
(foo bar baz) @prologic But it doesn't give us a /conv/ page because 'foo bar baz' is obviously an invalid hash. Let me try something.
I wonder, what's stopping users from not following the twt hash spec and prepending a Git commit hash to every new post? AFAIK yarnd will display the oldest post it has for a hash as the root, but it assumes that it's a reply to another post with that hash and that people replying should add to that chain. I'm not advocating for breaking the spec, but how do other clients handle this?
I wonder, what's stopping users from not following the twt hash spec and prepending a Git commit hash to every new post? AFAIK yarnd will treat the oldest post it has for a certain hash as the root and it will handle it as such when people reply, assuming that there is a post with that hash somewhere and people should be replying to
I wonder, what's stopping users from not following the twt hash spec and prepending a Git commit hash to every new post? AFAIK yarnd will treat the oldest post it has for a certain hash as the root, and it will handle it as such when people reply.
I wonder, what's stopping users from not following the twt hash spec and prepending a Git commit hash to every post? AFAIK yarnd will treat the oldest post it has for a certain hash as the root, and it will handle it as such when people reply.
@prologic, they're spreading. https://git.mills.io/Jeansonhacks/Jeansonhacks007
Test
Completely off topic: I was hoping to make a redirect rule for Scrolller to Libreddit, but it seems their post IDs are just random.

I did find that the original path on Reddit is in the page inside a stringified JSON object as a global variable. JSON.parse(window.scrolllerConfig).item.redditPath will return the path.

I suppose this could be done with a userscript, but the whole point is to not load the Scrolller page. Interestingly, the first one you posted has been deleted from Reddit.
Completely off topic: I was hoping to make a redirect rule for Scrolller to Libreddit, but it seems their post IDs are just random.

I did find that the original path on Reddit is in the page inside a stringified JSON object as a global variable defined in a script element. JSON.parse(window.scrolllerConfig).item.redditPath will return the path.

I suppose this could be done with a userscript, but the whole point is to not load the Scrolller page. Interestingly, the first one you posted has been deleted from Reddit.
This thread has some hilarious examples of prompt injection: https://nitter.1d4.us/simonw/status/1569451817897193473
"Bot prevention" was a poor choice of words. I meant "Unwanted bot prevention." A GPT-3 bot would be interesting @prologic. Just watch out for prompt injection.
@lyse I figured it was a milk vending machine, but I'd never heard of such a thing. Interesting. That's better than Canadian bagged milk.
@lyse I figured it was a milk vending machine, but I've never heard of such a thing. Interesting. That's better than Canadian bagged milk.
@movq @prologic It's Group Policy on a blockchain.
@prologic Oh, that's definitely one of my new favorite talks. Right up there with That Awesome Time I Was Sued For Two Billion Dollars. My favorite part was the Ruby unless statement.
@prologic I watched that last night, it's definitely one of my new favorite talks. Right up there with That Awesome Time I Was Sued For Two Billion Dollars. My favorite part was the Ruby unless statement.
@prologic Oh, that's definitely one of my new favorite talks. Right up there with That Awesome Time I Was Sued For Two Billion Dollars. My favorite part was the Ruby unless statement.
A very insightful chat tonight with @prologic, @ocdtrekkie, @darch, and @ocdtrekkie's friend Ian who is, presumably, still primarily using legacy social networks. Hopefully we can change that. Some things we talked about tonight:

* The rationale behind GoNix
* @darch's thoughts on social networking (https://blog.neotxt.dk/)
* Bringing people on to free platforms
* Bot prevention on Yarn.social (#981)
* Funding for Yarn.social
* Goryon, material design, and satanism
@ocdtrekkie It's not my video, but it's almost certainly real. It's really jarring to use to a stock browser after you're used to instant page loads.
@eaplmx I think we're on Web 3.11 for Workgroups now.
@abucci @movq uBlock Origin has some good annoyance filter lists available out of the box, you just need to enable them in the settings.
@prologic What about this? https://clbin.com/9BNBE
@prologic Like a BBS, then? We'd need some cool ASCII art.
@prologic I will, if my internet connection holds up. :)
@prologic I'll be there, if my internet connection holds up. :)
Here's a particularly egregious example of this: https://tube.mills.io/v/Ysz6i7etWF6mJNwXnekPEF
@lyse What do you mean, "tapped milk"?
@prologic It seems that LibWeb doesn't support textarea elements properly. You can select the textarea, but you can't type in it. The "Post" button works when JavaScript is disabled, but there's nothing in the textarea to post.
Recent updates to LibWeb have fixed the icons on the Yarn web client in Ladybird and the SerenityOS browser! Unfortunately, it still isn't possible to post using LibWeb browsers but we'll get there. :^)

twtxt.net's Discover page viewed in Ladybird on 2022-09-15

For comparison, here's a screenshot from 2022-07-22:
twtxt.net's Discover page viewed in Ladybird on 2022-07-22
@eaplmx I've been looking for a way to turn a Tor v3 address into a mnemonic phrase in the spirit of BIP39 but I can't find one. I found https://github.com/ryepdx/keyphrase in my adventures, maybe the concepts from it could help you.
@lyse I'd love to read it, man.
@lyse I think we've talked about this before, but my front-end for every video host is MPV+yt-dlp with some MPV userscripts to improve the experience, called by Newsboat and my Web browser using ff2mpv.

Lately, I've been thinking about a wrapper script for MPV that tries to find a copy of the video on disk before it lets MPV use yt-dlp to stream it. This would be used in conjunction with another program that grabs a list of articles in Newsboat with a specific tag to filter videos from non-videos. Videos marked unread are queued for download with yt-dlp if there isn't already a local copy. Videos marked read are deleted if there is a local copy. I think this could be done by interacting with Newsboat's database directly, but that isn't preferable.

How does ybeuter work? Is the source available?
@lyse I think we've talked about this before, but my front-end for every video host is MPV+yt-dlp with some MPV userscripts to improve the experience, called by Newsboat and my Web browser using ff2mpv.

Lately, I've been thinking about a wrapper script for MPV that tries to find a copy of the video on disk before it lets MPV use yt-dlp to stream it. This would be used in conjunction with another program that grabs a list of articles in Newsboat with a specific tag to filter videos from non-videos.

Videos marked unread are queued for download with yt-dlp if there isn't already a local copy. Videos marked read are deleted if there is a local copy. I think this could be done by interacting with Newsboat's database directly, but that isn't preferable.

How does ybeuter work? Is the source available?
@lyse I think we've talked about this before, but my front-end for every video host is MPV+yt-dlp with some MPV userscripts to improve the experience, called by Newsboat and my Web browser using ff2mpv.

Lately, I've been thinking about a wrapper script for MPV that tries to find a copy of the video on disk before it lets MPV use yt-dlp to stream it. This would be used in conjunction with another program that grabs a list of articles in Newsboat with a specific tag to filter videos from non-videos. Videos marked unread are queued for download with yt-dlp if there isn't already a local copy. Videos marked unread are deleted if there is a local copy. I think this could be done by interacting with Newsboat's database directly, but that isn't preferable.

How does ybeuter work? Is the source available?
@prologic You should read my blog :) https://mckinley.cc/blog/20220715.html
@lyse @prologic It's just an instance of Invidious, a simple front-end for YouTube.
@prologic Yeah, for a few hours last night. I went to bed early, it was nice.
@ocdtrekkie @prologic I wish I could have made it, but my internet connection was down.
yt-dlp works for this. You just have to have a US IP address. The download speeds are very good as well.
@thecanine Will it let you use it without connecting it to the internet?
@thecanine
@prologic I have more of a problem with Smart TVs than NFTs...
@prologic I have more of a problem with Smart TVs than with NFTs...