# I am the Watcher. I am your guide through this vast new twtiverse.
# 
# Usage:
#     https://watcher.sour.is/api/plain/users              View list of users and latest twt date.
#     https://watcher.sour.is/api/plain/twt                View all twts.
#     https://watcher.sour.is/api/plain/mentions?uri=:uri  View all mentions for uri.
#     https://watcher.sour.is/api/plain/conv/:hash         View all twts for a conversation subject.
# 
# Options:
#     uri     Filter to show a specific users twts.
#     offset  Start index for quey.
#     limit   Count of items to return (going back in time).
# 
# twt range = 1 196295
# self = https://watcher.sour.is?offset=169838
# next = https://watcher.sour.is?offset=169938
# prev = https://watcher.sour.is?offset=169738
@prologic it is the attention to detail. Evidently no one from Australia tried to login on my Sendgrid account, but the IP the used correlates to the GeoIP DB.
Interesting 🤔
Interesting 🤔
Notice the detail on the IP address mentioned. It is one assigned to Australia: https://ipgeolocation.io/browse/ip/203.0.113.78
This happened yesterday:

Screenshot of an email, allegedly from Sendgrid

The first give away is the sender, sendgrid@autovitalsinc.com. Not Sengrid. Now, check the URL on the link provided to check the account activity:


https://u906946.ct.sendgrid.net/ls/click?upn=u001.eXk7eIEvNT22LuyWQ0fseoc5VY1jItvxPoavh2wfNVs292YMzvTAPj5D6nek1U6K7UfW_AsM5Hq3TBeAGlZrT-2F3g23iWCcJRPGZ-2B58DJxpgMgOTjgWklNQiAdGiHqmR6FFVhfWZJhnu1PSRslMuKGg1XNZs5e1lGu8kmdKhv7otlghl6qLMXiiXYZcvaUB5NruWwSBFcLdvi31NY-2Fru5oyrcrugm2iLYA0u5TiufyvA7SNTo3sDHx6WtS-2FmfEyN2svb9k1S4QGRFhuDseidMiFm0f9Q-3D-3D


I was curious, so I follow it on my dedicated VM for these kind of things. It took me to a page looking exactly like a Sendgrid login, with a sendgrid.net URL. Upon entering yourmotherisahamster@gmail.com, as username, and yourfathersmellsofelderberries as password, it sent me to https://screenprank.com/gandalf/.

It was well done. This morning the same link renders a blank page with a "Not found" link that takes you to a 404. Hmm...
@bender That sucks 😢 Sorry to hear you didn't sleep well 😴
@bender That sucks 😢 Sorry to hear you didn't sleep well 😴
[47°09′40″S, 126°43′24″W] Transfer 25% complete...
@prologic well, that's a mighty reason not to do anything else! It has been already 8 hours, but I wish and hope you had a good night sleep. I woke up today, using 50% of my will power to do so.
****
Los viernes son lunes al revés. ⌘ Read more****
****
Esto es así ⌘ Read more****
rm -fr, remove french-style: merciless colonizer. thief and destroyer of worlds.
[47°09′30″S, 126°43′06″W] Transponder still failing -- switching to analog communication
[47°09′54″S, 126°43′24″W] Transponder jammed
@bender I was in bed 🤣
@bender I was in bed 🤣
🧮 USERS:1 FEEDS:2 TWTS:1084 ARCHIVED:78231 CACHE:2434 FOLLOWERS:17 FOLLOWING:14
maybe i'm overly restrained when it comes to making changes in a codebase, but i do a lot of the work in my head before committing to code.
running grey until after dark, it'll be around 40C until tuesday in the daytime so I can't really run the main server while I'm sat in my office. i'll have some pine quartz blades or my quattro arrays up some day to handle the lower power stuff so running grey doesn't mean the whole system is offline.
On my blog: Real Life in Star Trek, The Outcast https://john.colagioia.net/blog/2024/09/05/outcast.html #scifi #startrek #closereading
One last thing before I hit the hay. This endpoint could respond with the raw twt, when asking for text/plain (it serves HTML at the moment). Return the physical line from the feed. Maybe with a comment above for the feed URL. Or doesn't the registry format also include the URL separated with a tab somehow? I'm too lazy right now to look it up. Also, not sure how useful that would be. Anyway, good night.
@lyse Thanks
@prologic Perfect, thanks. For my own future reference: curl -H 'Accept: application/json' https://twtxt.net/twt/st3wsda
[47°09′22″S, 126°43′39″W] Transponder still failing
@quark Right, a little rain improves™ the situation so much… :-( Surprise, surprise, our rain has been delayed again.

Good hunting and bon appétit! :-) I never had Puerto Rico's national dish, but the photos look delicious. Yum!

I also tried ice cream, but I reckon I simply stick to your last tip instead. :-)
@prologic come on, provide us with a one liner curl foo that does just that, don't be lazy! :-P
I reckon, this is as raw as you can get, @falsifian: curl -sH "Accept: application/json" https://twtxt.net/twt/st3wsda | jq You can piece it together from created and text (and twter.uri).
@bender Haha I aggressively unfollow feds that are like this now 🤣
@bender Haha I aggressively unfollow feds that are like this now 🤣
@bender So far I've been following feeds fairly liberally. I'll check to see if we have anything in common and lean toward following, just because this is new to me and it feels like a small community. But I'm still figuring out what I want. Later I'll probably either trim my follower list or come up with some way to prioritize the feeds I'm more interested in.
@falsifian Yes hit a Twt permalink URI and ask for application/ json
@falsifian Yes hit a Twt permalink URI and ask for application/ json
@prologic Specifically, I could view yarnd's copy here, but only as rendered for a human to view: https://twtxt.net/twt/st3wsda
@lyse Uggh 🥵 That sounds awful and reminds me of our very odd little 3-day heat wave we had last week 😱
@lyse Uggh 🥵 That sounds awful and reminds me of our very odd little 3-day heat wave we had last week 😱
@bender Thanks! 🙇‍♂️
@bender Thanks! 🙇‍♂️
@movq thanks for getting to the bottom of it. @prologic is there a way to view yarnd's copy of the raw twt? The edit didn't result in a visible change; being able to see what yarnd originally downloaded would have helped me debug.
I see 🤔 Thanks!
I see 🤔 Thanks!
@quark Oh shit, second place! :-O


$ du -h .config/twtxt/cache*.db 
13M   .config/twtxt/cache2.db  # contains read status for each twt (very inefficient format)
7,0M  .config/twtxt/cache.db   # the actual cache by the original twtxt reference implementation


Yeah, wrong place for caches.
@prologic I don't. I will pay close attention on any oddity I see next time. Sorry!
@movq This ☝️
@movq This ☝️
@bender Do you recall what you were clicking through? 🤦‍♂️
@bender Do you recall what you were clicking through? 🤦‍♂️
@bender I follow feeds that are somewhat interesting to me. At least for the most part.
@lyse 31°C here, feels like 33°C, with a lovely 75% of humidity. It has been raining, on and off (to make matter "better") the whole day until now. No horses here, but if you go outside you will smell the same smell of farm animals (like goats, or pigs). That's because two or three kilometres from here there are private farms, and when the wind blows in such way, well, we are reminded of their existence.

I haven't left the house, so it feels well under air conditioning. In two more hours I will call it quits from the work day, and will have to dash to the grocery to get supplies for tonight's meal (arroz con gandules). I will let you know how it truly feels out there then. :-D

For those swollen fingers, nothing better than a mildly cold shower! Oh, and paws off the keyboard! :-P
@lyse 31°C here, feels like 33°C, with a lovely 75% of humidity. It has been raining, on and off (to make matter "better") the whole day until now. No horses here, but if you go outside you will smell the same smell of farm animals (like goats, or pigs). That's because two or three kilometres from here there are private farms, and when the wind blows in such way, well, we are reminded of their existence.

I haven't left the house, so it feels well under air conditioning. In two more hours I will call it quits from the work day, and will have to dash to the grocery to get supplies for tonight's meal (arroz con gandules). I will let you know how it truly feels out there then. :-D

For those swollen fingers, nothing better than a mildly cold shower! Oh, and paws off the keyboard! :-P
@prologic Thanks for the honor, but I'm not sure why I specifically should be part of a deciding committee here. :-D I get it, I just fear people might misunderstand your message here.

I have to read up on the twtxt registry documentation on the weekend (too tired at the moment), but it should probably be no real issue to integrate that API into yarnd.
@abucci well, those are top ten "twtxtrs" (as in, how many twtxts they have produced). @prologic sure is a conversational fellow. :-D
@abucci well, those are top ten "twtxtrs" (as in, how many twtxts they have produced). @prologic sure is a conversational fellow. :-D
When we passed a few horses in the forest, there was really strong soup odor in the air. It didn't smell like horse at all, but soup. Maybe they've been soup horses, chickens were out of stock.

29°C, zero wind, extremely humid, luckily the sun was behind the clouds. I'm soaking wet, sweat ran down in streams and dripped in my eyes, it burned a bit. The sky is getting a little dark, I hope the thunderstorm and rain are really arriving here later. Rain had always been finally cancelled the couple last days.

I'm gotta go cool off my fingers now, they're swollen from the heat.
@rrraksamam a few. :-) And thanks!
[47°09′11″S, 126°43′18″W] Transponder malfunction
@rrraksamam which one? 😅
@rrraksamam ya, entiendo. Asi nos pasa a todos, creo. Vamos, que nos gusta, nos gusta! 🙂
Err, Garner, rather.
Somebody likes Jennifer Gardner... 😊
@movq ha! Here are my top 10:


24056 "prologic"
5103 "lyse"
3932 "movq"
1984 "abucci"
1876 "adi"
1633 "fastidious"
1551 "jlj"
1455 "mckinley"
1413 "offgridliving
1280 "eaplmx"


Some of those I no longer follow, or do not exist, but their wisdom remains. LOL.
@movq ha! Here are my top 10:


24056 "prologic"
5103 "lyse"
3932 "movq"
1984 "abucci"
1876 "adi"
1633 "fastidious"
1551 "jlj"
1455 "mckinley"
1413 "offgridliving
1280 "eaplmx"


Some of those I no longer follow, or do not exist, but their wisdom remains. LOL.
@bender The size … it depends. 🥴

$ du -sh ~/Mail/twt
244M /home/user/Mail/twt

But:

$ du -sh --apparent-size ~/Mail/twt
33M /home/user/Mail/twt

There are about 60k twts in there.

Regarding one-way junk: True. Looks like I mostly unfollowed those, I don’t really have that in my inbox. 🤔

These are the Top 10, btw:

$ awk '/^From: / { user\n++ } END { for (u in user) { print user\n, u } }' * | sort -k1rn | head -n 10
24020 "prologic"
5269 "lyse"
3928 "movq"
2285 "adi"
1985 "abucci"
1713 "mckinley"
1415 "off_grid_living"
1352 "darch"
1280 "eaplmx"
956 "bender"*
@bender The size … it depends. 🥴

$ du -sh ~/Mail/twt
244M /home/user/Mail/twt

But:

$ du -sh --apparent-size ~/Mail/twt
33M /home/user/Mail/twt

There are about 60k twts in there.

Regarding one-way junk: True. Looks like I mostly unfollowed those, I don’t really have that in my inbox. 🤔

These are the Top 10, btw:

$ awk '/^From: / { user[$2]++ } END { for (u in user) { print user[u], u } }' * | sort -k1rn | head -n 10
24020 "prologic"
5269 "lyse"
3928 "movq"
2285 "adi"
1985 "abucci"
1713 "mckinley"
1415 "off_grid_living"
1352 "darch"
1280 "eaplmx"
956 "bender"*
@bender The size … it depends. 🥴

$ du -sh ~/Mail/twt
244M /home/user/Mail/twt

But:

$ du -sh --apparent-size ~/Mail/twt
33M /home/user/Mail/twt

There are about 60k twts in there.

Regarding one-way junk: True. Looks like I mostly unfollowed those, I don’t really have that in my inbox. 🤔

These are the Top 10, btw:

$ awk '/^From: / { user[$2]++ } END { for (u in user) { print user[u], u } }' * | sort -k1rn | head -n 10
24020 "prologic"
5269 "lyse"
3928 "movq"
2285 "adi"
1985 "abucci"
1713 "mckinley"
1415 "off_grid_living"
1352 "darch"
1280 "eaplmx"
956 "bender"*
@bender The size … it depends. 🥴

$ du -sh ~/Mail/twt
244M /home/user/Mail/twt

But:

$ du -sh --apparent-size ~/Mail/twt
33M /home/user/Mail/twt

There are about 60k twts in there.

Regarding one-way junk: True. Looks like I mostly unfollowed those, I don’t really have that in my inbox. 🤔

These are the Top 10, btw:

$ awk '/^From: / { user[$2]++ } END { for (u in user) { print user[u], u } }' * | sort -k1rn | head -n 10
24020 "prologic"
5269 "lyse"
3928 "movq"
2285 "adi"
1985 "abucci"
1713 "mckinley"
1415 "off_grid_living"
1352 "darch"
1280 "eaplmx"
956 "bender"*
@bender The size … it depends. 🥴

$ du -sh ~/Mail/twt
244M /home/user/Mail/twt

But:

$ du -sh --apparent-size ~/Mail/twt
33M /home/user/Mail/twt

There are about 60k twts in there.

Regarding one-way junk: True. Looks like I mostly unfollowed those, I don’t really have that in my inbox. 🤔

These are the Top 10, btw:

$ awk '/^From: / { user[$2]++ } END { for (u in user) { print user[u], u } }' * | sort -k1rn | head -n 10
24020 "prologic"
5269 "lyse"
3928 "movq"
2285 "adi"
1985 "abucci"
1713 "mckinley"
1415 "off_grid_living"
1352 "darch"
1280 "eaplmx"
956 "bender"*
@movq good idea, considering it might occasionally not work at all (because of edited twtxts).
@movq good idea, considering it might occasionally not work at all (because of edited twtxts).
@dbucklin very nice, thank you for sharing! I like that kind of retailers too, so those are on my list now. 🙂
@dbucklin very nice, thank you for sharing! I like that kind of retailers too, so those are on my list now. 🙂
Oh, and about the "There’s so little going on in general, it hardly matters. 😅", the issue *I* have with it is that it is mostly one-way junk. Why to bother following that which never engages, or spews seemingly non-sense into the air (or both!)? 😅
@movq this is @quark's follow list:


anth http://a.9srv.net/tw.txt
bender https://twtxt.net/user/bender/twtxt.txt
johanbove https://johanbove.info/twtxt.txt
ionores https://twtxt.net/user/ionores/twtxt.txt
lyse https://lyse.isobeef.org/twtxt.txt
mckinley https://twtxt.net/user/mckinley/twtxt.txt
movq https://www.uninformativ.de/twtxt.txt
prologic https://twtxt.net/user/prologic/twtxt.txt
stigatle https://yarn.stigatle.no/user/stigatle/twtxt.txt
thecanine https://twtxt.net/user/thecanine/twtxt.txt
xuu https://txt.sour.is/user/xuu/twtxt.txt
abucci https://anthony.buc.ci/user/abucci/twtxt.txt
soren https://darch.dk/twtxt.txt
aelaraji https://aelaraji.com/twtxt.txt
dbucklin https://www.davebucklin.com/twtxt.txt
falsifian https://www.falsifian.org/twtxt.txt
off_grid https://twtxt.net/user/off_grid_living/twtxt.txt
yarn_police https://twtxt.net/user/yarn_police/twtxt.txt


And the Maildir size is 212MB. What size (not followers list) is yours?
@movq right, there is no way to forbid anyone from editing their text files. There is no hand holding here. :-D Thank you very much for shedding light into this "mystery"!
@bender On twtxt, I follow all feeds that I can find (there are *some* exceptions, of course). There’s so little going on in general, it hardly matters. 😅

And I just realized: Mutt’s layout helps a lot. Skimming over new twts is really easy and it’s not a big loss if there are a couple of shitposts™ in my “timeline”. This is very different from Mastodon (both the default web UI and all clients I’ve tried), where the timeline is always *huge*. Posts take up a lot of space on screen. Makes me think twice if I want to follow someone or not. 😅

(I mostly only follow Hashtags on Mastodon anyway. It’s more interesting that way.)
@bender On twtxt, I follow all feeds that I can find (there are *some* exceptions, of course). There’s so little going on in general, it hardly matters. 😅

And I just realized: Mutt’s layout helps a lot. Skimming over new twts is really easy and it’s not a big loss if there are a couple of shitposts™ in my “timeline”. This is very different from Mastodon (both the default web UI and all clients I’ve tried), where the timeline is always *huge*. Posts take up a lot of space on screen. Makes me think twice if I want to follow someone or not. 😅

(I mostly only follow Hashtags on Mastodon anyway. It’s more interesting that way.)
@bender On twtxt, I follow all feeds that I can find (there are *some* exceptions, of course). There’s so little going on in general, it hardly matters. 😅

And I just realized: Mutt’s layout helps a lot. Skimming over new twts is really easy and it’s not a big loss if there are a couple of shitposts™ in my “timeline”. This is very different from Mastodon (both the default web UI and all clients I’ve tried), where the timeline is always *huge*. Posts take up a lot of space on screen. Makes me think twice if I want to follow someone or not. 😅

(I mostly only follow Hashtags on Mastodon anyway. It’s more interesting that way.)
@bender On twtxt, I follow all feeds that I can find (there are *some* exceptions, of course). There’s so little going on in general, it hardly matters. 😅

And I just realized: Mutt’s layout helps a lot. Skimming over new twts is really easy and it’s not a big loss if there are a couple of shitposts™ in my “timeline”. This is very different from Mastodon (both the default web UI and all clients I’ve tried), where the timeline is always *huge*. Posts take up a lot of space on screen. Makes me think twice if I want to follow someone or not. 😅

(I mostly only follow Hashtags on Mastodon anyway. It’s more interesting that way.)
@falsifian @prologic @bender The twt was edited. In my cache, it also has hash st3wsda and it started like this:

(#yqke7sq) I've been sketching out some …

When fetching the feed *now*, the twt starts like this and the current twt gets the hash 6mdqxrq:

(#yqke7sq) I've been sketching out some …

This can’t be avoided, really. Publishing twts and then editing them is like doing a git push --force after rewriting the commit history. Chaos will ensue. 😅
@falsifian @prologic @bender The twt was edited. In my cache, it also has hash st3wsda and it started like this:

(#yqke7sq) I've been sketching out some …

When fetching the feed *now*, the twt starts like this and the current twt gets the hash 6mdqxrq:

(#yqke7sq) I've been sketching out some …

This can’t be avoided, really. Publishing twts and then editing them is like doing a git push --force after rewriting the commit history. Chaos will ensue. 😅
@falsifian @prologic @bender The twt was edited. In my cache, it also has hash st3wsda and it started like this:

(#yqke7sq) I've been sketching out some …

When fetching the feed *now*, the twt starts like this and the current twt gets the hash 6mdqxrq:

(#yqke7sq) I've been sketching out some …

This can’t be avoided, really. Publishing twts and then editing them is like doing a git push --force after rewriting the commit history. Chaos will ensue. 😅
@falsifian @prologic @bender The twt was edited. In my cache, it also has hash st3wsda and it started like this:

(#yqke7sq) I've been sketching out some …

When fetching the feed *now*, the twt starts like this and the current twt gets the hash 6mdqxrq:

(#yqke7sq) I've been sketching out some …

This can’t be avoided, really. Publishing twts and then editing them is like doing a git push --force after rewriting the commit history. Chaos will ensue. 😅
Weird, this just happened. It went away on page reload.

Screenshot of Yarn.social showing duplicated headers.
@prologic it hasn't been solved, that's why we are here on this yarn, still. LOL. I believe the hash mismatch is happening because of an edited twtxt. I don't follow the OP, so I have no way to check (not that I am certain it could be possible), but I have seeing similar issues in the past as a result of an edit. That's one of the reasons I don't edit anymore. 😬
@prologic I tried hosting my own Matrix server once, I got wrecked! 🤣 and I believe that experience established a good base line for me to avoid self-hosting anything federated (except for a TWTX feed).

Skill issues aside (since I'm willing to learn), my internet speed is a huge limitation (I have less than 1Mb Up) 🫠 Then again, there's the Running stuff off of my RPI's SD Card 😅 that's just asking for trouble.

Seriously, I should get a proper Job, at least I'll be able to afford my curiosities 🙃
@prologic I tried hosting my own Matrix server once, I got wrecked! 🤣 and I believe that experience established a good base line for me to avoid self-hosting anything federated (except for a TWTX feed).

Skill issues aside (since I'm willing to learn), my internet speed is a huge limitation (I have less than 1Mb Up) 🫠 Then again, there's the Running stuff off of my RPI's SD Card 😅 that's just asking for trouble.

Seriously, I should get a proper Job, at least I'll be able to afford my curiosities 🙃
@prologic I tried hosting my own Matrix server once, I got wrecked! 🤣 and I believe that experience established a good base line for me to avoid self-hosting anything federated (except for a TWTX feed).

Skill issues aside (since I'm willing to learn), my internet speed is a huge limitation (I have less than 1Mb Up) 🫠 Then again, there's the Running stuff off of my RPI's SD Card 😅 that's just asking for trouble.

Seriously, I should get a proper Job, at least I'll be able to afford my curiosities 🙃
[47°09′17″S, 126°43′32″W] --bad checksum--
@falsifian You are totally right. The specs are at least "open enough" for us to consider that as an implementation detail. We, and by we I mean @movq @lyse @bender @xuu and others should discuss this in more detail I believe and try to see if we can agree on what we're trying to solve.

> Does yarnd provide an API for finding twts? Is it similar?

No, it doesn't. But yarns (_the search engine/crawler wrote_) seems more fitting here. It's been discussed before, the possibility of building a "Twtxt Register v1" compatible API for yarns. I _think_ a search engine + crawler + registry (_especially ones that can form a bit of a "distributed network_) are far more useful I _think_ in order to support the _actual_ decentralised Twtxt / Yarn ecosystem (_which is how I prefer to describe it_).
@falsifian You are totally right. The specs are at least "open enough" for us to consider that as an implementation detail. We, and by we I mean @movq @lyse @bender @xuu and others should discuss this in more detail I believe and try to see if we can agree on what we're trying to solve.

> Does yarnd provide an API for finding twts? Is it similar?

No, it doesn't. But yarns (_the search engine/crawler wrote_) seems more fitting here. It's been discussed before, the possibility of building a "Twtxt Register v1" compatible API for yarns. I _think_ a search engine + crawler + registry (_especially ones that can form a bit of a "distributed network_) are far more useful I _think_ in order to support the _actual_ decentralised Twtxt / Yarn ecosystem (_which is how I prefer to describe it_).
@falsifian Ahh but this is solved now with the new single shot fetch?
@falsifian Ahh but this is solved now with the new single shot fetch?
[47°09′34″S, 126°43′43″W] Re-taking samples
https://www.mattlayman.com/blog/2024/layman-guide-python-built-in-functions/
[47°09′13″S, 126°43′10″W] Transfer aborted
The actual end-user problem is that I can't see the thread properly when using neomutt+jenny.
@prologic One of your twts begins with (#st3wsda): https://twtxt.net/twt/bot5z4q

Based on the twtxt.net web UI, it seems to be in reply to a twt by @cuaxolotl which begins "I’ve been sketching out...".

But jenny thinks the hash of that twt is 6mdqxrq. At least, there's a very twt in their feed with that hash that has the same text as appears on yarn.social (except with ' instead of ’).

Based on this, it appears jenny and yarnd disagree about the hash of the twt, or perhaps the twt was edited (though I can't see any difference, assuming ' vs ’ is just a rendering choice).
@prologic I believe you when you say registries as designed today do not crawl. But when I first read the spec, it conjured in my mind a search engine. Now I don't know how things work out in practice, but just based on reading, I don't see why it can't be an API for a crawling search engine. (In fact I don't see anything in the spec indicating registry servers shouldn't crawl.)

(I also noticed that https://twtxt.readthedocs.io/en/latest/user/registry.html recommends "The registries should sync each others user list by using the users endpoint". If I understood that right, registering with one should be enough to appear on others, even if they don't crawl.)

Does yarnd provide an API for finding twts? Is it similar?
@bender I've sort of lost the plot here a bit 🤦‍♂️ What's the problem we're trying to figure out? 🤔
@bender I've sort of lost the plot here a bit 🤦‍♂️ What's the problem we're trying to figure out? 🤔
@prologic why would you think it is changed in jenny? Falsifian (I still can’t mention while on mobile) said jenny, and the manual calculation match. Yarn seems to be the one at odd.