# I am the Watcher. I am your guide through this vast new twtiverse.
# 
# Usage:
#     https://watcher.sour.is/api/plain/users              View list of users and latest twt date.
#     https://watcher.sour.is/api/plain/twt                View all twts.
#     https://watcher.sour.is/api/plain/mentions?uri=:uri  View all mentions for uri.
#     https://watcher.sour.is/api/plain/conv/:hash         View all twts for a conversation subject.
# 
# Options:
#     uri     Filter to show a specific users twts.
#     offset  Start index for quey.
#     limit   Count of items to return (going back in time).
# 
# twt range = 1 196296
# self = https://watcher.sour.is?offset=166956
# next = https://watcher.sour.is?offset=167056
# prev = https://watcher.sour.is?offset=166856
FWIW I'm still trying to find the the cause of the mult-GB avatars that both @stigatle and @abucci 's pods were both teying yo download. The flaw has since been fixed in the code but I'm still trying to investigate the source 🤞
FWIW I'm still trying to find the the cause of the mult-GB avatars that both @stigatle and @abucci 's pods were both teying yo download. The flaw has since been fixed in the code but I'm still trying to investigate the source 🤞
@bender Hehe 🤣
@bender Hehe 🤣
🧮 USERS:1 FEEDS:2 TWTS:1042 ARCHIVED:77001 CACHE:2369 FOLLOWERS:17 FOLLOWING:14
@prologic somebody is playing with a new toy.
@prologic people went to sleep? I mean, at 03:00 I am on my second, or third sleep. 😜

Oh, AEST! No idea, but it might have been @xuu. LOL.
Hmmm something happened last night at ~3am (AEST) that decrased traffic to my pod quite considerably... Hmmm? Anyone have any ideas? 💡 ~
Hmmm something happened last night at ~3am (AEST) that decrased traffic to my pod quite considerably... Hmmm? Anyone have any ideas? 💡 ~
@xuu wow, such a remote place, eh? I imagine you living somewhere in the Himalayas, driving to town once every couple of months to resupply.

Don’t get me wrong, it is attractive, until my Internets are threaten. Then all bets are off. 😂
@lyse lovely! I would have picked this one as a cover.

Bloody sunset
On my blog: Real Life in Star Trek, Hero Worship https://john.colagioia.net/blog/2024/07/25/hero-worship.html #scifi #startrek #closereading
@lyse But stuff is still “mostly usable”, isn’t it? It’s not like it became impossible to write a letter because everything has gotten so slow.

That’s what I meant by “absolute” performance: A human being tolerates a system boot up time of 0.5-2 minutes, for example, so there’s an absolute/fixed duration that any task is allowed to take. Boot: 0.5-2 minutes. Opening Word: 1-10 seconds. Saving an image file: 1-10 seconds. Time until the next song starts to play when you click “next track”: 0-5 seconds. Stuff like that. As long as we don’t exceed those durations, people will be more or less happy.

Wasted potential? Ab-so-fucken-lutely.

(Maybe I’m repeating myself. I’m tired. Sorry. 😅)
@lyse But stuff is still “mostly usable”, isn’t it? It’s not like it became impossible to write a letter because everything has gotten so slow.

That’s what I meant by “absolute” performance: A human being tolerates a system boot up time of 0.5-2 minutes, for example, so there’s an absolute/fixed duration that any task is allowed to take. Boot: 0.5-2 minutes. Opening Word: 1-10 seconds. Saving an image file: 1-10 seconds. Time until the next song starts to play when you click “next track”: 0-5 seconds. Stuff like that. As long as we don’t exceed those durations, people will be more or less happy.

Wasted potential? Ab-so-fucken-lutely.

(Maybe I’m repeating myself. I’m tired. Sorry. 😅)
@lyse But stuff is still “mostly usable”, isn’t it? It’s not like it became impossible to write a letter because everything has gotten so slow.

That’s what I meant by “absolute” performance: A human being tolerates a system boot up time of 0.5-2 minutes, for example, so there’s an absolute/fixed duration that any task is allowed to take. Boot: 0.5-2 minutes. Opening Word: 1-10 seconds. Saving an image file: 1-10 seconds. Time until the next song starts to play when you click “next track”: 0-5 seconds. Stuff like that. As long as we don’t exceed those durations, people will be more or less happy.

Wasted potential? Ab-so-fucken-lutely.

(Maybe I’m repeating myself. I’m tired. Sorry. 😅)
@lyse But stuff is still “mostly usable”, isn’t it? It’s not like it became impossible to write a letter because everything has gotten so slow.

That’s what I meant by “absolute” performance: A human being tolerates a system boot up time of 0.5-2 minutes, for example, so there’s an absolute/fixed duration that any task is allowed to take. Boot: 0.5-2 minutes. Opening Word: 1-10 seconds. Saving an image file: 1-10 seconds. Time until the next song starts to play when you click “next track”: 0-5 seconds. Stuff like that. As long as we don’t exceed those durations, people will be more or less happy.

Wasted potential? Ab-so-fucken-lutely.

(Maybe I’m repeating myself. I’m tired. Sorry. 😅)
@lyse Uhh, nice. Haven’t seen a sunset like that in a while, I think. 🤔
@lyse Uhh, nice. Haven’t seen a sunset like that in a while, I think. 🤔
@lyse Uhh, nice. Haven’t seen a sunset like that in a while, I think. 🤔
@lyse Uhh, nice. Haven’t seen a sunset like that in a while, I think. 🤔
@bender haha funny! though i just realized my ISP is the only one with fiber pulled to the property so i would have to get a phone line from them some how. The other ISP in the area is basically a mobile hotspot.
@bender haha funny! though i just realized my ISP is the only one with fiber pulled to the property so i would have to get a phone line from them some how. The other ISP in the area is basically a mobile hotspot.
[47°09′19″S, 126°43′18″W] --bad checksum--
Xuu has shutdown his pod now, probably to avoid losing connectivity.
@xuu need a modem? I got a couple of 33.6K, and one 28.8K. :-P
@abucci Can you please check if you reported @xuu's IP address 162.211.155.2 on mistake and let his ISP know that this was some false alarm? They're monitoring his traffic to your server and treating this as continued abuse. :-(

(His twts have been synced to your yarnd by yarnd's gossip protocol.)
> We received the abuse report below regarding network abuse from the IP address indicated.
> On researching I see that HTTPS (tcp 443) traffic is continuing and originating from you NAT IP address 100.64.x.x
> This was further found to be originating from your firewall/router at 192.168.x.x (MAC D8:58:D7:x:x:x).
> This abuse is continuing and constitues a violation of \n Acceptable Use Policy and Terms of Service.
> Please take action to identify the source of the abuse and prevent it from continuing.
> Failure to stop the abuse may result in suspension or cancellation of service.
>
> Thank you,
> We received the abuse report below regarding network abuse from the IP address indicated.
> On researching I see that HTTPS (tcp 443) traffic is continuing and originating from you NAT IP address 100.64.x.x
> This was further found to be originating from your firewall/router at 192.168.x.x (MAC D8:58:D7:x:x:x).
> This abuse is continuing and constitues a violation of Senawave's Acceptable Use Policy and Terms of Service.
> Please take action to identify the source of the abuse and prevent it from continuing.
> Failure to stop the abuse may result in suspension or cancellation of service.
>
> Thank you,
> We received the abuse report below regarding network abuse from the IP address indicated.
> On researching I see that HTTPS (tcp 443) traffic is continuing and originating from you NAT IP address 100.64.x.x
> This was further found to be originating from your firewall/router at 192.168.x.x (MAC D8:58:D7:x:x:x).
> This abuse is continuing and constitues a violation of [ISP] Acceptable Use Policy and Terms of Service.
> Please take action to identify the source of the abuse and prevent it from continuing.
> Failure to stop the abuse may result in suspension or cancellation of service.
>
> Thank you,
> We received the abuse report below regarding network abuse from the IP address indicated.
> On researching I see that HTTPS (tcp 443) traffic is continuing and originating from you NAT IP address 100.64.x.x
> This was further found to be originating from your firewall/router at 192.168.x.x (MAC D8:58:D7:x:x:x).
> This abuse is continuing and constitues a violation of [ISP] Acceptable Use Policy and Terms of Service.
> Please take action to identify the source of the abuse and prevent it from continuing.
> Failure to stop the abuse may result in suspension or cancellation of service.
>
> Thank you,
@xuu wow, not cool.
he emailed my ISP about causing logging abuse. This is the only real ISP in my area, its gonna basically send me back to dialup.
he emailed my ISP about causing logging abuse. This is the only real ISP in my area, its gonna basically send me back to dialup.
@xuu For what reason?
@abucci Just making sure you're seeing @xuu's twt, in case he's still on your blacklist:

> Hey so.. i just got an email from my ISP saying they will terminate my service. Did i break something @abucci ?
>
> – https://txt.sour.is/twt/oohzbqa
Hey so.. i just got an email from my ISP saying they will terminate my service. Did i break something @abucci ?
Hey so.. i just got an email from my ISP saying they will terminate my service. Did i break something @abucci ?
@abucci No worries! All in the name of better reliability and security 😅
@abucci No worries! All in the name of better reliability and security 😅
@stigatle Thanks! Sooo cold 🥶
@stigatle Thanks! Sooo cold 🥶
@stigatle no problems 👌 one problem solved at least 🤣
@prologic sleep well!
@stigatle no problems 👌 one problem solved at least 🤣
Anyway, I'm gonna have to go to bed... We'll continue this on the weekend. Still trying to hunt down some kind of suspected mult-GB avatar using @stigatle 's pod's cache:


$ (echo "URL Bytes"; sort -n -k 2 -r < avatars.txt | head) | column -t
URL                                                                                                       Bytes
https://birkbak.neocities.org/avatar.jpg                                                                  667640
https://darch.neocities.org/avatar.png                                                                    652960
http://darch.dk/avatar.png                                                                                603210
https://social.naln1.ca/media/0c4f65a4be32ff3caf54efb60166a8c965cc6ac7c30a0efd1e51c307b087f47b.png        327947
...


But so far nothing much... Still running the search...
Anyway, I'm gonna have to go to bed... We'll continue this on the weekend. Still trying to hunt down some kind of suspected mult-GB avatar using @stigatle 's pod's cache:


$ (echo "URL Bytes"; sort -n -k 2 -r < avatars.txt | head) | column -t
URL                                                                                                       Bytes
https://birkbak.neocities.org/avatar.jpg                                                                  667640
https://darch.neocities.org/avatar.png                                                                    652960
http://darch.dk/avatar.png                                                                                603210
https://social.naln1.ca/media/0c4f65a4be32ff3caf54efb60166a8c965cc6ac7c30a0efd1e51c307b087f47b.png        327947
...


But so far nothing much... Still running the search...
@prologic @abucci my /tmp is fine now, no avatars there. I have to drive my daughter to a birthday party now, but I keep things running and I'll check when I get back.
Out of interest, are you able to block whole ASN(s)? I blocked the entirely of teh AWS and Facebook ASN(s) recently.
Out of interest, are you able to block whole ASN(s)? I blocked the entirely of teh AWS and Facebook ASN(s) recently.
@abucci Oh 🤣 Well my IP is a known subnet and static, so if you need to know what it is, Email me 😅
@abucci Oh 🤣 Well my IP is a known subnet and static, so if you need to know what it is, Email me 😅
@abucci Seems to be okay now hmmm
@abucci Seems to be okay now hmmm
@abucci Hmm I can see your twts on my pod now 🤔
@abucci Hmm I can see your twts on my pod now 🤔
@stigatle Sweet, thank you! I've been shooting myself in the foot over here and want to make sure the situation is getting fixed!
@abucci yeah I can see it :)
@stigatle @prologic testing 1 2 3 can either of you see this?
Hmm, I wonder if I banned too many IPs and caused these issues for myself 😆
twts are taking a very long time to post from yarn after the latest upgrade. Like a good 60 seconds.
@prologic I don't know if this is new, but I'm seeing:


Jul 25 16:01:17 buc yarnd[1921547]: time="2024-07-25T16:01:17Z" level=error msg="https://yarn.stigatle.no/user/stigatle/twtxt.txt: client.Do fail: Get \\"https://yarn.stigatle.no/user/stigatle/twtxt.txt\\": dial tcp 185.97.32.18:443: i/o timeout (Client.Timeout exceeded while awaiting headers)" error="Get \\"https://yarn.stigatle.no/user/stigatle/twtxt.txt\\": dial tcp 185.97.32.18:443: i/o timeout (Client.Timeout exceeded while awaiting headers)"


I no longer see twts from @stigatle at all.
[47°09′21″S, 126°43′24″W] Reading: 1.12 Sv
@prologic Have you been seeing any of my replies?
@abucci / @abucci Any interesting errors pop up in the server logs since the the flaw got fixed (_unbounded receieveFile()_)? 🤔
@abucci / @abucci Any interesting errors pop up in the server logs since the the flaw got fixed (_unbounded receieveFile()_)? 🤔
Hmmm 🧐


for url in $(jq -r '.Twters[].avatar' cache.json | sed '/^$/d' | grep -v -E '(twtxt.net|anthony.buc.ci|yarn.stigatle.no|yarn.mills.io)' | sort -u); do echo "$url $(curl -I -s -o /dev/null -w '%header{content-length}' "$url")"; done
...


😅 Let's see... 🤔
Hmmm 🧐


for url in $(jq -r '.Twters[].avatar' cache.json | sed '/^$/d' | grep -v -E '(twtxt.net|anthony.buc.ci|yarn.stigatle.no|yarn.mills.io)' | sort -u); do echo "$url $(curl -I -s -o /dev/null -w '%header{content-length}' "$url")"; done
...


😅 Let's see... 🤔
It shows up in my twtxt feed so that's good.
@movq My issue is, now that we have the chance of getting something fast, people artificially slow it down again. Wether they think it's cool that they added some slow animation or just lack of knowledge or whatever. The absolute performance does not translate to the relative performance that I observe. Completely wasted potential. :-(

In today's economy, nobody optimizes something if it can be just called good enough with the next generation hardware. That's especially the mindset of big coorporations.

Anyway, getting sidetracked from the original post. :-)
@prologic will do, thanks for the tip!
This is a test. I am not seeing twts from @stigatle and it seems like @prologic might not be seeing twts from me. Do people see this?
@prologic I am not seeing twts from @stigatle anymore. Are you seeing twts from me?
@stigatle The one you sent is fine. I'm inspecting it now. I'm just saying, do yourself a favor and nuke your pod's garbage cache 🤣 It'll rebuild automatically in a much more prestine state.
@stigatle The one you sent is fine. I'm inspecting it now. I'm just saying, do yourself a favor and nuke your pod's garbage cache 🤣 It'll rebuild automatically in a much more prestine state.
@prologic you want a new cache from me - or was the one I sent OK for what you needed?
That was also a source of abuse that also got plugged (_being able to fill up the cache with garbage data_)
That was also a source of abuse that also got plugged (_being able to fill up the cache with garbage data_)
Ooof


$ jq '.Feeds | keys[]' cache.json | wc -l
4402


If you both don't mind dropping your caches. I would recommend it. Settings -> Poderator Settings -> Refresh cache.
Ooof


$ jq '.Feeds | keys[]' cache.json | wc -l
4402


If you both don't mind dropping your caches. I would recommend it. Settings -> Poderator Settings -> Refresh cache.
Ooof


$ jq '.Feeds | keys[]' cache.json | wc -l
4402


If you both don't mind dropping your caches. I would recommend it. Settings -> Poderator Settings -> Reset cache.
@prologic

./tools/dump_cache.sh: line 8: bat: command not found
No Token Provided



I don't have bat on my VPS and there is no package for installing it. Is cat a reasonable alternate?
@prologic No worries, thanks for working on the fix for it so fast :)
@prologic Yup. Didn't regret climbing these three hundred odd meters of elevation. :-)
@stigatle Thank you! 🙏
@stigatle Thank you! 🙏
@prologic Try hitting this URL:

https://twtxt.net/external?nick=nosuchuser&uri=https://foo.com

Change nosuchuser to any phrase at all.

If you hit https://twtxt.net/external?nick=nosuchuser , you're given an error. If you hit that URL above with the uri parameter, you can a legitimate-looking page. I think that is a bug.
@prologic here you go:
https://drive.proton.me/urls/XRKQQ632SG#LXWehEZMNQWF
@stigatle Ta. I hope my theory is right 😅
@stigatle Ta. I hope my theory is right 😅
@prologic Hitting that URL returns a bunch of HTML even though there is no user named lovetocode999 on my pod. I think it should 404, and maybe with a delay, to discourage whatever this abuse is. Basically this can be used to DDoS a pod by forcing it to generate a hunch of HTML just by doing a bogus GET like this.
@prologic thank you. I run it now as you said, I'll get the files put somewhere shortly.
But just have a look at the yarnd server logs too. Any new interesting errors? 🤔 No more multi-GB tmp files? 🤔
But just have a look at the yarnd server logs too. Any new interesting errors? 🤔 No more multi-GB tmp files? 🤔
@stigatle You want to run backup_db.sh and dump_cache.sh They pipe JSON to stdout and prompt for your admin password. Example:


URL=<your_pod_url> ADMIN=<your_admin_user> ./tools/dump_cache.sh > cache.json
@stigatle You want to run backup_db.sh and dump_cache.sh They pipe JSON to stdout and prompt for your admin password. Example:


URL=<your_pod_url> ADMIN=<your_admin_user> ./tools/dump_cache.sh > cache.json
I'm seeing GETs like this over and over again:

"GET /external?nick=lovetocode999&uri=https://vuf.minagricultura.gov.co/Lists/Informacin%20Servicios%20Web/DispForm.aspx?ID=8375144 HTTP/1.1" 200 35861 17.077914ms


always to nick=lovetocode999, but with different uris. What are these calls?
@stigatle Worky, worky now! :-)

Mate, these are some really nice gems! What a stunning landscape. I love it. Holy cow, that wooden church looks really sick. Even though, I'm not a scroll guy and prefer simple, straight designs, I have to say, that the interior craftmanship is something to admire.
@prologic so, if I'm correct the dump tool made a pods.txt and a stats.txt file, those are the ones you want? or do you want the output that it spits out in the console window?
@prologic so, if I'm correct the dump tool made a pods.txt and a stats.txt file, those are the ones you want?
Just thinking out loud here... With that PR merged (_or if you built off that branch_), you _might_ hopefully see new errors popup and we might catch this problematic bad feed in the act? Hmmm 🧐