# I am the Watcher. I am your guide through this vast new twtiverse.
# 
# Usage:
#     https://watcher.sour.is/api/plain/users              View list of users and latest twt date.
#     https://watcher.sour.is/api/plain/twt                View all twts.
#     https://watcher.sour.is/api/plain/mentions?uri=:uri  View all mentions for uri.
#     https://watcher.sour.is/api/plain/conv/:hash         View all twts for a conversation subject.
# 
# Options:
#     uri     Filter to show a specific users twts.
#     offset  Start index for quey.
#     limit   Count of items to return (going back in time).
# 
# twt range = 1 196319
# self = https://watcher.sour.is?offset=167599
# next = https://watcher.sour.is?offset=167699
# prev = https://watcher.sour.is?offset=167499
@prologic Yep. Doesn't matter if JS in turned on or not. So it is somewhere hiding in the Go core. Some replacement going berserk, I'd say.

It happens to each bracketed text individually: https://lyse.isobeef.org/tmp/bracketed-text/triple.png

But then the question still is, why on earth does it happen to old twts, too? I'm getting into my code excavator.
It's also (_expectedly_) in the feed file on disk:


2024-08-04T21:22:05+10:00\t[foo][foo=][foo][foo=]
It's also (_expectedly_) in the feed file on disk:


2024-08-04T21:22:05+10:00	[foo][foo=][foo][foo=]
It's also (_expectedly_) in the feed file on disk:


2024-08-04T21:22:05+10:00	[foo][foo=][foo][foo=]
@lyse Holy fucking shit! You're right! You got me out of bed for this one, I spun my local dev instance and entered a Twt with [foo] and ended up with [foo][foo=][foo][foo=] wut da actual fuq?! 🤔
@lyse Holy fucking shit! You're right! You got me out of bed for this one, I spun my local dev instance and entered a Twt with [foo] and ended up with [foo][foo=][foo][foo=] wut da actual fuq?! 🤔
The reason I think this is some kind of attack is based on the repeated content and some of its uniqueness 🤔 This is so uncharacteristic if both victims 🤔
The reason I think this is some kind of attack is based on the repeated content and some of its uniqueness 🤔 This is so uncharacteristic if both victims 🤔
@stigatle Nice one 🥳 Kooking really good! 👌
@stigatle Nice one 🥳 Kooking really good! 👌
@lyse I'm not sure this is a bug to be honest? What possible code could cause this?! 🤔
@lyse I'm not sure this is a bug to be honest? What possible code could cause this?! 🤔
Righto, @prologic, I just checkout out current main of yarnd (commit 5101ec240ddb0e5e39809bf8a7b847508b3ac298) and ran make dev. After registering a user and logging in, I then entered a twt with double bracketed text (without the equal sign on the second one, though) and it was expanded into eight brackets. So, this is clearly a bug. Let me dig deeper.

I hope I zoomed in enough, so you can read the stuff on my screenshot: https://lyse.isobeef.org/tmp/bracketed-text/bug.png
@lyse Thank you! 🙏
@lyse Thank you! 🙏
[47°09′19″S, 126°43′41″W] Working impossible due to thunderstorm
@prologic To clarify, I meant some kind of a cache poisoning attack using the gossipping mechanism to inject garbage on purpose. Not hijacked user accounts.

However, since this all relates to bracketed text, I do not find an attack of some sort very likely. It's probably just a bug somewhere.
Media upload works, light\dark theme enabled. Tested it on debian\windows - works out of the box, statusbar moved to bottom for cleaner UI. Next is working more on ui when it refreshes the timelines. .
Media upload works, light\\dark theme enabled. Tested it on debian\\windows - works out of the box, statusbar moved to bottom for cleaner UI. Next is working more on ui when it refreshes the timelines. .
@prologic Here's an attempt at an analysis: https://lyse.isobeef.org/tmp/bracketed-text/

I just set up a cronjob to fetch and analyze both feeds every six hours. I probably have to do some dedup, otherwise the list gets out of handy rather quickly.
@lyse Yes

> How do the feeds look on disk? Do they already contain this bracketed text?

Because the handle just serves the Twtxt file directly.
@lyse Yes

> How do the feeds look on disk? Do they already contain this bracketed text?

Because the handle just serves the Twtxt file directly.
@bender / @mckinley could you both please change your password immediately? I will also work on some other security hardening that I have a hunch about, but will not publicize for now.
@bender / @mckinley could you both please change your password immediately? I will also work on some other security hardening that I have a hunch about, but will not publicize for now.
Hmm I see it! It's so obvious 🤦‍♂️ I smell an attack of some kind.
Hmm I see it! It's so obvious 🤦‍♂️ I smell an attack of some kind.
@lyse No problems! Is it stils in the list when I redo this search, it'll be gone 😅
@lyse No problems! Is it stils in the list when I redo this search, it'll be gone 😅
@lyse This is really weird. Do you have an example of this bracketed text? Re peers, I'm aware of all the peers, nothing surprising there.
@lyse This is really weird. Do you have an example of this bracketed text? Re peers, I'm aware of all the peers, nothing surprising there.
Running from Little Debbie: 12.20 miles, 00:10:43 average pace, 02:10:47 duration
was hoping for a breeze or some rain from this tropical depression "debbie" but saw none of that until i had stopped. broke it up in 4 mile blocks and walked a bit to regain composure/heart rate. had some at the end with speed. funny how the only people i saw out today were runners.
#running
Running from Little Debbie: 12.20 miles, 00:10:43 average pace, 02:10:47 duration
was hoping for a breeze or some rain from this tropical depression "debbie" but saw none of that until i had stopped. broke it up in 4 mile blocks and walked a bit to regain composure/heart rate. had some at the end with speed. funny how the only people i saw out today were runners.
#running
Running from Little Debbie: 12.20 miles, 00:10:43 average pace, 02:10:47 duration
was hoping for a breeze or some rain from this tropical depression "debbie" but saw none of that until i had stopped. broke it up in 4 mile blocks and walked a bit to regain composure/heart rate. had some at the end with speed. funny how the only people i saw out today were runners.
#running
And now, @bender's feed changed, too. Bracketed text got duplicated once again.

How do the feeds look on disk? Do they already contain this bracketed text?

For reference, I just placed a copy of the feed here: https://lyse.isobeef.org/tmp/bracketed-text/bender-2024-08-04-10-34.txt

I haven't marked the changed twts by @mckinley as read last time, so I don't know if something changed there as well. In any case, current snapshot: https://lyse.isobeef.org/tmp/bracketed-text/mckinley-2024-08-04-10-39.txt

Yarnd gossipping might be the reason for the truncated stuff, @prologic. Who are your peers? Any obvious broken yarnd version or even some kind of an attacker involved? But there must be something else broken in yarnd for the bracketed text to be duplicated.
@bender This one had me laugh real hard! :'-D Well done, mate.
@prologic Ah, I already forgot that I had a backup user Let's get rid of this guy. :-)
[47°09′12″S, 126°43′43″W] Working impossible due to blizzard
https://www.teotimepacreau.fr/blog/modelisation-base-de-donnees/ #sql
@prologic In that screenshot (https://twtxt.net/media/7c3rEWveU64SAxrXZ6CDYS.png), all the bracketed stuff is duplicated again, compared to lyse’s original twt. I suspect that’s the cause for the changed hash.

I could not reproduce this by manually duplicating those text areas in lyse’s twt. I end up with the hash pjdciga instead, but I probably mistyped something.
@prologic In that screenshot (https://twtxt.net/media/7c3rEWveU64SAxrXZ6CDYS.png), all the bracketed stuff is duplicated again, compared to lyse’s original twt. I suspect that’s the cause for the changed hash.

I could not reproduce this by manually duplicating those text areas in lyse’s twt. I end up with the hash pjdciga instead, but I probably mistyped something.
@prologic In that screenshot (https://twtxt.net/media/7c3rEWveU64SAxrXZ6CDYS.png), all the bracketed stuff is duplicated again, compared to lyse’s original twt. I suspect that’s the cause for the changed hash.

I could not reproduce this by manually duplicating those text areas in lyse’s twt. I end up with the hash pjdciga instead, but I probably mistyped something.
@prologic In that screenshot (https://twtxt.net/media/7c3rEWveU64SAxrXZ6CDYS.png), all the bracketed stuff is duplicated again, compared to lyse’s original twt. I suspect that’s the cause for the changed hash.

I could not reproduce this by manually duplicating those text areas in lyse’s twt. I end up with the hash pjdciga instead, but I probably mistyped something.
[47°09′37″S, 126°43′23″W] Wind speed: 43kph -- batteries low
@prologic Still expands to _almost_ the correct raw twt, though: https://movq.de/v/c6243a9e61/s.png
@prologic Still expands to _almost_ the correct raw twt, though: https://movq.de/v/c6243a9e61/s.png
@prologic Still expands to _almost_ the correct raw twt, though: https://movq.de/v/c6243a9e61/s.png
@prologic Still expands to _almost_ the correct raw twt, though: https://movq.de/v/c6243a9e61/s.png
🧮 USERS:1 FEEDS:2 TWTS:1051 ARCHIVED:77202 CACHE:2435 FOLLOWERS:17 FOLLOWING:14
@prologic, what makes your mention of my handle show differently like this?

@mckinley That's actually all I used it for myself 👌 All those other "bells 'n whistles" are really just Traefik supporting lots of alternate setups and drivers for discovery, etc.
@mckinley That's actually all I used it for myself 👌 All those other "bells 'n whistles" are really just Traefik supporting lots of alternate setups and drivers for discovery, etc.
@bender LOL 🤣
@bender LOL 🤣
@bender LOL 🤣
@bender LOL 🤣
@prologic no worries! It pains me to find bugs in Yarn, though. I want it to be flawless, you know, like Microsoft Windows. 🤭
@mckinley Caddy is simpler and act as both, web server *and* a reverse proxy. Traefik is only—albeit on steroids—a reverse proxy.
@mckinley it is opt-in because all your devices logged into the same iCloud account need to be at a compatible level. You also have to have a designated recovery account member which, obviously, you need to manually add.
@prologic ah, fuck it, don’t worry. I consider one the original (I flip a coin to pick which), and the other’s a backup, just in case. 😂

$ wc -l inactive.txt
152 inactive.txt

$ wc -l inactive.txt
152 inactive.txt
👋 At some point over the next day or two I will be deleting the following feeds/accounts:

https://gist.mills.io/prologic/ae61ae2bfba6401e8955a33394fd858b

If anyone spots anything on this list that shouldn't be deleted, please let me know! 🙏
👋 At some point over the next day or two I will be deleting the following feeds/accounts:

https://gist.mills.io/prologic/ae61ae2bfba6401e8955a33394fd858b

If anyone spots anything on this list that shouldn't be deleted, please let me know! 🙏
@lyse@
@lyse@
We'll kind of the backend fixes it or grid to 🤣
We'll kind of the backend fixes it or grid to 🤣
@movq@ does not hmmm
@movq@ does not hmmm
@prologic works
@prologic works
The mobile autocomplete bug is something I can reproduce and likely fix soon™ -- I _think_ its happenning because I accidentally nuked this pod's cache the other day (sorry!) 😢 -- But it is also a bug 🐛
The mobile autocomplete bug is something I can reproduce and likely fix soon™ -- I _think_ its happenning because I accidentally nuked this pod's cache the other day (sorry!) 😢 -- But it is also a bug 🐛
Like what was this meant to be anyway?


"[Scheduled][Scheduled=][Scheduled][Scheduled=][Scheduled][Scheduled=][Scheduled][Scheduled=]"
Like what was this meant to be anyway?


"[Scheduled][Scheduled=][Scheduled][Scheduled=][Scheduled][Scheduled=][Scheduled][Scheduled=]"
As for @mckinley 's odd Twt, I only see one instance of this:


2023-01-09T22:42:37Z	(#dusjj6a) @<lyse https://lyse.isobeef.org/twtxt.txt> As far as I know, they're still visible in the Web UI. Although, in the mobile app and youtube.com, I believe it tells you that the video isn't available without having to click on it. They don't tell you that in the RSS feed, and I agree; it gets annoying.

If we had a custom feed generator that hooks directly into the YouTube API, I'll bet we could find that information and put "[Scheduled][Scheduled=][Scheduled][Scheduled=][Scheduled][Scheduled=][Scheduled][Scheduled=]" in the title for premieres and remove it when the video is available.


And I have no fucking clue how this happened. I can't imagine anything in the yarnd codebase would be responsible for this weirdness 🤣
As for @mckinley 's odd Twt, I only see one instance of this:


2023-01-09T22:42:37Z	(#dusjj6a) @<lyse https://lyse.isobeef.org/twtxt.txt> As far as I know, they're still visible in the Web UI. Although, in the mobile app and youtube.com, I believe it tells you that the video isn't available without having to click on it. They don't tell you that in the RSS feed, and I agree; it gets annoying.

If we had a custom feed generator that hooks directly into the YouTube API, I'll bet we could find that information and put "[Scheduled][Scheduled=][Scheduled][Scheduled=][Scheduled][Scheduled=][Scheduled][Scheduled=]" in the title for premieres and remove it when the video is available.


And I have no fucking clue how this happened. I can't imagine anything in the yarnd codebase would be responsible for this weirdness 🤣
As for @mckinley 's odd Twt, I only see one instance of this:


2023-01-09T22:42:37Z\t(#dusjj6a) @<lyse https://lyse.isobeef.org/twtxt.txt> As far as I know, they're still visible in the Web UI. Although, in the mobile app and youtube.com, I believe it tells you that the video isn't available without having to click on it. They don't tell you that in the RSS feed, and I agree; it gets annoying.

If we had a custom feed generator that hooks directly into the YouTube API, I'll bet we could find that information and put "[Scheduled][Scheduled=][Scheduled][Scheduled=][Scheduled][Scheduled=][Scheduled][Scheduled=]" in the title for premieres and remove it when the video is available.


And I have no fucking clue how this happened. I can't imagine anything in the yarnd codebase would be responsible for this weirdness 🤣
@mckinley Nah it wasn't me, trust me 🤣 I _actually_ use Traefik for my ingres.
@mckinley Nah it wasn't me, trust me 🤣 I _actually_ use Traefik for my ingres.
I don't think I'm smart enough to figure this out 😅
I don't think I'm smart enough to figure this out 😅
I can't explain this. I'm leaning towards a peering pod being responsible for producing a different hash, and twtxt.net pulling that in from a peer. But that would only happen if my pod doesn't have the Root Twt ans asked its peers for it. And that implies other pods are producing incorrect/different hashes "somehow". So all of that seems highly unlikely tbh.
I can't explain this. I'm leaning towards a peering pod being responsible for producing a different hash, and twtxt.net pulling that in from a peer. But that would only happen if my pod doesn't have the Root Twt ans asked its peers for it. And that implies other pods are producing incorrect/different hashes "somehow". So all of that seems highly unlikely tbh.
bsormva is not a hash found in @lyse 's feed at all according to yarnc debug which is printing the hash and corresponding Twt per line.
bsormva is not a hash found in @lyse 's feed at all according to yarnc debug which is printing the hash and corresponding Twt per line.
That is this one:


ta6uu5q 2024-08-03T19:30:00+02:00	(#puxvjcq) Hmmm, what is going on here? ...
That is this one:


ta6uu5q 2024-08-03T19:30:00+02:00	(#puxvjcq) Hmmm, what is going on here? ...
That is this one:


ta6uu5q 2024-08-03T19:30:00+02:00\t(#puxvjcq) Hmmm, what is going on here? ...
A equivalent yarnc debug <url> only sees the 2nd hash
A equivalent yarnc debug <url> only sees the 2nd hash
Computers aren't meant to give me three different answers 🤣
Computers aren't meant to give me three different answers 🤣
@movq / @lyse / @xuu any ideas wut da fuq is going on here?! 🤣
@movq / @lyse / @xuu any ideas wut da fuq is going on here?! 🤣
In fact I cannot produce eitehr of these hashes:


$ pbpaste | ./yarnc hash -u https://lyse.isobeef.org/twtxt.txt -t 2024-08-03T19:30:00+02:00 -
bsormva


What da fuq?!


$ bat https://twtxt.net/twt/7hraijq | jq -r '.text' | ./yarnc hash -u https://lyse.isobeef.org/twtxt.txt -t 2024-08-03T19:30:00+02:00 -
bsormva
In fact I cannot produce eitehr of these hashes:


$ pbpaste | ./yarnc hash -u https://lyse.isobeef.org/twtxt.txt -t 2024-08-03T19:30:00+02:00 -
bsormva


What da fuq?!


$ bat https://twtxt.net/twt/7hraijq | jq -r '.text' | ./yarnc hash -u https://lyse.isobeef.org/twtxt.txt -t 2024-08-03T19:30:00+02:00 -
bsormva
Yeah, this looks like a hash collision to me right? Same twt, same timestamp, same twter, produces two different hashes? I'm not even sure how da fuq this is even possible?


$ diff <(bat https://twtxt.net/twt/7hraijq | jq '.') <(bat https://twtxt.net/twt/ta6uu5q | jq '.')
10c10
<   "hash": "7hraijq",
---
>   "hash": "ta6uu5q",
Yeah, this looks like a hash collision to me right? Same twt, same timestamp, same twter, produces two different hashes? I'm not even sure how da fuq this is even possible?


$ diff <(bat https://twtxt.net/twt/7hraijq | jq '.') <(bat https://twtxt.net/twt/ta6uu5q | jq '.')
10c10
<   "hash": "7hraijq",
---
>   "hash": "ta6uu5q",
@lyse (_ahh auto-complete is broken only on Mobile?_) @xuu is our hashing reached a point where it's broken and needs to be dumped? 🤔_