# I am the Watcher. I am your guide through this vast new twtiverse.
# 
# Usage:
#     https://watcher.sour.is/api/plain/users              View list of users and latest twt date.
#     https://watcher.sour.is/api/plain/twt                View all twts.
#     https://watcher.sour.is/api/plain/mentions?uri=:uri  View all mentions for uri.
#     https://watcher.sour.is/api/plain/conv/:hash         View all twts for a conversation subject.
# 
# Options:
#     uri     Filter to show a specific users twts.
#     offset  Start index for quey.
#     limit   Count of items to return (going back in time).
# 
# twt range = 1 60780
# self = https://watcher.sour.is?uri=https://twtxt.net/user/prologic/twtxt.txt&offset=60180
# next = https://watcher.sour.is?uri=https://twtxt.net/user/prologic/twtxt.txt&offset=60280
# prev = https://watcher.sour.is?uri=https://twtxt.net/user/prologic/twtxt.txt&offset=60080
πŸ“£ I'm going to try and restore a few accounts tonight 🀞
@andros Looking πŸ‘€ Thanks! πŸ™
@kate Fair enough! πŸ˜‚ Also a good approach, change the environment 🀣
@bender Ack πŸ™‡β€β™‚οΈ
@movq Good luck 🀞
@movq All lies 🀣
Definitely open to taking on users πŸ‘ŒI only have open registrations turned off because of spam accounts and my pod being the most popular amongst spammers 🀣
Fuck
Got anywhere in mind? 🧐
@xuueapl.me@eapl.me This is actually pretty cool 😎
Can you confirm the fix temporarily in browser before I make the CSS change? I'm rubbish at CSS 🀣
Ahhh! It's all Soren's fault 🀣


commit ea9eaaf3d3977701dcb84b927c77c4f921bdbf43
Author: sorenpeter <sorenpeter@noreply@mills.io>
Date:   Sat Sep 24 23:34:07 2022 +0000

    Replacing Pico.css with Simple.css (#990)

    Replacing pico.css with simple.css along with some small UI changes
@bender Didn't we fix this ages ago?! πŸ€¦β€β™‚οΈ
@bender Fair enough πŸ‘Œ
@bender It does right 🀣 (_no highlights though, will have to figure out how I refactor the display code to support that_)
Search syntax appears to be:


hello
"hello world"
hello AND world
hello OR world
hello NOT world
"this is a phrase"
@lyse I'm open to other suggestions 🀣 But _hopefully_ both adding the additional prompt, not allowing it to enter shell history and removing from my shell history prevents me from doing such silly things in haste by pressing ^R and using fuzzy search which if you type fast you sometimes get wrong πŸ˜‘
I had a day like that just today, in fact I've had one of those weeks, actually no wait, I've had one of those months! 🀯
FYI: I've re-opened up search for anonymous use. So things like this now work without having to have an account on this pod or login. πŸ‘Œ #search #twtxt
Fixed.
Confirmed. Fix inbound.
Is it just me or is there a display bug for "Yarn"(s) that are duplicating the root twt? πŸ€”
This is an example of what I _believe_ every SRE should master and whatever Post Incident Review (PIR) _should_ focus on. Where did the system fail. What are the missing or incomplete Safety Controls.
Then I cleaned up my shell history of all of the invocations I ever made of dkv rm ... to make sure I never ever have this so easily accessible in my shell history (^R):


$ awk '
  /^#/ { ts = $0; next }
  /^dkv rm/ { next }
  { if (ts) print ts; ts=""; print }
' ~/.bash_history > ~/.bash_history.tmp && mv ~/.bash_history.tmp ~/.bash_history && history -r
So I re-write this shell alias that I used all the time alias dkv="docker rm" to be a much safer shell function:


dkv() {
  if [[ "$1" == "rm" && -n "$2" ]]; then
    read -r -p "Are you sure you want to delete volume '$2'? [Y/n] " confirm
    confirm=${confirm:-Y}
    if [[ "$confirm" =~ ^[Yy]$ ]]; then
      # Disable history
      set +o history

      # Delete the volume
      docker volume rm "$2"

      # Re-enable history
      set -o history
    else
      echo "Aborted."
    fi
  else
    docker volume "$@"
  fi
}
This weekend (_as some of you may now_) I accidently nuke this Pod's entire data volume πŸ€¦β€β™‚οΈ What a disastrous incident 🀣 I decided instead of trying to restore from a 4-month old backup (_we'll get into why I hadn't been taking backups consistently later_), that we'd start a fresh! πŸ˜… Spring clean! 🧼 -- Anyway... One of the things I realised was I was missing a very critical Safety Controls in my own ways of working... I've now rectified this...
@bender Please remind me to fix this after I'm done with this cachet branch and it's merged 🀞
Okay that's a bug πŸ˜†
Does mentioning @prologicdoesnm.p.psf.lt not work? 🧐
@bender I know! πŸ˜‚ Thankfully I think I fixed most problematic bugs 🀞
@movq Just keep them for a while.
@bender Whoohoo! πŸ₯³
@lyse Very nice 😊
@lyse Haha yeah 🀣 And thanks I'll take a look but honestly it'll be fine πŸ˜†
@movq Nah it's okay 🀣 It would only be a partial restore anyway 😭
I need to get Peering working again on this branch! That will drag in many Twts Twts I now no longer have 😭
At least I've fixed many bugs with the new SQLiteCache 🀣
@aelaraji Sounds cool! 😎
@movq We'll recover just fine right ? πŸ˜† It's only 4 months worth of posts πŸ˜… Not like it's the end of the world πŸ˜‚
Oh well, let's just start over! 🀣
Oh well. I've gone and done it again! This time I've lost 4 months of data because for some reason I've been busy and haven't been taking backups of all the things I should be?! πŸ€” Farrrrk 🀬
FindTwter fixed
Mentions view fixed
@aelaraji That's pretty cool! 😎
@bender As you know it's very hard to have consistency in decentralised system. Not as hard in a distributed system, and easy in a centralised one haha 🀣
@bender Working?
@bender Testing forking?
@bender C'
B
A
@bender What do you mean? The way things are written hopefully hasn't changed at all 🀞
Works okay hmmm what's up?
@bender Like this?
woot fixed edits and deletes 🀣 Okay okay g'night (_fer realz now!_) 😴
One more update 🀣
@bender Okay πŸ‘
if someone can help me create a list of things that are either not working or behaving in an unexpected/indescribable way, please list them, in some order of priority, and I'll focus on fixing them tomorrow. G'night! 😴 (_leaving this pod on the highly experimental SqliteCache backend_)
I think yes
Do I add a function to delete a Twt if it was an Edit/Delete from the user?
Because this cache behaves very differently compared to the old cache. Feeds are never overwritten like they used to!
Fuck! 🀣 Yes this makes sense! Now you'll see edits too 😱
Wait...
Confirmed fix πŸ₯³
EDIT: Testing a fucked typo if I can
@bender Soonβ„’ πŸ”œ
Fixed. Rolling out shortly...
@bender Thanks I'll fix that!
@bender LOL πŸ˜‚ Bitcask is still the user database bahahaha πŸ˜…
@bender Of course 🀣
@bender Hah, gonna have to go to bed though I think. I hope no-one minds too badly that there are going to be bugs for and weird stuff for a bit. I'll try to fix more things tomorrow (Sunday). πŸ‘Œ
Pod feels a bit snapper for sure....
@bender Dunno 🀷 so far uptime is 4m so maybe not? 🀣
@bender Answered :D
@bender I did, but wasn't sure so I asked 🀣
@bender Backend is SQLite now. I switched drivers and so far so good. 'm seeing very stable CPU utilization, and a stable memory consumption of under 100MB
@bender As soon as the drop the notion of "old" twts, drop the re-fetch all feeds. Yes.
_some_ things _may_ be broken or not quite right. LMK! πŸ™
@bender Sort of, still fixing bugs. I had so much trouble with this sqlite driver, that in the end I swithced it to another driver and so far so good 😊 fingers crossed 🀞
Is this thing on?! 🀣
@kat Did you build the software for it? πŸ€”
Okay. let's see how things work now.... (_sorry for all the fucking around!_)
Hmmm
Okay we've survived for ~6m now (_beating our 4m numerous OOM kills_) 🀣 Also massive improvements in performance so far πŸ₯³~
Hmm I spoke to soonβ„’ :( πŸ™ Another OOM kill :/ πŸ˜•


 time="2025-04-05T03:26:12Z" level=info msg="[mem] Feeds fetch started: HeapAlloc=34MB Goroutines=31"
 time="2025-04-05T03:27:17Z" level=info msg="[mem] Feeds fetch done: HeapAlloc=67MB Goroutines=92"
 time="2025-04-05T03:29:23Z" level=info msg="[mem] Feeds fetch started: HeapAlloc=56MB Goroutines=41"
 time="2025-04-05T03:29:23Z" level=info msg="[mem] Feeds fetch done: HeapAlloc=49MB Goroutines=39"
 time="2025-04-05T03:30:15Z" level=info msg="[mem] Feeds fetch started: HeapAlloc=36MB Goroutines=41"
 time="2025-04-05T03:30:15Z" level=info msg="[mem] Feeds fetch done: HeapAlloc=54MB Goroutines=40"
 time="2025-04-05T03:31:07Z" level=info msg="[mem] Feeds fetch started: HeapAlloc=53MB Goroutines=41"
 time="2025-04-05T03:31:21Z" level=info msg="[mem] Feeds fetch started: HeapAlloc=65MB Goroutines=95"
 time="2025-04-05T03:31:21Z" level=info msg="[mem] Feeds fetch done: HeapAlloc=69MB Goroutines=91"


Hmmm? πŸ€”
We're also running on the new SQLiteCache πŸ‘Œ
@xuu That last commit might have fixed the memory problem 🀞
Seem to be holding at 100-120MB RSS πŸ™
Trump is a fool !
@bender Holy moly! 🀯
@bender Haha 🀣
I'll fix this soon πŸ˜‚
@andros DRM is a solution to the wrong problem.
@bender So true πŸ˜₯
@javivf Welcome indeed πŸ€—
@eldersnake Same!
@andros That's a shame. But here's the thing about piracy. If I could download a PDF or MP3/OGG of a Book I'd happily pay for this. If I could download an MP3/OGG of some Music I would pay for this. If I could download an MP4 of a Movie I would pay for this.

The reason piracy exists at all is economics and greed. If you make something affordable and convenient, there's no need to steal.
Okay trying SqliteCache again with a slight tweak. I _might_ have to rethink the hard-coded LIMIT 50.
Going back to old MemoryCache for now and going to bed πŸ›Œ
Hmmm this needs a bit more work.
And we're back. On the new Sqlitecache πŸ₯³