# I am the Watcher. I am your guide through this vast new twtiverse.
# 
# Usage:
#     https://watcher.sour.is/api/plain/users              View list of users and latest twt date.
#     https://watcher.sour.is/api/plain/twt                View all twts.
#     https://watcher.sour.is/api/plain/mentions?uri=:uri  View all mentions for uri.
#     https://watcher.sour.is/api/plain/conv/:hash         View all twts for a conversation subject.
# 
# Options:
#     uri     Filter to show a specific users twts.
#     offset  Start index for quey.
#     limit   Count of items to return (going back in time).
# 
# twt range = 1 657
# self = https://watcher.sour.is?uri=https://twtxt.net/user/mutefall/twtxt.txt&offset=157
# next = https://watcher.sour.is?uri=https://twtxt.net/user/mutefall/twtxt.txt&offset=257
# prev = https://watcher.sour.is?uri=https://twtxt.net/user/mutefall/twtxt.txt&offset=57
@caesar if you haven't had a chance, i encourage you to join the pod's weekly call. i did and it was a good time. plenty of opportunities to talk shoppe realtime and ask some questions. i think @prologic is going to post the notice soon?
while that's not slow per say, one has to take into account pulling the feed into the cache layer and having payload rendered in a browser. the round trip of data from out in the wild back to the server involves network calls (cpu-bound), cache-stuffing (cpu+memory-bound), and disk read/writes. by the time it's done it can be a costly operation compared to a db-backed data store.
@caesar i'm working on some of the pod help pages this weekend which helps to unpack some of the design decisions from both technical and philosophical stances.

to give things a perspective, try measuring the time of running curl on top1m even on a fast link it can be up to 1s
@mckinley i passed the flag capture and went directly to your blog noting the panasonic toughbook. those and this post brought a tear to my tired eyes
> I guess I should go read the code before asking too many questions, but I’m a little puzzled why the same issues with a feed being huge don’t present an issue every time you want to poll for updates?

you're reading from cache, so it's quicker. memory will always have significantly faster iops vs disk-bound read operations. also recommend giving the codebase a look. there's always room for contributors. i'm planning to take a crack at a few issues.
@prologic indeed. even with a db backing it pagination is usually an expensive query operation in addition to loading it into the heap.

i think there's an rfc for adding pagination to the hardest problems in computer science. if there isn't, there should be :-)
put aside the idea of a 1mb text file. what if your pod user's grow their feed to 60mb over months or years? how do you effectively cache and serve this without putting more compute resources behind it? to illustrate, go to github and try loading rawtext one of those big blocklists with tons of entries. takes 10-20s, yes? similar principal applies here.

pagination would be a good way to work around this but without js being involved i'm not sure how it would be done.
> A twtxt file is literally a flat file containing a list of all of a person’s posts. Surely simply displaying all of that person’s posts in Yarn should be the easiest possible thing to do, way easier than threading etc. Why would it require “investing heavily in

displaying a flat text file is in principle not problematic at all. the crux of the situation is the scale-factor.
> Sure. Possible. Infinite scroll on an SSR isn’t really possible without significant use of JS AFIAK.

accurate. the backend has to be able to catch the event from the browser which there's a disconnect unless you have some sort of client-side hook to trigger the backend to advance the next pagination call.
@prologic

there's an ancillary note to take into consideration. post-arpa days (1987..1994) if one wanted, one could reach the end of the internet. in those days backhauls were t1 and the data contained within the internet could be held on a hard drive.

twtxt/yarn in general allows me to reach the end. at least for the day. there's no infinite feeds to overwhelm the visitor which in my view is quite nice compared to the endless pit that is the current state of the web.

we are very very small. but we are profoundly capable of very very big things. --hawking
@prologic node01.dc01.happycrunchybits.lol seems legit?
@screem you might have a better go with kvm vs vmware. i gave macos its day in court last year with kvm and it worked pretty well.

https://github.com/foxlet/macOS-Simple-KVM here's a repo i found that might be useful
@nexeq bring your friends and see how they get along here. you may even decide to start your own pod one day. that's that great thing about the protocol. you're not locked in but there's interoperability between pods.
@nexeq searx is great, i host my own. but if you follow the money trail to my systems the nsa still knows it's me searching for old thinkpads and temperature tolerances for thermal paste :-)

public search instances are likely a good call. i still use duckduckgo myself as i have work and research to be done, and it's a better alternative than google in my personal view.

no apologies needed, we're all friendly here
@nexeq thus you can consider the feed of tweets pseudo-ephemeral in nature in that as time progresses so do does content.

i hope this makes some sense. at first i didn't understand it until i read the techspec and tore apart the client code then it was a this is brilliant moment.

there's quite a bit of philosophical thought that goes into how things work here. this is not twitter, and that's by design.
@nexeq given lightweight nature of yarnd and the twtxt protocol in general relying on a text file and a cache layer, a poderator who would desire to have n(x) timeline (i.e. all tweets from 2017 to today) would have to invest heavily in infrastructure and the protocol twtxt and client yarnd would have to be redesigned from the ground up.

now that being said, let's say there's a post that turns into a yarn and people respond to it frequently it may be more prevalent and show up in your feed if you are indeed engaged in said yarn.
@prologic i may need to write a document on this.

@nexeq there's a couple of ways to look at this. first your feed is what you make it. based on whom you follow. there's also this concept which i don't think has been officially named, but the idea is the a timeline such as in a timeseries moves along as life does. thus what you see today may not be here next week
if not brave, duckduckgo, google, what search engines are you using?
@prologic need a proper name for my dc which is half up. dc01.<noidea>.tld
looks like you made some progress on it. maybe during our next session we can work out the kinks. i need to give dendrite a proper run soon
@david that reminds me, i need to get back on libera. how i do miss irc and i can bridge into my matrix server should i choose to do so.

@prologic that's the spirit!
@prologic absolutely reasonable. i go back and forth with my interest in cloudflare. there's a subset of people on the internet who shake their fist cloudflare is a mitm!!! well ffs, what do you expect? they are proxying global traffic for you. this is how things work. if you want to use their service you have to accept some basic perceived risks

the way i see it. if they housed lulzsec for 10+ months without a takedown my concern for my pod and infrastructure is minimal should i choose to use them.
@carsten greatly appreciated! but i have 2x /29 blocks at this time. this is more of a network routing thought experiment :-)

how do you like inwx as a domain registrar?
i maintain a private matrix homeserver merely to bridge other communication methods that people i correspond with might use. comes in handy in that aspect.

i really miss freenode
matrix is one of those love || hate situations i'm afraid. when implemented properly and the plants properly watered it can work just fine. but the documentation like many things are lacking, especially a year or two ago when many were trying to shoehorn it into docker. ended up solving this problem after a few days but synapse was resource intensive.

dendrite works (or did), but the number of issues not being addressed is concerning and their api/system components are not in full parity with synapse (yet).
i'm in the same space as @novaburst although i boil my water and pour into a glass french-press. in some ways i am my own coffee machine. i tried an electronic one, didn't enjoy the coffee as much and was too pricey
@prologic curious on your setup here. are you running argo tunnel or are you proxying at cloudflare some other way to your swarm backend(s)?
> People don’t want to run their own servers, and never will.

that's quite the broad generalisation. moxie seems to be speaking for the people. while the subset of operators who run their own systems are tiny compared to the global internet this is like saying people don't want to drink coffee, they are addicted to the caffeine i'm long past even recognising caffeine in my system but i love a good cup of coffee as a focal point
@novaburst i'd gladly pay you tuesday for some .dotfiles today :-)
@prologic absolutely. accessibility is everything. i'm legally blind without my spectacles so i get it. and should my vision further deteriorate i'd be right there along with you on a shiny mac-something because bsd-core :muscle
the macbook air circa 2011-2014 were very nice. got my hands on one and loaded linux on it. lasted forever until i dropped it. while i'm not a personal fan of the ecosystem, i respect it as they do make things quite easy and mostly does just work. if i were doing business i'd give them a go
@novaburst this is well done. something tells me you .dotfiles would be a fun adventure :-)
but yes. finding yarn has been delightful. reminds me of the bbs days which was also recently revived with the advent of the tilde and smol movement(s). my long-game is to carve out my own piece of the grid, have a few laughs, learn, and experiment.
@prologic aye, this is the way. i'd rather meet a small handful of solid comrades to chat with than a void full of shallow two-dimensional people who consider human connection a thumbs up on their world's biggest pickle post
@prologic that'd be lovely. i just stumbled upon lichess and played a couple of games. got stomped proper :-)
in working on my pod, i decided to get s/stupid/clever/ with the mesh network for my infrastructure. goal is to global load-balance and give say an .au visitor a traffic path close to them while someone in the .us or .nl would have their own.

all that noted, any leads on some decent vps providers in .au, .uk, .jp, an .is ?
also will suggest:

- ensuring you persist your metrics data as it's nice to be able to look back for patterns (docker volume to tier2 storage like rusty-spinners is ace here)
- get to know grafana. use some of the dashboards that are available, edit them to learn how the queries and designs work.
- if you hook up alert-manager, give gotify a try.

if you have questions maybe we can have a chat on our weekend call
+1 for what @prologic recommended. having good optics into your systems is helpful. i found a second-hand tv for 20$ usd, mounted it on the wall, i can keep an eye on things and delude myself into thinking i'm running some sort of mission-control :-)
@prologic chess is a wonderful game. i've recently dusted off my chess set to get back into thinking mode.
@adi let me have a think on it if you don't mind. :-)

i've nothing up at the moment as i'm still trying to wrangle my machines and auto-deploy swarm. i've been so polluted by kubernetes for years the simplicity of swarm is forcing me to unlearn adhd-plumbing
interesting. the bird site just released their v3 tor gateway.

twitter3e4tixl4xyajtrzo62zg5vztmjuricljdp2c5kshju4avyoid.onion

doesn't surprise me alec worked on this, he did the same at facebook. great work even if i won't use the platform.
@prologic i like to think we are there as well or very close. mass-adoption is the problem due to the ux (which we spoke on yesterday in general), and tooling being too complex for the non-technical enthusiast. most people don't want to learn, then want instant-on. as for the quote. beautiful :-)

@caesar valid point, and likely right. the yarn ecosystem is microscopic compared to the abyss that is the internet, but that will not stop us from having strong communities and making progress. i'm in this for the long-game
@screem my theory on this is because corporations haven't learned how to make money with oss/foss yet. also, i do understand the abundance of customisations available for the android ecosystem. even with an alternate os such as graphene or calyx you're still riding base android. took me a good week to get the device how i like it. now it sits in a drawer until i go out and need to yarn or signal
@adi currently evaluating, but so far it's very simple which is nice. i'm admittedly having to unlearn some complexities introduced by hugo and other static generators. i don't consider this negative at all
@adi you're quite welcome
@carsten the cartoon was a good read and accurate :-)
i feel linux on the desktop hit mass-acceleration when gaming started to work properly on it. my theory is if you had native support for games in linux and gpus worked amazingly (i'm talking to you, nvidia), you'd see an even bigger adoption of linux as it gives most computer enthusiasts what they want and is still an open system full of possibilities and choice.
@prologic it's a nice bit of tech. however i'd love to port or rebuild with golang for no other reason that i need the practise, but likely it could be further optimised for more efficient operations.

here's sort of the workflow it serves for me:

inbound email ---> mx for tastycornpuffs.com ---> messaged wrapped in pgp ---> target mx (personal) ---> my inbox
decentralised and disconnected ---> move towards centralisation ---> full centralisation and tight-coupling of systems ---> netizens asking why? ---> move to decentralisation ---> decentralised and distributed (we likely are here now) ---> the future?

there's an idea i scrapped together (likely i'm not the first to coin this).

when the system fails you, you build your own system
then again, i come from a different era. 80/90s bbs/early-stage internet. thirty years ago we didn't have tracking, government surveillance was minimal at best (they didn't know what they had at the time), corporate control of networks was nil, and every human had the freedom to explore network(s).

it's interesting to watch what's occurred over the decades. the internet fundamentally has not changed. only the people and technologies riding on top of it have.
similar happens when ig or fb goes down. a good portion of the internet goes full tilt. i never want to be so dependent on something that it creates a sense of rulership and control over me. i'd rather be engaged with an ecosystem that allows a netizen to stake their claim of the internet, link up with their friends vs the concept of friends and followers that have been psychologically optimised for most people driven by dopamine and fomo
my own view: if i truly wanted an account there and was upset about being suspended to the point i had to contact the bird and ask for passage then i'm officially hooked on their system, yes? i think as human beings we all want to be connected in some way, it's in our genetic code. but at what cost? giving twitter every datapoint you have, a copy of your passport, and dna sample?

in the end, none of this truly matters. but companies like twitter want to induce a synthetic worry for not being a part of their ecosystem.
@david you're considered an established user for having your account for such a long time, twitter already recognises you. likely you've never been sim-walled, yes? new accounts these days that meet a certain criteria are almost always suspended or sim-walled.

@prologic i signed up to namelock my handle (suspension works too), uploaded an avatar, followed some computer scientists, liked a couple of posts, let the account sit for a while, then boom.
frankly, there's something refreshing and honourable about being suspended from twitter
@novaburst remind me to tell you the story about my experience with google's burrito surveillance one day

@screem once i stop procrastinating and write it would be happy to set you up proper. google voice is still about the only non-sim-bound way of validating most accounts. apparently the bird site still will reject sip-carrier e.164 numbers as seen in my deadlocking test
@prologic 65516 68162 55401 70333 20849 52655 66910 24952 43174 80113 82214 86079
visualisation of a dns cascade over wireguard. not a bad round trip (ms) for a dual layer setup. now let's pump traffic through it and measure it for the next 12 hours, shall we?


@prologic i've accounts with twilio and telnyx for years so likely one or both. regarding the email situation there's many self-hosted options for that already well-baked and ready to serve. simplelogin is what i run currently for private use, but i'm considering making a public host for friends and such.

@screem 100% i consider this and also the email thought @prologic mentioned as a watered-down firewall of sorts. remind me to share this during our next call. or i should write about it?
@prologic this reminds me of my thought experiment where if microsoft open-sourced windows (or apple on os-x) i theorise that both companies would sell more things

i may even run osx or windows at that point.

i currently run grapheneos and the user experience is fairly well put together, it's fully google-stripped and you can use many apps you know and love. but i'm not a good pitchman. i've rocked a flip-phone for a decade.
@technicalsuwako i hope you don't mind me borrowing this informative infographic :-)
@adi yes, @prologic mentioned to me on our last call. https://git.mills.io/prologic/shops

@movq i'm going to give bundlewrap a shot after my day ends. i'm always interested in new config and orchestration tools. on marketing, i don't think anyone is good at it. i sure am not. i couldn't sell free money. :-)
@adi xmr deployed. send me your pgp pub key if you want the txid
we were discussing signal in another yarn last week. i'm looking at building a service that would let a potential signal user for a small fee purchase an e.164 sip number to use for registering signal or other mfa situations. of course sms-based mfa is rather insecure due to the triviality of ss7 network exploitation but i'm sure not all of us have operative-level threat models.

i'm looking to build this in golang since it's wicked fast and as always keep it open so that people can fork and stand up their own system should they choose to.

what are your thoughts?
@movq on your blog you mentioned bundlewrap for configuration management. i'm fond of ansible and use it at work but wanted to know how you like bundlewrap vs something like ansible.
@adi indeed an idea. i've been unplugged from the commercial clouds for years. do you suggest one to use for mass storage as a test? wrap your assets in cryptography and the concern dwindles.
as we discussed this is a small yet quality network space. while everyone is invited i feel there's mechanics in play that would exclude those who's sole focus is to have their brain chemed up with ye older slot pull

want dopamine? have a chocolate. i am. cheers!
in modern day social networking sites the dopamine trigger comes in the form of notifications and likes. many of us are all too quick to pull the lever.

twtxt lacking such mechanisms forces me to be thoughtful in my posts and replies. similar to if i was writing a letter to a friend far away.
yesterday we were discussing twtxt spec and yarn client. the subject of likes came up or the lack thereof.

slept on this and i think it's a brilliant idea. here's why.
@prologic very reasonable, sir. i particularly enjoy some of the history of my existence but i will be the first to admit how lovely fibre is :)
@prologic

in a way, sir you hypothesised :)
@movq this required a bookmark to see how it unfolds.

counter-hypothesis: as humans perhaps we experience an etching period where a specific time in our lives or an era has a deep and profound imprint on our psyche. as the world changes, we adapt but there may be a point where on marks that moment of wonder and never let it go.

i wax poetically about the past and hold onto things i care for. but instead of being frozen in time i prefer to meld older constructs with new. thus satisfying that nostalgic need while deriving new memories that bring the ages together.
@adi should have clarified, no memory outside of what's soldered onboard. apologies.
@adi really the same configuration
@novaburst well put. it's the plague of the internet.
@adi .us
was fun meeting chatting up a swarm... :)

i'm off for a coffee and walk, sleep well gents.
@mutefall locally might be tricky. i'm not plugged into the .ro market. but if you wanted one i could send you one if you are okay just paying shipping. i find these all the time.
@prologic no worries, mate. it could be my complex firewall and aversion to webrtc which i will sort through. jitsi seems to work for whatever reason.
@eldersnake i think of them the same way. now if one were to use cryptography to sign and verify the unique existence of the nft by the artist and i had a way to collecting it in my own "art room" i might purchase one direct from the artist.
@adi i was using hugo for my soon to be launched (we've all heard that before, eh?) site, then started working with mkws 🧑‍🍳🤌
@adi i found mind for 20$ usd/ea. what part of the world are you in? if you have problems finding them let me know i'm sort of the thinkpad whisperer :)
@prologic the way i deal with these things... well let me diagram it:



too early for a proper screenshot. this was my previous swarm setup in 2020 which i'm reviving. swarm master and workers communicate over a dedicated mesh and traffic routes to an external vps traffic gateway. the reverse proxy that handles things is traefik which i've rather customised for my own pursuits.

this method allows me to host swarm anywhere i want be it at home, work, at the coffee shop off a cellular modem
@prologic sounds great, looking forward to some of your recipes. have you seen any of funk-penguin's work?

@adi these were three t480s i rescued from a government auction for near theft prices. no memory, disks, chargers, or batteries but that didn't stop me from giving them a good home :-)
@prologic i had joined and saw your name but then the peer connection dropped.
weekend goals:

- teardown old thinkpads for service
- optimize storage on swarm cluster
- test deploy yarnd pod on a temporary domain
- go for a long ride
i'm considering a scheduled jitsi soon. anyone want to shoot the breeze? i'm allergic to timezones
@caesar no worries. mailfence makes it much easier for multiple accounts under a shared domain. it's a bit pricier but worth it. more or less equates to what you'd pay for two users on google workspace.

really any mail system (even yahoo and gmail) support pgp. i seem to lean towards encrypting emails and signing from the terminal then simply use a mail client or web browser to send the message. but i get what you're saying, nice to have the automatic bits. helps with people adopting better posture
@prologic i likely have a working prosody stack you can try on your swarm cluster. depending on how you're wired up for networking may need some tweaks.
@caesar have used both in the past. both are well done, especially if you want to use an imap/pop client. custom domains are supported on both, but i will note mailfence does a better job with rfc compliance on email in general

signal's apps are open-source as well as their server infrastructure. the only thing that's not open is their interface to the intel sgx enclaves (blackboxes). i can accept this risk.

minimal metadata is exposed account id(s), last connection date, account creation date in unix timestamps much less than the facebook
thoughts from the abyss[0]:

- psyche debt > fiscal debt > tech debt
- learning to let things go is analogous to swedish death cleaning of the self
- don't build a skyscraper to house a post-it note
- procrastination is likely a sign you're manifesting complexity
- never underestimate the potato, it's a mighty fine nightshade
@lyse this is the path. when i need to clear the cpu i go for a proper walk and get myself out of my head. cheers
@carsten mailbox at one time they had a tor gateway for connectivity to their service, is this still active?
@lyse this is really a comforting scene. hopefully you were able to get lost for a while.
-- on breaking dns which would ripple effect the entire internet
++ on dumping their ca(s)

the internet, in my view has always been nation-agnostic. we are one world, one nation. and like every nation you have good s/citizens/netizens/ and bad s/citizens/netizens/

we don't close roads because criminals drive cars, we just shoot their tires out
@screem signal + xmpp (bridged via matrix for aggregation)

although i am looking to upgrade to @ullarah suggestion.

mailbox.org is a good provider, been around a long time. there's another one called mailfence which is also very good and have been around a good while. protonmail is great, but i pull down all my mail and protonmail makes this complicated with a bridge (paid feature).

germany handing over data to foreign governments likely will vary by the relationship and policy between the requesting party and germany itself. due process is still a thing there.
nokturnal thoughts[0]:

- the pursuit of perfection is a fool's game
- related to the aforementioned, finding the perfect domain name is an abyss-like vortex
- eventually the construct of perfection comes organically
- trying to deadname/squat your id accounts on all platforms says i care too much about centralised rubbish
- note to author, you use too many bulletpoints
@prologic it could be worse. you could be 100r.co and relying on a nice submarine to give you radio data every fortnight.

that's really not bad considering your region. when i was in the eu it was rather expensive. now i have more bandwidth than i know what to do with so i create traffic crawlers that crawl the top 1m websites to keep the cache hot
@kevin i believe we need to have a chat. this is just my game.

i'm also rather inspired by your small website. meanwhile i'm trying to bribe @prologic into learning css for me.
@prologic $89 buckaroos with a /29
@prologic you're paying for the noc-monkey who flips hard drives when they catch on fire. running this at home, you're the noc-monkey thus the disparity in value/cost