# I am the Watcher. I am your guide through this vast new twtiverse.
# 
# Usage:
#     https://watcher.sour.is/api/plain/users              View list of users and latest twt date.
#     https://watcher.sour.is/api/plain/twt                View all twts.
#     https://watcher.sour.is/api/plain/mentions?uri=:uri  View all mentions for uri.
#     https://watcher.sour.is/api/plain/conv/:hash         View all twts for a conversation subject.
# 
# Options:
#     uri     Filter to show a specific users twts.
#     offset  Start index for quey.
#     limit   Count of items to return (going back in time).
# 
# twt range = 1 14
# self = https://watcher.sour.is/conv/3bwkpna
I also totally get whet you're saying about a twtxt file *potentially* growing to be huge. I guess that, and the fact that it's necessary to work around it with a significant caching architecture, is a major downside to the model of twtxt itself which I hadn't considered.
@caesar Major downside hwo? I don't see it as a downside at all, it's the same idea as a logfile, and generally you have (or should have) rotation policies around it. How you manage this is up to you. yarnd for example has a builtin weekly job that ensures feeds on a pod are kept under 1MB in size (configurable) and are rotated when they exceed this. Pods will also refuse to fetch feeds over this size too.
@caesar Major downside hwo? I don't see it as a downside at all, it's the same idea as a logfile, and generally you have (or should have) rotation policies around it. How you manage this is up to you. yarnd for example has a builtin weekly job that ensures feeds on a pod are kept under 1MB in size (configurable) and are rotated when they exceed this. Pods will also refuse to fetch feeds over this size too.
@prologic I didn't know the rotation was already implemented in yarnd. Is there a way to see the full history of a feed? I don't see a prev metadata tag on yours.
@mckinley Not yet... Coming soon™ 😅 I was a little slower to adopt that particular spec. I'll get around it it don't worry!
@mckinley Not yet... Coming soon™ 😅 I was a little slower to adopt that particular spec. I'll get around it it don't worry!
@prologic No worries. It doesn't affect me much, I'm just curious.
@mckinley Judging by the size of your feed currently and how frequently / infrequently you post to it, I'd say you won't have to worry for another couple of years 🤣


$ curl -v -o /dev/null https://twtxt.net/user/mckinley/twtxt.txt 2>&1 | grep -i 'Content-Length'
< content-length: 107361
@mckinley Judging by the size of your feed currently and how frequently / infrequently you post to it, I'd say you won't have to worry for another couple of years 🤣


$ curl -v -o /dev/null https://twtxt.net/user/mckinley/twtxt.txt 2>&1 | grep -i 'Content-Length'
< content-length: 107361
@caesar i'm working on some of the pod help pages this weekend which helps to unpack some of the design decisions from both technical and philosophical stances.

to give things a perspective, try measuring the time of running curl on top1m even on a fast link it can be up to 1s
while that's not slow per say, one has to take into account pulling the feed into the cache layer and having payload rendered in a browser. the round trip of data from out in the wild back to the server involves network calls (cpu-bound), cache-stuffing (cpu+memory-bound), and disk read/writes. by the time it's done it can be a costly operation compared to a db-backed data store.
@caesar if you haven't had a chance, i encourage you to join the pod's weekly call. i did and it was a good time. plenty of opportunities to talk shoppe realtime and ask some questions. i think @prologic is going to post the notice soon?
@mutefall Notice is here
@mutefall Notice is here