# I am the Watcher. I am your guide through this vast new twtiverse.
#
# Usage:
# https://watcher.sour.is/api/plain/users View list of users and latest twt date.
# https://watcher.sour.is/api/plain/twt View all twts.
# https://watcher.sour.is/api/plain/mentions?uri=:uri View all mentions for uri.
# https://watcher.sour.is/api/plain/conv/:hash View all twts for a conversation subject.
#
# Options:
# uri Filter to show a specific users twts.
# offset Start index for quey.
# limit Count of items to return (going back in time).
#
# twt range = 1 17
# self = https://watcher.sour.is/conv/b6juizq
@codebuzz I have some shell scripts that handle some of the log formatting details, but I mostly write my mesages by hand. Lately I've been browsing twtxt.net since they aggregate most of the known network. I have a couple of demo aggregators sitting around, but I'm in the middle of some infra rebuilds so a lot of my services are offline rn. They're both built on a simple social graph analysis that extracts urls for your direct follows the follows listed on each of those feeds (friend-of-a-friend replication). certain formatting operations are awkward with my setup, so I may write an app of some kind in the future. likely gemini-based, but I have a number of projects ahead of that one in the queue.
@cuaxolotl what problem does building a social graph solve?
@cuaxolotl what problem does building a social graph solve?
@cuaxolotl This is largely by accident and not on purpose:
> Lately I’ve been browsing twtxt.net since they aggregate most of the known network
@cuaxolotl This is largely by accident and not on purpose:
> Lately I’ve been browsing twtxt.net since they aggregate most of the known network
FoaF is just a way to crawl the network. I prefer this method because it frontloads the network traffic using a heuristic that covers most of the content i'm likely to be interested in anyway and allows for some level of discovery. Also graph traversal is fun.
FoaF is just a way to crawl the network. I prefer this method because it frontloads the network traffic using a heuristic that covers most of the content i'm likely to be interested in anyway and allows for some level of discovery. Also graph traversal is fun.
@cuaxolotl The reason I ask is that I maintain the Twtxt search engine and crawler service that basically does exactly this, so I'm curious what you're trying to solve by doing this yourself? Not that that's a bad idea. I just want to understand what you are trying to achieve. 🤗
@cuaxolotl The reason I ask is that I maintain the Twtxt search engine and crawler service that basically does exactly this, so I'm curious what you're trying to solve by doing this yourself? Not that that's a bad idea. I just want to understand what you are trying to achieve. 🤗
same as you i assume. i like making my own stuff.
same as you i assume. i like making my own stuff.
@cuaxolotl Good enough 😅 LMK if I can help in any way then, what I built isn't perfect, but the crawler is able to crawl the entire space in ~15m or so (_every day_)~
@cuaxolotl Good enough 😅 LMK if I can help in any way then, what I built isn't perfect, but the crawler is able to crawl the entire space in ~15m or so (_every day_)~
Well, having added more accounts, I have noticed aggregating (+/- 15min refresh, and caching afterwards) is still a hefty proces. Something I am pondering on how to do that one better.
@Codebuzz It currently takes my yarnd pod here around ~2m on average to fetch, process and cache ~700 feeds.
@Codebuzz It currently takes my yarnd pod here around ~2m on average to fetch, process and cache ~700 feeds.
Thats quite good. I'm just thinking of how to keep things simple without needing databases (as a client app), for now it uses cache.. Would be great if a follower would have an AP-layerI over a twtxt and just fetch from the most recent timestamp known in my cache.