# I am the Watcher. I am your guide through this vast new twtiverse.
# 
# Usage:
#     https://watcher.sour.is/api/plain/users              View list of users and latest twt date.
#     https://watcher.sour.is/api/plain/twt                View all twts.
#     https://watcher.sour.is/api/plain/mentions?uri=:uri  View all mentions for uri.
#     https://watcher.sour.is/api/plain/conv/:hash         View all twts for a conversation subject.
# 
# Options:
#     uri     Filter to show a specific users twts.
#     offset  Start index for quey.
#     limit   Count of items to return (going back in time).
# 
# twt range = 1 2
# self = https://watcher.sour.is/conv/7uvxela
prologic
twtxt.net
View Thread
So... Indexing an old archive of this pod:


$ time ./index_archive ./data
...
ERRO[1046] error decoding archived twt data/archive/fd/bcbe3e.json  error="unexpected end of JSON input"
ERRO[1048] error decoding archived twt data/archive/fe/51cf5a.json  error="unexpected end of JSON input"

real    17m35.719s
user    16m42.758s
sys     3m48.809s


There were (sadly) 332 broken archived twts my archive 😢

Other interesting tidbits: 


$ du -s -h data/archive/
5.0G    data/archive/

$ du -s -h data/index/
555M    data/index/


@xuu Feel like working on an archive compaction? 😅
prologic
twtxt.net
View Thread
So... Indexing an old archive of this pod:


$ time ./index_archive ./data
...
ERRO[1046] error decoding archived twt data/archive/fd/bcbe3e.json  error="unexpected end of JSON input"
ERRO[1048] error decoding archived twt data/archive/fe/51cf5a.json  error="unexpected end of JSON input"

real    17m35.719s
user    16m42.758s
sys     3m48.809s


There were (sadly) 332 broken archived twts my archive 😢

Other interesting tidbits: 


$ du -s -h data/archive/
5.0G    data/archive/

$ du -s -h data/index/
555M    data/index/


@xuu Feel like working on an archive compaction? 😅