# I am the Watcher. I am your guide through this vast new twtiverse.
# 
# Usage:
#     https://watcher.sour.is/api/plain/users              View list of users and latest twt date.
#     https://watcher.sour.is/api/plain/twt                View all twts.
#     https://watcher.sour.is/api/plain/mentions?uri=:uri  View all mentions for uri.
#     https://watcher.sour.is/api/plain/conv/:hash         View all twts for a conversation subject.
# 
# Options:
#     uri     Filter to show a specific users twts.
#     offset  Start index for quey.
#     limit   Count of items to return (going back in time).
# 
# twt range = 1 25
# self = https://watcher.sour.is/conv/yrv75ka
prologic
twtxt.net
View Thread
Twtxt is still very much alive and well. I just wrote a quick tool to crawl as much of the Twtxt network as I could and here's what the results are:


Crawled 516 feeds
Found 52464 twts


That means there are >500 unique Twtxt feeds/users, and over ~52k Twts posted to date. 😳~
prologic
twtxt.net
View Thread
Twtxt is still very much alive and well. I just wrote a quick tool to crawl as much of the Twtxt network as I could and here's what the results are:


Crawled 516 feeds
Found 52464 twts


That means there are >500 unique Twtxt feeds/users, and over ~52k Twts posted to date. 😳~
movq
www.uninformativ.de
View Thread
@prologic Would be interesting to plot these values. I suspect there are tons and tons of feeds with just a few messages and then some bots with thousands of twts. 🥴
movq
www.uninformativ.de
View Thread
@prologic Would be interesting to plot these values. I suspect there are tons and tons of feeds with just a few messages and then some bots with thousands of twts. 🥴
vain@www.uninformativ.de
www.uninformativ.de
View Thread
@prologic Would be interesting to plot these values. I suspect there are tons and tons of feeds with just a few messages and then some bots with thousands of twts. 🥴
movq
www.uninformativ.de
View Thread
@prologic Would be interesting to plot these values. I suspect there are tons and tons of feeds with just a few messages and then some bots with thousands of twts. 🥴
xuu@dev.txt.sour.is
dev.txt.sour.is
View Thread
@prologic that seems to match my numbers. are you picking up the few gophers out there?

kinda makes me wonder about the ~300k you have cached. y'all got the library of alexandria over there.~
xuu
txt.sour.is
View Thread
@prologic that seems to match my numbers. are you picking up the few gophers out there?

kinda makes me wonder about the ~300k you have cached. y'all got the library of alexandria over there.~
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic that seems to match my numbers. are you picking up the few gophers out there? \n\nkinda makes me wonder about the ~300k you have cached. y'all got the library of alexandria over there.~
xuu
txt.sour.is
View Thread
@prologic that seems to match my numbers. are you picking up the few gophers out there? \n\nkinda makes me wonder about the ~300k you have cached. y'all got the library of alexandria over there.~
xuu
txt.sour.is
View Thread
@prologic that seems to match my numbers. are you picking up the few gophers out there?

kinda makes me wonder about the ~300k you have cached. y'all got the library of alexandria over there.~
lyse
lyse.isobeef.org
View Thread
@prologic @vain @xuu Wow! Did you account for duplicates such as the twtxt.net URL change?
xuu@dev.txt.sour.is
dev.txt.sour.is
View Thread
@vain @lyse @prologic Nope.. i have updated my gist to include the feeds listing. feeds.txt
xuu
txt.sour.is
View Thread
@vain @lyse @prologic Nope.. i have updated my gist to include the feeds listing. feeds.txt
xuu
txt.sour.is
View Thread
@vain @lyse @prologic Nope.. i have updated my gist to include the feeds listing. feeds.txt
xuu@txt.sour.is
txt.sour.is
View Thread
@vain @lyse @prologic Nope.. i have updated my gist to include the feeds listing. feeds.txt
movq
www.uninformativ.de
View Thread
@xuu Yeah, looks like most of the feeds are tiny and then there are a couple of huge bots: https://dump.uninformativ.de/v/898bef509f/feedhisto.txt (Or I’m too tired and stupid, which is always a valid option.) (Also, I didn’t clean up for duplicates, either.)
vain@www.uninformativ.de
www.uninformativ.de
View Thread
@xuu Yeah, looks like most of the feeds are tiny and then there are a couple of huge bots: https://dump.uninformativ.de/v/898bef509f/feedhisto.txt (Or I’m too tired and stupid, which is always a valid option.) (Also, I didn’t clean up for duplicates, either.)
movq
www.uninformativ.de
View Thread
@xuu Yeah, looks like most of the feeds are tiny and then there are a couple of huge bots: https://dump.uninformativ.de/v/898bef509f/feedhisto.txt (Or I’m too tired and stupid, which is always a valid option.) (Also, I didn’t clean up for duplicates, either.)
movq
www.uninformativ.de
View Thread
@xuu Yeah, looks like most of the feeds are tiny and then there are a couple of huge bots: https://dump.uninformativ.de/v/898bef509f/feedhisto.txt (Or I’m too tired and stupid, which is always a valid option.) (Also, I didn’t clean up for duplicates, either.)
prologic
twtxt.net
View Thread
@xuu Nope, I'm not crawling any twtxt feeds over gopher at the moment, colly doesn't support it 🤣

@lyse Yeah I haven't really taken that into account I don't think. Hmmm 🤔

@xuu Where should we take this next? I _really_ think_ we should build a Twtxt Search Engine 😀 It _might_ help a lot with the "discovery" aspects/problems Twtxt always had back in the day (~2016)?_~
prologic
twtxt.net
View Thread
@xuu Nope, I'm not crawling any twtxt feeds over gopher at the moment, colly doesn't support it 🤣\n\n@https://lyse.isobeef.org/twtxt.txt> Yeah I haven't really taken that into account I don't think. Hmmm 🤔\n\n@https://txt.sour.is/user/xuu/twtxt.txt> Where should we take this next? I _really_ think_ we should build a Twtxt Search Engine 😀 It _might_ help a lot with the "discovery" aspects/problems Twtxt always had back in the day (~2016)?~_
prologic@twtxt.net
twtxt.net
View Thread
@xuu Nope, I'm not crawling any twtxt feeds over gopher at the moment, colly doesn't support it 🤣\n\n@https://lyse.isobeef.org/twtxt.txt> Yeah I haven't really taken that into account I don't think. Hmmm 🤔\n\n@https://txt.sour.is/user/xuu/twtxt.txt> Where should we take this next? I _really_ think_ we should build a Twtxt Search Engine 😀 It _might_ help a lot with the "discovery" aspects/problems Twtxt always had back in the day (~2016)?_~
prologic
twtxt.net
View Thread
@xuu Nope, I'm not crawling any twtxt feeds over gopher at the moment, colly doesn't support it 🤣

@lyse Yeah I haven't really taken that into account I don't think. Hmmm 🤔

@xuu Where should we take this next? I _really_ think_ we should build a Twtxt Search Engine 😀 It _might_ help a lot with the "discovery" aspects/problems Twtxt always had back in the day (~2016)?_~
lyse
lyse.isobeef.org
View Thread
@xuu @vain @prologic Thanks, great analyses! Ha, I have 42 twts! Well, now not anymore. Using the target URL of a redirect and not the original one should get rid of at least some of the duplicates. I would have expected some of the EMPTY flagged ones as FAILED instead, esp. all the ones with the invalid TLDs (see mine for example). But I need to have a deeper look at the code. Cool project nevertheless!