# I am the Watcher. I am your guide through this vast new twtiverse.
# 
# Usage:
#     https://watcher.sour.is/api/plain/users              View list of users and latest twt date.
#     https://watcher.sour.is/api/plain/twt                View all twts.
#     https://watcher.sour.is/api/plain/mentions?uri=:uri  View all mentions for uri.
#     https://watcher.sour.is/api/plain/conv/:hash         View all twts for a conversation subject.
# 
# Options:
#     uri     Filter to show a specific users twts.
#     offset  Start index for quey.
#     limit   Count of items to return (going back in time).
# 
# twt range = 1 13
# self = https://watcher.sour.is/conv/wgwmt4q
tkanos
twtxt.net
View Thread
@lyse @mckinley @eaplmx the code you requested to follow all the twtxt mentions (https://github.com/tkanos/we-are-twtxt on the folder search)

- I adjusted the code to take for the rank, taking in account only the mentions and not weighted by the number of tweets)
- the files generated will be in your home folder
prologic
twtxt.net
View Thread
@tkanos Nice 👌
prologic
twtxt.net
View Thread
@tkanos Nice 👌
lyse
lyse.isobeef.org
View Thread
@tkanos Cool! I like your disclaimer in the readme. :-D A few things caught my eye while scrolling through the code, the ones I remember:

1. The protocol switch in lines 141ff could make use of else if.
2. There's a superfluous space in the progress bar in line 155.
3. The User-Agent header contains a { which I reckon is a typo. Also, I reckon the URL and nick should be swapped, in case you wanted to stick to this discoverability format.
4. The feed discovery regex assumes that filenames always are twtxt.txt which will miss a few feeds, such as @anth's and @hxii's. Parsing mentions would be more accurate. And then also parse all mentions from a twt, not just the first one.
5. If you're still bored, adding support for archived feeds would be a thing. :-)

Anyways, nice work!
tkanos
twtxt.net
View Thread
Thanks @lyse,

1. even a switch case would be better
2. done quickly
3. done quickly
4. yes BUT if I check only mentions I will loose the header following one, (that are not always in the same style: cf : https://niplav.github.io/twtxt.txt ), I can do both and only loose the redirection BUT I need to be bored again to do that change (1 hour I ll say)
5. (easy) when i will be bored again it will be on the pipe.

Thanks
tkanos
twtxt.net
View Thread
Thanks @lyse,

1 - even a switch case would be better
2 - done quickly
3 - done quickly
4- yes BUT if I check only mentions I will loose the header following one, (that are not always in the same style: cf : https://niplav.github.io/twtxt.txt ), I can do both and only loose the redirection BUT I need to be bored again to do that change (1 hour I ll say)
5 - (easy) when i will be bored again it will be on the pipe.

Thanks
tkanos
twtxt.net
View Thread
one other thing cool that you have not noticed but i want to share. Is that the first iterative version was taking 8 minutes to go on all twtxt / download/ parse.
The new version using heavily go routines takes around 10 seconds.
prologic
twtxt.net
View Thread
@tkanos That's pretty impressive 👌
prologic
twtxt.net
View Thread
@tkanos That's pretty impressive 👌
eaplmx
twtxt.net
View Thread
@tkanos amazing!
eaplmx
twtxt.net
View Thread
@tkanos 10 seconds?!
Is it opening all the connections at the same time?

(I don't use Go in a daily basis, but I think I understand the code a bit), great work!
tkanos
twtxt.net
View Thread
@eaplmx Nop but a lot at least.
lyse
lyse.isobeef.org
View Thread
Very nice, @tkanos! 1. Indeed. :-) 4. Ah, now I see. Didn't think of follow metadata. Ten seconds is really awesome. :-)