# I am the Watcher. I am your guide through this vast new twtiverse.
# 
# Usage:
#     https://watcher.sour.is/api/plain/users              View list of users and latest twt date.
#     https://watcher.sour.is/api/plain/twt                View all twts.
#     https://watcher.sour.is/api/plain/mentions?uri=:uri  View all mentions for uri.
#     https://watcher.sour.is/api/plain/conv/:hash         View all twts for a conversation subject.
# 
# Options:
#     uri     Filter to show a specific users twts.
#     offset  Start index for quey.
#     limit   Count of items to return (going back in time).
# 
# twt range = 1 162
# self = https://watcher.sour.is?uri=https://txt.sour.is/user/https://txt.sour.is/user/xuu/twtxt.txt/twtxt.txt&offset=62
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic in theory shouldn't need to let users add feeds.. if they get mentioned by a tracked feed they will get added automagically. on a pod it would just need to scan the twtxt feed to know about everyone.
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic sounds about right. I tend to try to build my own before pulling in libs. learn more that way. I was looking at using it as a way to build my twt mirroring idea. and testing the lex parser with a wide ranging corpus to find edge cases. (the pgp signed feeds for one)
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic the add function just scans recursivley everything.. but the idea is to just add and any new mentions then have a cron to update all known feeds
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic yeah it reads a seed file. I'm using mine. it scans for any mention links and then scans them recursively. it reads from http/s or gopher. i don't have much of a db yet.. it just writes to disk the feed and checks modified dates.. but I will add a db that has hashs/mentions/subjects and such.
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic It is pretty basic, and depends on some local changes i am still working out on my branch.. https://gist.github.com/JonLundy/dc19028ec81eb4ad6af74c50255e7cee
xuu@txt.sour.is
txt.sour.is
View Thread
@lyse @prologic very curious... i worked on a very similar track. i built a spider that will trace off any follows = comments and mentions from other users and came up with:\n
\ntwters:  744\ntotal:  52073\n
xuu@txt.sour.is
txt.sour.is
View Thread
Yesterday was rough. Finding it hard to focus on things today.
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic that I do. lol. I am xuu on hackint.org and freenode
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic sure. I don't use signal much because I have to disclose my personal phone. Telegram? https://www.t.me/xypheri
xuu@txt.sour.is
txt.sour.is
View Thread
@xuu @prologic Your feed was great for catching edge cases ;)
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic https://github.com/JonLundy/twtxt/tree/xuu/integrate-lextwt I made a stats command for the new parser that extracts a bunch of info about a twtxt file. run like: go run ./cmd/stats https://twtxt.net/user/prologic/twtxt.txt
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic yep@some of the lexer is directly copied from monkey-lang. love that book series.
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic ah I need to add an edge case for naked urls with fragments.
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic yep. it actually extracts everything at parse time. like mentions/tags/links/media. so they can be accessed and manipulated without additional parsing. it can then be output as MarkDown
xuu@txt.sour.is
txt.sour.is
View Thread
@adi @prologic using regex. which can be a rather inexact science ;)
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic kinda.. It can parse the twts into an AST.. but most of the formatting out expects a string to do regex over rather then the parsed AST. thats what i am working out next.
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic as promised! https://github.com/JonLundy/twtxt/blob/xuu/integrate-lextwt/types/lextwt/lextwt_test.go# https://txt.sour.is/search?tag=L435-L461>\n\nthe lexer is nearing completion.. the tough part left is rooting out all the formatting code.
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic ooh I am adding that to my test suite
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic @gareppa Tis fake.. that is the name of the tower in Die Hard. A movie that takes place on Christmas Eve. The actual name of the Nakatomi Plaza is the FOX Plaza.
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic went over to watch this one in my home town last night. https://youtu.be/kUZB0_Jx3iE\n\n# https://txt.sour.is/search?tag=ChristmasLights> #2020
xuu@txt.sour.is
txt.sour.is
View Thread
@oevl @prologic for the most part a subject is just the content in the perens. Usually it's a tag. It appears near the start after any mentions. It can also contain text like (re: subjects)
xuu@txt.sour.is
txt.sour.is
View Thread
@xuu yikes the style sheet for blogs needs help.
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic (#5jqioeq) Wrote up a blog post here: https://txt.sour.is/blog/xuu/2020/12/21/twtxt-auto-discovery
xuu@txt.sour.is
txt.sour.is
View Thread
New Blog Post Twtxt Auto Discovery by @xuu ๐Ÿ“
xuu@txt.sour.is
txt.sour.is
View Thread
@deadguy @prologic been stewing on a discovery proto for twtxt. support for defining multiple ways to host/mirror a twtxt file. while being low tech enough to still be scriptable with basic Unix commands.
xuu@txt.sour.is
txt.sour.is
View Thread
@adi @prologic I'll give it a spin first thing in the AM
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic when its ready.. this is still in beta.
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic yeah it would replace rice. best part is that it's in the go build step so you don't need to do any prep work with make.
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic after stewing on it. I really like the idea of a wiki. throw it on the roadmap after DMs ๐Ÿ˜†
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic (#keh22ka) maybe a custom linking method on a pod level? like can pass a template that gets translated. ex https://{domain}/wiki/{nick}/{tag} + !somepage -> https://sour.is/wiki/xuu/somepage
xuu@txt.sour.is
txt.sour.is
View Thread
So excited for Go embedded files. https://golangtutorial.dev/tips/embed-files-in-go/
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic found it!\n\n> 2020-07-25T00:52:27.000000Z\ta new twtxt/weewiki feature: any word starting with '!' will translate to an internal weewiki reference in my HTML renderer. Example: here is my !wiki_index
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic https://txt.sour.is/twt/cwqmygq
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic I see them is why I ask. like here #cwqmygq they use both hashtag and bangtag?
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic ๐Ÿ˜ that is me testing locally. does it notify you somehow when I follow?
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic do you have any info on how the '!' tags are supposed to work? are they just a different kind of hash tag?
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic one.. kinda sorta option would be to tailor a workflow for each of the archs.. see https://github.com/JonLundy/twtxt/runs/1568071072?check_suite_focus=true
xuu@txt.sour.is
txt.sour.is
View Thread
@xuu @prologic oh.. you are.. maybe i don't understand the issue with building?
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic have you tried using the macos github build environment? looks like they have a windows one too.
xuu@txt.sour.is
txt.sour.is
View Thread
more or less. :D\nwhat are you using to ci/cd? i dont see a travisci/circleci/etc in the repo.. i can put something together to bump the patch level on master branch merges.
xuu@txt.sour.is
txt.sour.is
View Thread
There is.. but we lost the transform file to make it legible.
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic you think its time the version to get bumped? :)
xuu@txt.sour.is
txt.sour.is
View Thread
@xuu @prologic had to up my twt size a bit.
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic \n
\n-----BEGIN CRYPTUTIL ENCRYPTED MESSAGE-----\nl0GwFAQpx3ed+bZlcQ+pexbynFzZOm8EI/FivGbWQ16whyTkToVv8S2GSAjrsJoT\n37MdaBDpoitli/f/aP130b6O6SnK/LdHHJ1DTvWgxB14sq9b4mRtk7HvYzA=\n-----END CRYPTUTIL ENCRYPTED MESSAGE-----\n
xuu@txt.sour.is
txt.sour.is
View Thread
i have no clue how salt works :|
xuu@txt.sour.is
txt.sour.is
View Thread
@xuu @prologic This? Fingerprint: 161c614f08e4ed4d1c8e5410f8c457e6878574dbab7c9ac25d474de67db1bdad
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic I use https://key.sour.is/id/me@sour.is\n\n> I would need an out-of-band way to verify your public keyโ€™s fingerprint though ๐Ÿคฃ
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic Ok.. so using NaCL boxes. yeah its just a combo of using secretbox with a generated key/nonce. and then using the pubkey box to encrypt the key/nonce for each device.
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic \n> Can we not have clients sign their own public keys before listing them on their Podโ€™s account?\n\nYeah.. we probably could. when they setup an account they create a master key that signs any subsequent keys. or chain of signatures like keybase does.
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic def would be a wider discussion on preventing the pod from adding its own key to a users device list. Or using device keys to authenticate instead of user/pass.
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic pod should probably track revocation of device keys and delete the encryptedkeys that are paired with revoked keys
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic device gets the cypertext and uses it's device key to decrypt one of the keys and then decrypts the cypertext.
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic sender generates an AES key encrypts message. gets the device list for user and encrypts key for each device. sends the encryptedkeys+cypertext.
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic for encryption. we can have browser/app generate ec25519 keypair. store the private on device and add pub to list of devices for the user on pod.
xuu@txt.sour.is
txt.sour.is
View Thread
i am guessing you are using some form of webmention to notify the target of the DM? which loads it into a store for the user to read?
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic ๐Ÿ‘‹ I can take a stab at it when I am done with the changes I am working on.
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic my bad.. my next one is more fun.
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic I see.. so using an ec25519 key as identity? and some kind of certificate to define the location of a feed? or maybe a DHT like Kademlia? TwTorrent ;)
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic kinda like how MX records work.
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic My thoughts on it being if they switched from a different way of hosting the file or multiple locations for redundancy..\n\nI have an idea of using something like SRV records where they can define weighted url endpoints to reach.
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic just an off the wall question about hashes. why not use the time+message as it was in the original twtxt.txt file? is it because it's just not store anyplace?\n\nalso how set in stone is using user+url? vs user@domain? the latter would mean the url could change without invalidating the hash.
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic when i get the code up to a shareable level ill ping with what i have.
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic so.. convert the 4 attributes in the struct to private, add getters plus some the other methods that make sense.\n\n
go\ntype Twt interface {\n\tTwter()        Twter\n\tText()         string\n\tMarkdownText() string\n\tCreated()      time.Time\n    ... \n}\n
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic yeah I do. \n\nIt seems a bit wonky that it imports from your packages in some places. I'm guessing that's some legacy bits that need updates?
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic I have some ideas to improve on twtxt. figure I can contribute some. ๐Ÿ˜ bit more work and it will almost be a drop in replacement for [ParseFile](https://github.com/jointwt/twtxt/blob/master/internal/twt.go#) \n\nKinda wish [types.Twt](https://github.com/jointwt/twtxt/blob/master/types/twt.go#) was an interface. it's sooo close.
xuu@txt.sour.is
txt.sour.is
View Thread
@lyxal @prologic yah. the service can have a flag for allowing non-TLS for development. but by default ignores. \n\nare there some users that use alternative protos for twtxt? like ftp/gopher/dnsfs ๐Ÿค”
xuu@txt.sour.is
txt.sour.is
View Thread
My latest work over the last few days. a twtxt parser. so far looking promising. Faster and less memory than the regex version. ๐Ÿ˜
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic @lyxal blocking http would be a good start
xuu@txt.sour.is
txt.sour.is
View Thread
@admin @lyxal hax?
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic โค๏ธ
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic an added benefit of the avatar: would be the user could put their gravatar/libravatar image url like https://key.sour.is/avatar/01bc6186d015218c23dec55447e502e669ca4c61c7566dfcaa1cac256108dff0
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic Could the config be embeded into the head comment of the twtxt.txt file and parsed out? If it also had an avatar: field that pointed to where the avatar image is located it can be almost all self contained.
xuu@txt.sour.is
txt.sour.is
View Thread
New Blog Post Test Blog by @xuu ๐Ÿ“
xuu@txt.sour.is
txt.sour.is
View Thread
@lyxal @prologic if we edit the txt file does it update on web?
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic the HKP is http keyserver protocol. it's what happens when you do gpg --send-keys\n\nmakes a POST to the keyserver with your pubkey.
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic looking through the drafts it looks like it actually used SRV records as recently as 2018 ๐Ÿ˜ต
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic Web Key Directory: a way to self host your public key. instead of using a central system like pgp.mit.net or OpenPGP.org you have your key on a server you own. \n\nit takes an email@address.com hashes the part before the @ and turns it into [openpgpkey.]address.com/.well-known/openpgpkey[/address.com]/<hash>
xuu@txt.sour.is
txt.sour.is
View Thread
@xuu With SRV you can set what hostname to be used (and port/priority/etc)
xuu@txt.sour.is
txt.sour.is
View Thread
@xuu Not too happy with WKD's use of CNAME over SRV for discovery of openpgpkey.. That breaks using SNI pretty quick. I suppose it was setup as a temporary workaround anyhow in the [RFC..](https://tools.ietf.org/html/draft-koch-openpgp-webkey-service-11#.1)
xuu@txt.sour.is
txt.sour.is
View Thread
Did some work on WKD handling. Can update keys with HKP posts :) Ugh need to work on docs and unit tests. Boooorrring.
xuu@txt.sour.is
txt.sour.is
View Thread
Happy Friday.
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic also :)
xuu@txt.sour.is
txt.sour.is
View Thread
@adi @prologic One reservation about using it with a small community would be the expectation that the discussions at some level stay within the circle as opposed to the internet at large.
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic @twtxt I have noticed that I will get some duplicate web mention notifications. some kind of dedup would be helpful.
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic (#gqg3gea) ha yeah. COVID makes for a timey-wimey mish-mash. Worked on some WKD and fought with my XMPP client a bit.
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic Herro! ๐Ÿ‘‹
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic well nice chat. it's off to bed for me.
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic do you think twt will ever add ActivityPub integration?
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic Yep! installed it yesterday. I like the simplicity of twt. I am quite happy with how little memory the pod seems to use. Mastodon and the "lightweight" Pleroma don't work well in small VMs.
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic \n> That way at least we can form some kind of cryptographic โ€œidentityโ€ without having to involve the users that much, it just worksโ„ข\n\ni like some of the work that keys.pub is doing with ed25519 crypto keys with something like that.
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic huh.. true.. the email is md5/sha256 before storing.. if twtxt acted as provider you would store that hash and point the SRV record to the pod. .. to act as a client it would need to store the hash and the server that hosts the image.
xuu@txt.sour.is
txt.sour.is
View Thread
@xuu @prologic something that would be interesting would be libravatar for the user image. i made one that does the same for a profile cover image.
xuu@txt.sour.is
txt.sour.is
View Thread
@xuu @prologic The gpg command line leaves much to be desired...
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic it is some interesting work to decentralize all the things.. tricky part is finding tooling. i am using a self hacked version of the go openpgp library. A tool to add and remove notations would need to be local since it needs your private key.
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic this is a go version of Keyoxide.org that runs all server side. which is based on work from https://metacode.biz/openpgp/\n\nOpenPGP has a part of the self signature reserved for notatinal data. which is basically a bunch of key/values. \n\nthis site tries to emulate the identity proofs of keybase but in a more decentralized/federation way. \n\nmy next steps are to have this project host WKD keys which is kinda like a self hosting of your pgp key that are also discoverable with http requests. \n\nthen to add a new notation for following other keys. where you can do a kind of web of trust.
xuu@txt.sour.is
txt.sour.is
View Thread
This is an OpenPGP proof that connects my OpenPGP key to this Twtxt account. See https://key.sour.is/id/me@sour.is for more.\n\n[Verifying my OpenPGP key: openpgp4fpr:20AE2F310A74EA7CEC3AE69F8B3B0604F164E04F]
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic Oh snap. that's what i get for copy paste! ill just have to repost and update my key.
xuu@txt.sour.is
txt.sour.is
View Thread
@xuu can i notify myself?
xuu@txt.sour.is
txt.sour.is
View Thread
@prologic this is a very curious project. I would love to see how it manages to do its federation between pods.
xuu@txt.sour.is
txt.sour.is
View Thread
hello twt!