# I am the Watcher. I am your guide through this vast new twtiverse.
# 
# Usage:
#     https://watcher.sour.is/api/plain/users              View list of users and latest twt date.
#     https://watcher.sour.is/api/plain/twt                View all twts.
#     https://watcher.sour.is/api/plain/mentions?uri=:uri  View all mentions for uri.
#     https://watcher.sour.is/api/plain/conv/:hash         View all twts for a conversation subject.
# 
# Options:
#     uri     Filter to show a specific users twts.
#     offset  Start index for quey.
#     limit   Count of items to return (going back in time).
# 
# twt range = 1 57
# self = https://watcher.sour.is/conv/5nagu7q
@prologic Re: Chat system, What if the base specification included a system for per-user arbitrary JSON storage on the server? Kind of like XEP-0049, but expanded upon. Two kinds of objects: public and private. Public objects can be queried by anyone, private objects cannot and must be encrypted with the user's private key. Public keys could be stored there, as well as anything else defined by extensions. Roster, user block list, avatar, etc.
@mckinley Sounds okay. So just a simple KV Store / Object store right? Private objects are things specific a a User and it's Client? Contact List, Blocklist, Message History, etc?
@mckinley Sounds okay. So just a simple KV Store / Object store right? Private objects are things specific a a User and it's Client? Contact List, Blocklist, Message History, etc?
@prologic Yes, but I was thinking more along the lines of auxiliary data like contact lists. Message history might be better off with its own system that provides more powerful query options and pagination.
Do you think it would be feasible to encrypt the private object names as well?
@mckinley I see... But... If we are to build an actual goodness to gracious self-hosted and decentralised Chat system (ala IndieWeb-style) surely the "data" would be small right? I mean do you really have more than a few dozen folks you _actually_ regularly "talk to" on any given chat system/paltform?

My thinking here is that a simple KV store is enough here and place the burden on the "client" to deal with things like querying the data.
@mckinley I see... But... If we are to build an actual goodness to gracious self-hosted and decentralised Chat system (ala IndieWeb-style) surely the "data" would be small right? I mean do you really have more than a few dozen folks you _actually_ regularly "talk to" on any given chat system/paltform?

My thinking here is that a simple KV store is enough here and place the burden on the "client" to deal with things like querying the data.
@prologic bookmarking for the weekend's call.
@prologic The way I picture this, and I may be way off from where you are, the client needs to be able to make the following requests for messages:
1. All messages from and to all users sent after X date (when starting up)
2. All messages from and to a specific user sent before Y date (when viewing history)
All of this would need to utilize pagination so we aren't sending 500 messages to a client all at once.
These query options could be added to the JSON object store I'm proposing, but I don't see how they might be used outside of storing messages.
@prologic The way I picture this, and I may be way off from where you are, the client needs to be able to make the following requests for messages:
1. All messages from and to all users sent after X date (when starting up)
2. All messages from and to a specific user sent before Y date (when viewing history)

All of this would need to utilize pagination so we aren't sending 500 messages to a client all at once.
These query options could be added to the JSON object store I'm proposing, but I don't see how they might be used outside of storing messages.
@mckinley Ahhh I see what you're saying now... I was thinking more along the lines of, if messages are e2e encrypted, there is no need to keep history since there's no possible way to view history from a new client, etc. But with a self-hosted broker/server design this is actually possible and probably needs to be a thing, so you're right I think.

Shall we focus on the design/spec of the actual base protocol first though? I'd like to get a design/spec we can all agree on and a poc server/client implemented with a goal of I can message @mckinley@mckinley.cc
@mckinley Ahhh I see what you're saying now... I was thinking more along the lines of, if messages are e2e encrypted, there is no need to keep history since there's no possible way to view history from a new client, etc. But with a self-hosted broker/server design this is actually possible and probably needs to be a thing, so you're right I think.

Shall we focus on the design/spec of the actual base protocol first though? I'd like to get a design/spec we can all agree on and a poc server/client implemented with a goal of I can message @mckinley@mckinley.cc
@prologic Wait, I've been assuming that one account = one private key. Is that not your intention? If the same private key is used on all devices, a new client should be able to see old messages. That's not the model of XMPP+OMEMO, and it may be a challenge to use mobile devices as well as "real" computers with the same account, but it is a possibility. If one account = one private key it makes the private JSON store much less complicated. It would streamline group chats as well, which is a problem for XMPP+OMEMO.
@prologic Wait, I've been assuming that one account = one private key. Is that not your intention? If the same private key is used on all devices, a new client should be able to see old messages. That's not the model of XMPP+OMEMO, and it may be a challenge to use mobile devices as well as "real" computers with the same account, but it wouldn't be insurmountable. If one account = one private key it makes the private JSON store much less complicated. It would streamline group chats as well, which is a problem for XMPP+OMEMO.
@mckinley Hmmm need to think about this some more I believe. I'm pretty sure Apple's iMessages and even Signal uses the notion of "Device Keys". To be honest though, I _think_ this rather complicates things at the spec and implementation level, and that's something I'd like to avoid straight up.
@mckinley Hmmm need to think about this some more I believe. I'm pretty sure Apple's iMessages and even Signal uses the notion of "Device Keys". To be honest though, I _think_ this rather complicates things at the spec and implementation level, and that's something I'd like to avoid straight up.
I _personally_ think it's okay to share the same "Private Key" with "all your devices" as long as it is a) transported securely and b) stored at rest securely.
I _personally_ think it's okay to share the same "Private Key" with "all your devices" as long as it is a) transported securely and b) stored at rest securely.
For instance I normally use the same RSA key/pair on all my workstations for my ssh client, because that's me, no-matter where I am. The only exception to this rule is I usually create a separate key for any "work" / " company" I am a part of.
For instance I normally use the same RSA key/pair on all my workstations for my ssh client, because that's me, no-matter where I am. The only exception to this rule is I usually create a separate key for any "work" / " company" I am a part of.
@prologic I do the same with SSH keys and share that exception. I would also prefer to have a separate IM account for work. The only advantages I can think of to a device key setup are as follows:
1. You can verify that a device key belongs to the person to whom you mean to send messages. If you verify the keys with someone in person and suddenly start receiving messages from their account with a different key, you'll know something's going on.
2. It's easier to register new clients, because you can just type in a username and a password.
If a username and a private key is all that's needed to sign in and there is no username/password/TOFU then an impersonation attack like that would be equivalent to the attacker grabbing an OMEMO private key from a trusted device, is it not? Point #1 would effectively become irrelevant, right? Plus, how many people are *really* meeting up in person to verify their OMEMO keys?
If a username and a private key is all that's needed to sign in and there is no username/password/TOFU then an impersonation attack like that would be equivalent to the attacker grabbing an OMEMO private key from a trusted device, is it not? Point #1 would effectively become irrelevant, right? Plus, how many people are *really* meeting up in person to verify their OMEMO keys?
I have a very weak understanding of cryptography, I'm sure there's something I'm missing.
I _think_ the general rule of thumb here is if you accept TOFU (Trust On First Use) and your "Private Key" is compromised "somehow", you should re-create a new Key pair and let your contacts know. In turn your contacts _should_ delete their "trust" of the old key and mark it as "invalid".

The problem with things like certificate or key revocation as is possible with GPG (IHMO) is that this makes the system a bit centralised where you have to rely on publishing your key(s) or certificates to a "key server" and if your key(s) having to publish a revocation.
I _think_ the general rule of thumb here is if you accept TOFU (Trust On First Use) and your "Private Key" is compromised "somehow", you should re-create a new Key pair and let your contacts know. In turn your contacts _should_ delete their "trust" of the old key and mark it as "invalid".

The problem with things like certificate or key revocation as is possible with GPG (IHMO) is that this makes the system a bit centralised where you have to rely on publishing your key(s) or certificates to a "key server" and if your key(s) having to publish a revocation.
For example with Signal, if your keys changes (which are generated and stored on your device, but AFAIK there are no per-device keys), if your keys change your contacts get alerted of this fact (UX problem) and you have an opportunity to "verify their new keys" (somehow).
For example with Signal, if your keys changes (which are generated and stored on your device, but AFAIK there are no per-device keys), if your keys change your contacts get alerted of this fact (UX problem) and you have an opportunity to "verify their new keys" (somehow).
Do you know protonmail, they use PGP for all the mail between the users of their own platform.
@tkanos Keyword there being "their platform" πŸ˜…
@tkanos Keyword there being "their platform" πŸ˜…
I would HIGHLY recommend reading up on the keybase architecture. They designed device key system for real time chat that is e2e secure. https://book.keybase.io/security

A property of ec keys is deriving new keys that can be determined to be "on curve." bitcoin has some BIPs that derive single use keys for every transaction connected to a wallet. And be derived as either public or private chains. https://qvault.io/security/bip-32-watch-only-wallets/
I would HIGHLY recommend reading up on the keybase architecture. They designed device key system for real time chat that is e2e secure. https://book.keybase.io/security

A property of ec keys is deriving new keys that can be determined to be "on curve." bitcoin has some BIPs that derive single use keys for every transaction connected to a wallet. And be derived as either public or private chains. https://qvault.io/security/bip-32-watch-only-wallets/
https://book.keybase.io/docs/chat/crypto
https://book.keybase.io/docs/chat/crypto
Thanks @xuu I will.
I was also reading the cryptographic part of Scuttlebutt: https://ssbc.github.io/scuttlebutt-protocol-guide/
Reading Keybase.io as suggested by @xuu . Very good for now
But the real issue is that if you use an app it’s easy to store the private asymmetrical key and use it β€œlocally” but through web …. Maybe somehow the site can load/ask it locally and store it in the browser cache and then use it in JavaScript to decrypt
But the real issue is that if you use an app it’s easy to store the private asymmetrical key and use it β€œlocally” but through web …. Maybe somehow the site can load/ask it locally and store it in the browser cache.
Here are my thought about the subject : https://git.mills.io/tkanos/encrption-twtxt-paper/wiki/Research
@tkanos This is a good write up and good summary of the approaches πŸ‘Œ This is specifically for encrypted feeds however and one thing you're missing is encrypted feeds should be a separate feed in the first place and marked as such.
@tkanos This is a good write up and good summary of the approaches πŸ‘Œ This is specifically for encrypted feeds however and one thing you're missing is encrypted feeds should be a separate feed in the first place and marked as such.
The other thing I wanted to point out is that this discussion around creating a spec for a IndieWeb-style chat system (inspired somewhat by twtxt too). Your write-up is more akin to the current RFC for Encrypted Feeds -- Which you are welcome to review and share your thoughts and comments on.

Personally I _think_ Encrypted feeds is a much easier "thing" to support.
The other thing I wanted to point out is that this discussion around creating a spec for a IndieWeb-style chat system (inspired somewhat by twtxt too). Your write-up is more akin to the current RFC for Encrypted Feeds -- Which you are welcome to review and share your thoughts and comments on.

Personally I _think_ Encrypted feeds is a much easier "thing" to support.
Thanks, Yes of course, I was just doing a global overview. Of the basis, to be able to build on it.
And I was not aware of that RFC. Will be reading it.
Thanks, Yes of course, I was just doing a global overview. Of the basis, to be able to build on it.
Yes of course, I was just doing a global overview. Of the basis, to be able to build on it.
Actually it's good : RFC for Encrypted Feeds
@tkanos Thanks! I _think_ it just needs a few eyes on it, and probably a few things clarified, but I _think_ the general approach there that @xuu and I came up with I _think_ will work.
@tkanos Thanks! I _think_ it just needs a few eyes on it, and probably a few things clarified, but I _think_ the general approach there that @xuu and I came up with I _think_ will work.
What you did will be perfect for a one to one conversation.
But if I want to tell to everybody that I follow something encrypted. It won't be possible.
My bad, it will apparently be possible, I just don;t fully understand how, but keys can encrupt a message o multiple recipient.

echo -n "My secret πŸ€“" | keys encrypt -armor -sender gabriel@github \\
-recipient kex1ts0qw8fwkvle2f2xsqumetmr9ev5ppx22rl5hnycen68sanjzl7qnta629 \\
-recipient kex1ts0qw8fwkvle2f2xsqumetmr9ev5ppx22rl5hnycen68sanjzl7qnta627 > msg.enc
My bad, it will be possible.
My bad, it will apparently be possible, I just don't fully understand how, but keys can encrypt a message to multiple recipient.

echo -n "My secret πŸ€“" | keys encrypt -armor -sender gabriel@github \\
-recipient kex1ts0qw8fwkvle2f2xsqumetmr9ev5ppx22rl5hnycen68sanjzl7qnta629 \\
-recipient kex1ts0qw8fwkvle2f2xsqumetmr9ev5ppx22rl5hnycen68sanjzl7qnta627 > msg.enc
My bad, it will apparently be possible, I just don't fully understand how, but keys can encrupt a message o multiple recipient.

echo -n "My secret πŸ€“" | keys encrypt -armor -sender gabriel@github \\
-recipient kex1ts0qw8fwkvle2f2xsqumetmr9ev5ppx22rl5hnycen68sanjzl7qnta629 \\
-recipient kex1ts0qw8fwkvle2f2xsqumetmr9ev5ppx22rl5hnycen68sanjzl7qnta627 > msg.enc
My bad, it will apparently be possible, I just don't fully understand how, but keys can encrypt a message to multiple recipient.

echo -n "My secret πŸ€“" | keys encrypt -armor -sender gabriel@github \
-recipient kex1ts0qw8fwkvle2f2xsqumetmr9ev5ppx22rl5hnycen68sanjzl7qnta629 \
-recipient kex1ts0qw8fwkvle2f2xsqumetmr9ev5ppx22rl5hnycen68sanjzl7qnta627 > msg.enc
@tkanos That's right πŸ‘Œ
@tkanos That's right πŸ‘Œ