# I am the Watcher. I am your guide through this vast new twtiverse.
#
# Usage:
# https://watcher.sour.is/api/plain/users View list of users and latest twt date.
# https://watcher.sour.is/api/plain/twt View all twts.
# https://watcher.sour.is/api/plain/mentions?uri=:uri View all mentions for uri.
# https://watcher.sour.is/api/plain/conv/:hash View all twts for a conversation subject.
#
# Options:
# uri Filter to show a specific users twts.
# offset Start index for quey.
# limit Count of items to return (going back in time).
#
# twt range = 1 61083
# self = https://watcher.sour.is?uri=https://twtxt.net/user/prologic/twtxt.txt&offset=37191
# next = https://watcher.sour.is?uri=https://twtxt.net/user/prologic/twtxt.txt&offset=37291
# prev = https://watcher.sour.is?uri=https://twtxt.net/user/prologic/twtxt.txt&offset=37091
@abucci Thanks tip and reminder ๐ I always forget the special keyword to type on those rare occasions you need to bypass the bad/invalid cert. And yes I can confirm nitter.net is having cert issues, I actually confirmed this earlier today but forgot to mention to you... 
@abucci Thanks tip and reminder ๐ I always forget the special keyword to type on those rare occasions you need to bypass the bad/invalid cert. And yes I can confirm nitter.net is having cert issues, I actually confirmed this earlier today but forgot to mention to you...
@tkanos Right. I think @lyse, @abucci and I are converging (get it? ๐
) on a few key ideas:
- Converge currently exchanges missing twts between peering pods (It used to validate this, and so that should be fixed)
- Pods can mark Twts for deletion for feeds they follow where twts found in cache/archive that no longer exist in the feed.
- A similar function can be "codified" that supports periodic exchange of deletions
- A "trust" mechanism that involves human operators to build a a level of trust of peering pods that _can_ participate in deletions.
Did I miss anything so far? ๐ค
@tkanos Right. I think @lyse, @abucci and I are converging (get it? ๐
) on a few key ideas:
- Converge currently exchanges missing twts between peering pods (It used to validate this, and so that should be fixed)
- Pods can mark Twts for deletion for feeds they follow where twts found in cache/archive that no longer exist in the feed.
- A similar function can be "codified" that supports periodic exchange of deletions
- A "trust" mechanism that involves human operators to build a a level of trust of peering pods that _can_ participate in deletions.
Did I miss anything so far? ๐ค
Nope I suck! ๐
Nope I suck! ๐
@carsten It was called "Mario Bros" ๐
-- Gonna see if I can still play it ๐
@carsten It was called "Mario Bros" ๐
-- Gonna see if I can still play it ๐
@marado This kind of reminds me of umm that similar platform game where you bump turtles? shit what was it called?! ๐คฆโโ๏ธ
@marado This kind of reminds me of umm that similar platform game where you bump turtles? shit what was it called?! ๐คฆโโ๏ธ
So... The very long and controversial discussion many of us have been having over the last 24-48hrs or so... Would someone be willing to try to attempt to summarise all the different points and viewpoints we've made thus far into either a single Twt or an Issue so we can continue this in some logical fashion? ๐
So... The very long and controversial discussion many of us have been having over the last 24-48hrs or so... Would someone be willing to try to attempt to summarise all the different points and viewpoints we've made thus far into either a single Twt or an Issue so we can continue this in some logical fashion? ๐
@lyse You are right. Probably not worth doing that, has complexities of its own ๐
@lyse You are right. Probably not worth doing that, has complexities of its own ๐
@marado Actually that's been one of the goals all along when I first started out on this journey. So yes, we try very hard to ensure that whatever we do we build open standards for. Aside from the fact we use Markdown (but pretty much everyone else does too) I _think_ we've documented everything so far...
@marado Actually that's been one of the goals all along when I first started out on this journey. So yes, we try very hard to ensure that whatever we do we build open standards for. Aside from the fact we use Markdown (but pretty much everyone else does too) I _think_ we've documented everything so far...
@ocdtrekkie It's an interesting read indeed and the saga has been quite funny overall so far ๐
@ocdtrekkie It's an interesting read indeed and the saga has been quite funny overall so far ๐
@marado Also to be clear, feed redirects when a user deletes their account/feeds would be handled by the backend, no operator involvement required.
@marado Also to be clear, feed redirects when a user deletes their account/feeds would be handled by the backend, no operator involvement required.
@eaplmx To be honest I _think_ the Mobile App will be unaffected by any changes we make here as its basically a client to the Yarn API ๐
@eaplmx To be honest I _think_ the Mobile App will be unaffected by any changes we make here as its basically a client to the Yarn API ๐
@marado No need for any apologies ๐
Super glad to have someone like you around and welcome back btw to Yarn.social / Twtxt ๐ค
I _think_ there was a good reason to only allow the # prev field to be relative, so that feed authors (malicious ones) couldn't just arbitrarily point a feed to some other random feed that wasn't there or something. I _think_ @movq wrote the spec if I'm not mistaken. I agreed with it at the time, I think I still do and yarnd fully implements it as of a few weeks ago.
Why do you think redirects won't work for deleted feeds/accounts? I've seen that before I _think_ on Bitbucket of all places, where if you delete a repo, you can tell the system where the new location will be.
@marado No need for any apologies ๐
Super glad to have someone like you around and welcome back btw to Yarn.social / Twtxt ๐ค
I _think_ there was a good reason to only allow the # prev field to be relative, so that feed authors (malicious ones) couldn't just arbitrarily point a feed to some other random feed that wasn't there or something. I _think_ @movq wrote the spec if I'm not mistaken. I agreed with it at the time, I think I still do and yarnd fully implements it as of a few weeks ago.
Why do you think redirects won't work for deleted feeds/accounts? I've seen that before I _think_ on Bitbucket of all places, where if you delete a repo, you can tell the system where the new location will be.
@eaplmx Yeah the question is what does that mean in this discussion ๐
-- Also its not that we had to build moderation features to satisfy the App/Play stores for the Mobile Ap (Goryon), it was mostly really just misunderstanding. Most of the folks that go through the app submission approval process don't even understand their own guidelines. At some point we had to rename buttons to different labels just to get it approved, even though the functionality was the same. There are no "moderation" features at all and hopefully we can keep things that way.
@eaplmx Yeah the question is what does that mean in this discussion ๐
-- Also its not that we had to build moderation features to satisfy the App/Play stores for the Mobile Ap (Goryon), it was mostly really just misunderstanding. Most of the folks that go through the app submission approval process don't even understand their own guidelines. At some point we had to rename buttons to different labels just to get it approved, even though the functionality was the same. There are no "moderation" features at all and hopefully we can keep things that way.
@mckinley I was also thinking in general (need to make sure we have a feature request / issue for this) that the redirection would say last for 90 days before the pod stopped redirecting the feed.
@mckinley I was also thinking in general (need to make sure we have a feature request / issue for this) that the redirection would say last for 90 days before the pod stopped redirecting the feed.
@eaplmx Some of the main ideas and philosophy are documented here some of the mechanisms above like muting a feed and/or twt (or an entire yarn) had to be built to satisfy "content policies" for the mobile app for the APp and Play stores. That is to say, controls needed to exists for users to control what content they could or could not see. Apparently wasn't enough to just unfollow a user/feed. ๐คฆโโ๏ธ -- Which I guess is fair enough as the same content could be visible to that user on the Discover view. But of course if you logout and use a pod's web interface anonymously, we have no idea who you are, so all bets are off ๐
Honestly the whole "content" policy guidelines of both App and Play stores are just kind of silly and super pointless to me, and don't really make a terrible lot of sense in a "decentralised" world.
@eaplmx Some of the main ideas and philosophy are documented here some of the mechanisms above like muting a feed and/or twt (or an entire yarn) had to be built to satisfy "content policies" for the mobile app for the APp and Play stores. That is to say, controls needed to exists for users to control what content they could or could not see. Apparently wasn't enough to just unfollow a user/feed. ๐คฆโโ๏ธ -- Which I guess is fair enough as the same content could be visible to that user on the Discover view. But of course if you logout and use a pod's web interface anonymously, we have no idea who you are, so all bets are off ๐
Honestly the whole "content" policy guidelines of both App and Play stores are just kind of silly and super pointless to me, and don't really make a terrible lot of sense in a "decentralised" world.
@tkanos Haha that would be pretty funny actually ๐
I've done it before by accident, its kind of weird, and then you realise, "oh shit I muted myself" ๐
@tkanos Haha that would be pretty funny actually ๐
I've done it before by accident, its kind of weird, and then you realise, "oh shit I muted myself" ๐
@mckinley Oh that's probably a good point actually. Sorry, my mind is all over the place right now ๐
@mckinley Oh that's probably a good point actually. Sorry, my mind is all over the place right now ๐
As @mckinley its an important discussion to be had, so let's get this right ๐ -- We've gotten so many other things right, I have full confidence we can get this right too ๐ค
As @mckinley its an important discussion to be had, so let's get this right ๐ -- We've gotten so many other things right, I have full confidence we can get this right too ๐ค
@abucci Please don't apologise, we _knew_ this day would come. We just have to figure out the best way to solve this without breaking our own goals and vision.
@abucci Please don't apologise, we _knew_ this day would come. We just have to figure out the best way to solve this without breaking our own goals and vision.
@mckinley In order to complete "this ease" I _thin_ the only missing feature right now is "Delete and Redirect". That is to say, a user downloads their feed, deletes their account, puts in a redirect on the old pod and moves to a new one.
@mckinley In order to complete "this ease" I _thin_ the only missing feature right now is "Delete and Redirect". That is to say, a user downloads their feed, deletes their account, puts in a redirect on the old pod and moves to a new one.
I _actually_ wonder whether we're over thinking things a bit here entirely...
We already have several user controls:
- Unfollow a feed
- Mute a feed
- Mute a twt
And the pod operator of a multi-user pod can:
- Delete a feed
- Delete an account
- Shadow ban a feed
Because pods are sort of "distributed" I _really_ wonder whether we need to be all that concerned with deleting archived twts across the network at all? ๐ค It opens up a huge can of worms that _might_ be rather tricky to solve "right now"...
I _honestly_ think the best way to handle this as we grow/scale with more pods in the Yarn.social network is to just build up a strong positive community and just have a "zero tolerance" attitude towards abuse and just nuke offending feeds/accounts without question.
Hmmm? ๐ค As long as the software yarnd makes it easy for those (hopefully rare) abusive users to move their feeds elsewhere and take their data with them, maybe this is enough? ๐ค
I dunno ๐คทโโ๏ธ
I _actually_ wonder whether we're over thinking things a bit here entirely...
We already have several user controls:
- Unfollow a feed
- Mute a feed
- Mute a twt
And the pod operator of a multi-user pod can:
- Delete a feed
- Delete an account
- Shadow ban a feed
Because pods are sort of "distributed" I _really_ wonder whether we need to be all that concerned with deleting archived twts across the network at all? ๐ค It opens up a huge can of worms that _might_ be rather tricky to solve "right now"...
I _honestly_ think the best way to handle this as we grow/scale with more pods in the Yarn.social network is to just build up a strong positive community and just have a "zero tolerance" attitude towards abuse and just nuke offending feeds/accounts without question.
Hmmm? ๐ค As long as the software yarnd makes it easy for those (hopefully rare) abusive users to move their feeds elsewhere and take their data with them, maybe this is enough? ๐ค
I dunno ๐คทโโ๏ธ
@mckinley Ooof sorry no what I meant to say was yes your understanding is correct, and I also agree with you that we should try to avoid "policing" here.
@mckinley Ooof sorry no what I meant to say was yes your understanding is correct, and I also agree with you that we should try to avoid "policing" here.
@mckinley
> @prologic It seems very straight forward to do this automatically. When I delete my own post, how is that currently propagated to other pods?
In one of two ways:
- Users on the other pod follow your feed and therefore that pod has those Twts(s) archvied.
- Pods exchange "missing twts" each each other in a sort of "gossip" protocol.
@mckinley
> @prologic It seems very straight forward to do this automatically. When I delete my own post, how is that currently propagated to other pods?
In one of two ways:
- Users on the other pod follow your feed and therefore that pod has those Twts(s) archvied.
- Pods exchange "missing twts" each each other in a sort of "gossip" protocol.
@mckinley
> If a pod admin decides to delete a post on his pod as you have, that deletion should eventually be propagated throughout the network
The problem is this. โ๏ธ This is _very hard_ to do "automatically" without having to also worry about "malicious" pods or software propagating unintended or unwanted "deletes"
@mckinley
> If a pod admin decides to delete a post on his pod as you have, that deletion should eventually be propagated throughout the network
The problem is this. โ๏ธ This is _very hard_ to do "automatically" without having to also worry about "malicious" pods or software propagating unintended or unwanted "deletes"
@mckinley Your understanding is correct. And there inlines the debate @lyse and I had this morning. There is a conflict of "right to be forgotten" in EU GDPR law(s), what to do with clear violations of abuse (on your pod, and how it affects other peering pods or the network as a whole_)
I agree with you, I have no rights to determine what should or should not happen on say @jlj's pod as much as he does mine. The default Abuse Policy is just that, the default._
@mckinley Your understanding is correct. And there inlines the debate @lyse and I had this morning. There is a conflict of "right to be forgotten" in EU GDPR law(s), what to do with clear violations of abuse (on your pod, and how it affects other peering pods or the network as a whole_)
I agree with you, I have no rights to determine what should or should not happen on say @jlj's pod as much as he does mine. The default Abuse Policy is just that, the default._
So maybe we can not worry about the "operator abuse" aspect, as that _might_ be a moot point as there is no "proprietary data formats" here and tools exist to move your feed anywhere else you want/desire.
I _think_ for multi-user pods, _some_ level of trust has to exist between its users (a pod's small community) and its operator -- I mean after-all I run twtxt.net as a multi-user pod free of charge and free from ads, etc, etc ๐
So maybe we can not worry about the "operator abuse" aspect, as that _might_ be a moot point as there is no "proprietary data formats" here and tools exist to move your feed anywhere else you want/desire.
I _think_ for multi-user pods, _some_ level of trust has to exist between its users (a pod's small community) and its operator -- I mean after-all I run twtxt.net as a multi-user pod free of charge and free from ads, etc, etc ๐
@abucci Actually I agree with this. We already have all the tools a user needs to basically migrate their feed off a pod to another or even host their own. The only thing left I can think of is a feature to "Delete and Redirect" your feed to somewhere else.
@abucci Actually I agree with this. We already have all the tools a user needs to basically migrate their feed off a pod to another or even host their own. The only thing left I can think of is a feature to "Delete and Redirect" your feed to somewhere else.
@jason I don't really follow? ๐ค
@jason I don't really follow? ๐ค
@abucci In real terms though won't that just be a rogue network of pods distinct and different but running the same (or modified) software? ๐ค This is a hard problem to solve, because it a "human problem". My view has always been that, if you believe in that kind of stuff, and like to be with other abusive groups of people, fine. But be on your own, and in the open. In other words, do we care? We would just not peer with pods like that or follow feeds from such folks? Right? ๐ค Being a decentralised network the power is in your hands, however the original Yarn is really about how we deal with abuse on our pods...
Hmmm ๐ค I _really_ do feel like we've opened a can of worms here ๐
@abucci In real terms though won't that just be a rogue network of pods distinct and different but running the same (or modified) software? ๐ค This is a hard problem to solve, because it a "human problem". My view has always been that, if you believe in that kind of stuff, and like to be with other abusive groups of people, fine. But be on your own, and in the open. In other words, do we care? We would just not peer with pods like that or follow feeds from such folks? Right? ๐ค Being a decentralised network the power is in your hands, however the original Yarn is really about how we deal with abuse on our pods...
Hmmm ๐ค I _really_ do feel like we've opened a can of worms here ๐
@eaplmx No, not really. In an ideal world, n operator of a multi-user pod would just nuke an account entirely that violates the (default) Abuse Policy -- So really in hindsight that's what should have happened I guess, but maybe I was just being nice in this instance... Problem of course is that this now opens up a can of worms...
In the past we've had some interesting folks swing by and post interesting stuff (to say the least) -- you know the type of vulgar crap that nobody really cares for, or spam, etc. You don't generally have to do anything about that because you as an individual can choose to "Unfollow" that feed, don't follow them in the first place, mute them, mute the hash in question (for yourself), etc.
This is more of a case of direction violation of the abuse policy / community guidelines and more of how we want to shape things -- without going too far down the "let's moderate" everything BS
@eaplmx No, not really. In an ideal world, n operator of a multi-user pod would just nuke an account entirely that violates the (default) Abuse Policy -- So really in hindsight that's what should have happened I guess, but maybe I was just being nice in this instance... Problem of course is that this now opens up a can of worms...
In the past we've had some interesting folks swing by and post interesting stuff (to say the least) -- you know the type of vulgar crap that nobody really cares for, or spam, etc. You don't generally have to do anything about that because you as an individual can choose to "Unfollow" that feed, don't follow them in the first place, mute them, mute the hash in question (for yourself), etc.
This is more of a case of direction violation of the abuse policy / community guidelines and more of how we want to shape things -- without going too far down the "let's moderate" everything BS
Thinking about this some more... We could implement some kind of majority voting system whereby pods will only delete a Twt by hash from its archives there is a majority of votes within a network of peering pods? ๐ค This would avoid any kind of abuse, or mitigate it, as >50% would have to agree ๐
(oh wait where have I seen this before?! ๐คฆโโ๏ธ)
Thinking about this some more... We could implement some kind of majority voting system whereby pods will only delete a Twt by hash from its archives there is a majority of votes within a network of peering pods? ๐ค This would avoid any kind of abuse, or mitigate it, as >50% would have to agree ๐
(oh wait where have I seen this before?! ๐คฆโโ๏ธ)
@tkanos That's precisely what we're trying to avoid ๐คฃ
@tkanos That's precisely what we're trying to avoid ๐คฃ
@mckinley Yes this is "moderation" I guess, strictly speaking. Which I'm not too happy about, and _seems_ unavoidable. Your comment raises an interesting question... Of whether we should do this at all, for risk of being abused to moderate away "unpopular" Twts across the network just because a few (a cooperative) don't like them? (but are otherwise not in clear violation of community guidelines?) ๐ค
@mckinley Yes this is "moderation" I guess, strictly speaking. Which I'm not too happy about, and _seems_ unavoidable. Your comment raises an interesting question... Of whether we should do this at all, for risk of being abused to moderate away "unpopular" Twts across the network just because a few (a cooperative) don't like them? (but are otherwise not in clear violation of community guidelines?) ๐ค
@lyse Oh good point ๐ Yea I'll fix this ๐ Mind filing an issue against the search repo? ๐
@lyse Oh good point ๐ Yea I'll fix this ๐ Mind filing an issue against the search repo? ๐
As discussed on IRC just now (g'night @lyse ๐ด) couple of things we agree on:
- The API was always written like this, so changing it now is "too much work" ๐
so we'll leave it the way it is.
- We don't complete agree on the "automatic deletion" just yet, and there are clear problems with this when pods exchange twts with each other (Convergence)
- We should document a design proposal and discuss
As discussed on IRC just now (g'night @lyse ๐ด) couple of things we agree on:
- The API was always written like this, so changing it now is "too much work" ๐
so we'll leave it the way it is.
- We don't complete agree on the "automatic deletion" just yet, and there are clear problems with this when pods exchange twts with each other (Convergence)
- We should document a design proposal and discuss
The only reason its there in the first place and the only reason I use Cloudflare at all is so my home infra doesn't get DDoS'd ๐
The only reason its there in the first place and the only reason I use Cloudflare at all is so my home infra doesn't get DDoS'd ๐
Morning all ๐ฅฑ โ๏ธx1 
Morning all ๐ฅฑ โ๏ธx1
@marado Essentially a 3-way sync ๐
@marado Essentially a 3-way sync ๐
@marado Did you know you can sync your feed now quite easily with the Sync API and yarnc sync command? ๐ค
@marado Did you know you can sync your feed now quite easily with the Sync API and yarnc sync command? ๐ค