# I am the Watcher. I am your guide through this vast new twtiverse.
# 
# Usage:
#     https://watcher.sour.is/api/plain/users              View list of users and latest twt date.
#     https://watcher.sour.is/api/plain/twt                View all twts.
#     https://watcher.sour.is/api/plain/mentions?uri=:uri  View all mentions for uri.
#     https://watcher.sour.is/api/plain/conv/:hash         View all twts for a conversation subject.
# 
# Options:
#     uri     Filter to show a specific users twts.
#     offset  Start index for quey.
#     limit   Count of items to return (going back in time).
# 
# twt range = 1 61083
# self = https://watcher.sour.is?uri=https://twtxt.net/user/prologic/twtxt.txt&offset=29291
# next = https://watcher.sour.is?uri=https://twtxt.net/user/prologic/twtxt.txt&offset=29391
# prev = https://watcher.sour.is?uri=https://twtxt.net/user/prologic/twtxt.txt&offset=29191
@mutefall Re RFC 3339 timestamps, if I understand you correctly, I _think_ it's extremely unlikely for someone to repost a Twt (an edit) within the same second (at least not humanly possible). In any case, I've only validated the ideas so far in isolation, the algorithm(s) need to be built, feature gated, measured, understood and finally put in place with some UX (I like @ullarah's gugestion)
@tkanos is that somehow the same as:

> you are free to have your own opinion as long as that opinion matches mine

πŸ€¦β€β™‚οΈπŸ€¦β€β™‚οΈπŸ€¦β€β™‚οΈ
@tkanos is that somehow the same as:

> you are free to have your own opinion as long as that opinion matches mine

πŸ€¦β€β™‚οΈπŸ€¦β€β™‚οΈπŸ€¦β€β™‚οΈ
@mutefall Consider yourself bookmarked I will remind you what's your Twt has reached the age of two weeks πŸ˜†
@mutefall Consider yourself bookmarked I will remind you what's your Twt has reached the age of two weeks πŸ˜†
@david I'm reading Create an Apple ID for your child - Apple Support and I have some questions for you... (I've not used this yet):

- Can I control who my children can and cannot contact/message/etc?
@david I'm reading Create an Apple ID for your child - Apple Support and I have some questions for you... (I've not used this yet):

- Can I control who my children can and cannot contact/message/etc?
Can anyone recommend a safe, private and secure way for my children to have a way to message each other? Some kind of Chat / IM... Their primary devices are iPad(s) so it has to run on iPad OS or it must be an installable PWA (cc @markwylde @justamoment in case you know of any PWA App here)

Ideally I would either like to be in control of the server-side -- self-hosted with accounts I can control and who/what they are able to connect to, etc.

Or, completely local to a local network (i.e: p2p, wifi only).
Can anyone recommend a safe, private and secure way for my children to have a way to message each other? Some kind of Chat / IM... Their primary devices are iPad(s) so it has to run on iPad OS or it must be an installable PWA (cc @markwylde @justamoment in case you know of any PWA App here)

Ideally I would either like to be in control of the server-side -- self-hosted with accounts I can control and who/what they are able to connect to, etc.

Or, completely local to a local network (i.e: p2p, wifi only).
@mutefall They are essentially these
@mutefall They are essentially these
@mutefall The ideas I have in mind to deal wit this are basically to get good at "detecting edits" in the first place at ingestion time. I've played around with a few "text similarity" algorithms and I _think_ we can reasonably (with high confidence) say that Twt A' was an edit of Twt A -- We _would_ cache and archive them both, but in the User Interface collapse them and show the Twt A' (with a visual indication/link that it was an edit of Twt A)
@mutefall The ideas I have in mind to deal wit this are basically to get good at "detecting edits" in the first place at ingestion time. I've played around with a few "text similarity" algorithms and I _think_ we can reasonably (with high confidence) say that Twt A' was an edit of Twt A -- We _would_ cache and archive them both, but in the User Interface collapse them and show the Twt A' (with a visual indication/link that it was an edit of Twt A)
Hmmm do we still have a small issue somewhere? I didn't see my last message on this Yarn hit your pod πŸ€”
Hmmm do we still have a small issue somewhere? I didn't see my last message on this Yarn hit your pod πŸ€”
@ullarah Nice πŸ‘Œ
@ullarah Nice πŸ‘Œ
@ullarah I absolutely cannot do this kind of shit anymore πŸ˜‚
@ullarah I absolutely cannot do this kind of shit anymore πŸ˜‚
Hmmm πŸ€” Did I finally fix the last niggling little issue with the websub feature? πŸ€” -- For those of you who are running main or prologic/yarnd:latest and have websub enabled, please update to commit c6d87580 or later πŸ™‡β€β™‚οΈ
Hmmm πŸ€” Did I finally fix the last niggling little issue with the websub feature? πŸ€” -- For those of you who are running main or prologic/yarnd:latest and have websub enabled, please update to commit c6d87580 or later πŸ™‡β€β™‚οΈ
@mutefall Bahahahaha πŸ˜‚ If you wanna see what mine looks like here you go:

@mutefall Bahahahaha πŸ˜‚ If you wanna see what mine looks like here you go:

@mutefall It's pretty easy to delete or even edit a Twt you posted on Yarn.social πŸ˜‚ -- But it has unintended side-effects, due to the decentralised nature, you end up with UX problems where for example, someone makes a Twt A, realizes they've made a typo or mistake or something, then edits it (which is equivalent to delete + repost) and posts a new Twt A'

Dealing with this is hardβ„’ But I have some ideas πŸ˜…
@mutefall It's pretty easy to delete or even edit a Twt you posted on Yarn.social πŸ˜‚ -- But it has unintended side-effects, due to the decentralised nature, you end up with UX problems where for example, someone makes a Twt A, realizes they've made a typo or mistake or something, then edits it (which is equivalent to delete + repost) and posts a new Twt A'

Dealing with this is hardβ„’ But I have some ideas πŸ˜…
@mckinley Okay got the contents of the file.txt, but is there more to this? the contents makes no sense πŸ˜‚
@mckinley Okay got the contents of the file.txt, but is there more to this? the contents makes no sense πŸ˜‚
Morning ! πŸ₯±
Morning ! πŸ₯±
Okay it's 1am now. G'night folks I'm done πŸ€— #Closed
Okay it's 1am now. G'night folks I'm done πŸ€— #Closed
I keep getting errors like:


Unzip - Array buffer allocation failed
I keep getting errors like:


Unzip - Array buffer allocation failed
@mckinley I'm not sure I can solve this. I _think_ I've gotten as far as I can, but the password? Hmmm πŸ˜…
@mckinley I'm not sure I can solve this. I _think_ I've gotten as far as I can, but the password? Hmmm πŸ˜…
Anyone else coming?

@tkanos and I are here ! πŸ€—
Anyone else coming?

@tkanos and I are here ! πŸ€—
@carsten what's a CTF?
@carsten what's a CTF?
@tkanos

> Is this in relation to the so-called (dead and unmaintained) β€œreference client” ?
>
>> What ???????

Sorry dude, I withdraw my comment as I had no idea what each of your bullet points were referring to πŸ˜‚ I blame myself and being tired and having been up too many hours this week dealing with work 😒
@tkanos

> Is this in relation to the so-called (dead and unmaintained) β€œreference client” ?
>
>> What ???????

Sorry dude, I withdraw my comment as I had no idea what each of your bullet points were referring to πŸ˜‚ I blame myself and being tired and having been up too many hours this week dealing with work 😒
@justamoment Yup πŸ‘Œ I can help with the iOS side πŸ‘Œ And if we get this right I'd like to publish them as "Yarn" in both stores as "Yarn.social"
@justamoment Yup πŸ‘Œ I can help with the iOS side πŸ‘Œ And if we get this right I'd like to publish them as "Yarn" in both stores as "Yarn.social"
@tkanos well it'll be 10 PM for me πŸ˜‚ Time zones πŸ˜†
@tkanos well it'll be 10 PM for me πŸ˜‚ Time zones πŸ˜†
@caesar @tkanos @mutefall I _hope_ you guys can make it to this πŸ‘Œ There are lots of things I _think_ we should discuss. Also the regulars @thecanine @david @lyse @movq @ullarah if you can make it and anyone else that's interested!
@caesar @tkanos @mutefall I _hope_ you guys can make it to this πŸ‘Œ There are lots of things I _think_ we should discuss. Also the regulars @thecanine @david @lyse @movq @ullarah if you can make it and anyone else that's interested!
@tkanos Is this in relation to the so-called (dead and unmaintained) "reference client" ? πŸ€” I _would_ consider @<~duriny https://envs.net/~duriny/twtxt.txt>'s client twtr to be the new reference client once complete πŸ‘Œ
@tkanos Is this in relation to the so-called (dead and unmaintained) "reference client" ? πŸ€” I _would_ consider @<~duriny https://envs.net/~duriny/twtxt.txt>'s client twtr to be the new reference client once complete πŸ‘Œ
My backup tools for backing up local Docker Volumes in my Swarm Cluster is:


$ for node in $(seq -s ' ' -f 'dm%g' 1 6); do backup-all-volumes "${node}"; done


πŸ˜…
My backup tools for backing up local Docker Volumes in my Swarm Cluster is:


$ for node in $(seq -s ' ' -f 'dm%g' 1 6); do backup-all-volumes "${node}"; done


πŸ˜…
@mutefall Notice is here
@mutefall Notice is here
@justamoment

> I didn’t know about the plan to replace Goryon, how do you manage to compile it for Play store and the others?

I _believe_ we can package it up with with Cordova. But first let's just get it working and installed/deployed to https://yarn.social/app πŸ‘Œ As it's a PWA it's immediately installable on pretty much every device/os anyway that supports PWA (quite a few now)
@justamoment

> I didn’t know about the plan to replace Goryon, how do you manage to compile it for Play store and the others?

I _believe_ we can package it up with with Cordova. But first let's just get it working and installed/deployed to https://yarn.social/app πŸ‘Œ As it's a PWA it's immediately installable on pretty much every device/os anyway that supports PWA (quite a few now)
@mckinley So far I can only determine that it is some kind of compressed archive that _might_ contain a file.txt in it, but I can't be certain. I'm actually not really that familiar with CyberChef, it looks pretty cool though, probably just need to get used to it!
@mckinley So far I can only determine that it is some kind of compressed archive that _might_ contain a file.txt in it, but I can't be certain. I'm actually not really that familiar with CyberChef, it looks pretty cool though, probably just need to get used to it!
cc @eldersnake @david @jlj @laz @carsten @thecanine @mutefall @markwylde @deebs @caesar (probably forgetting a few folks, sorry!) -- Please have a look at the proposed layout/designs @justamoment is proposing in a new Mobile / Desktop app rewrite of Goryon (being rewritten as a PWA/SPA with MithrilJS) All opinions matter, even if you've never used Goryon before πŸ™‡β€β™‚οΈ
cc @eldersnake @david @jlj @laz @carsten @thecanine @mutefall @markwylde @deebs @caesar (probably forgetting a few folks, sorry!) -- Please have a look at the proposed layout/designs @justamoment is proposing in a new Mobile / Desktop app rewrite of Goryon (being rewritten as a PWA/SPA with MithrilJS) All opinions matter, even if you've never used Goryon before πŸ™‡β€β™‚οΈ
@mckinley Judging by the size of your feed currently and how frequently / infrequently you post to it, I'd say you won't have to worry for another couple of years 🀣


$ curl -v -o /dev/null https://twtxt.net/user/mckinley/twtxt.txt 2>&1 | grep -i 'Content-Length'
< content-length: 107361
@mckinley Judging by the size of your feed currently and how frequently / infrequently you post to it, I'd say you won't have to worry for another couple of years 🀣


$ curl -v -o /dev/null https://twtxt.net/user/mckinley/twtxt.txt 2>&1 | grep -i 'Content-Length'
< content-length: 107361
@mckinley It is isn't it πŸ˜‚ I still think it has one more minor issue I'm trying to iron out but yeah it's working great 🀣
@mckinley It is isn't it πŸ˜‚ I still think it has one more minor issue I'm trying to iron out but yeah it's working great 🀣
πŸ“£ FYI: Yarn.social video conf today, all are welcome, open floor, bring your ideas, opinions or just come socialise with us! πŸ€—

Yarn.social Video Conf

- Date/Time:
- Saturday, March 12 Β· 22:00 – 23:00 (AEST)
- Saturday, March 12 Β· 12:00 – 13:00 (UTC)
- Google Meet joining info
- Video call link: https://meet.google.com/xmr-syif-qkq
- Or dial: β€ͺ(US) +1 314-474-3042‬ PIN: β€ͺ331 203 613‬#)

> Apologies, but we'll use Google Meet this time, as sometimes folks can't join with my PeerCalls instance very well 😒

#Yarn.social #Conf
πŸ“£ FYI: Yarn.social video conf today, all are welcome, open floor, bring your ideas, opinions or just come socialise with us! πŸ€—

Yarn.social Video Conf

- Date/Time:
- Saturday, March 12 Β· 22:00 – 23:00 (AEST)
- Saturday, March 12 Β· 12:00 – 13:00 (UTC)
- Google Meet joining info
- Video call link: https://meet.google.com/xmr-syif-qkq
- Or dial: β€ͺ(US) +1 314-474-3042‬ PIN: β€ͺ331 203 613‬#)

> Apologies, but we'll use Google Meet this time, as sometimes folks can't join with my PeerCalls instance very well 😒

#Yarn.social #Conf
@mckinley Not yet... Coming soonβ„’ πŸ˜… I was a little slower to adopt that particular spec. I'll get around it it don't worry!
@mckinley Not yet... Coming soonβ„’ πŸ˜… I was a little slower to adopt that particular spec. I'll get around it it don't worry!
One more thing @caesar I forgot to add here is that the Cache Size and TTL are actually configurable at a Pod level via the -I, --max-cache-items and -C, --max-cache-ttl options which default to 150 and 240h by default. As you are a user on my pod at twtxt.net, these settings directly impact you. If you were to run your own pod (for example) you could choose to tweak these to your 'taste". @david for example runs his pod netbros.com with quite high Cache settings.
One more thing @caesar I forgot to add here is that the Cache Size and TTL are actually configurable at a Pod level via the -I, --max-cache-items and -C, --max-cache-ttl options which default to 150 and 240h by default. As you are a user on my pod at twtxt.net, these settings directly impact you. If you were to run your own pod (for example) you could choose to tweak these to your 'taste". @david for example runs his pod netbros.com with quite high Cache settings.
@tkanos That's right it does! I noticed you were testing a Feed hosted on a Github repo of yours πŸ€” It was lacking any Metadata -- The three most important ones are:


# Description =
# Avatar =
# Nick =


As documented in Yarn.social :: Manually πŸ‘Œ
@tkanos That's right it does! I noticed you were testing a Feed hosted on a Github repo of yours πŸ€” It was lacking any Metadata -- The three most important ones are:


# Description =
# Avatar =
# Nick =


As documented in Yarn.social :: Manually πŸ‘Œ
@mckinley Oh is this a challenge? πŸ€” I have to say this is new territory for me, can you give some guideance/hints? I'm a total n00b at this πŸ˜‚
@mckinley Oh is this a challenge? πŸ€” I have to say this is new territory for me, can you give some guideance/hints? I'm a total n00b at this πŸ˜‚
There are very good technical reasons for this design, but there are also very good human reasons for this too .

As my old man said to me many moons ago when I was first designing this (he helped and contributed ideas here!):

> If I said something X ago, I don't want someone to say "Hey but X ago you said this". What if I've changed my mind since then and now have a different opinion?

I'm paraphrasing here of course, we talk regularly on the phone, but a lot of ideas ans inspiration has come from my Dad πŸ‘Œ -- The idea here is that Humans forget, so should Yarn.social
There are very good technical reasons for this design, but there are also very good human reasons for this too .

As my old man said to me many moons ago when I was first designing this (he helped and contributed ideas here!):

> If I said something X ago, I don't want someone to say "Hey but X ago you said this". What if I've changed my mind since then and now have a different opinion?

I'm paraphrasing here of course, we talk regularly on the phone, but a lot of ideas ans inspiration has come from my Dad πŸ‘Œ -- The idea here is that Humans forget, so should Yarn.social
One thing I want to point out is that this "problem" (per se, remember it's a design decision) also exists in other places like:

Cache expired posts vanish from threads with no warning - yarn - Mills

As Twts fall off the active Cache and are archived in an on-disk Archive, Yarns and Twts eventually "disappear" (they don't really, they are still searchable and accessible as everything is content addressable).
One thing I want to point out is that this "problem" (per se, remember it's a design decision) also exists in other places like:

Cache expired posts vanish from threads with no warning - yarn - Mills

As Twts fall off the active Cache and are archived in an on-disk Archive, Yarns and Twts eventually "disappear" (they don't really, they are still searchable and accessible as everything is content addressable).
@mutefall

> you’re reading from cache, so it’s quicker. memory will always have significantly faster iops vs disk-bound read operations. also recommend giving the codebase a look. there’s always room for contributors. i’m planning to take a crack at a few issues.

It's even more than just "memory is faster than disk". The Cache is designed to have O(1) lookups on all Profile (think Feed) and User Timeline as well as Pod Discover views. This is very important for the UX.
@mutefall

> you’re reading from cache, so it’s quicker. memory will always have significantly faster iops vs disk-bound read operations. also recommend giving the codebase a look. there’s always room for contributors. i’m planning to take a crack at a few issues.

It's even more than just "memory is faster than disk". The Cache is designed to have O(1) lookups on all Profile (think Feed) and User Timeline as well as Pod Discover views. This is very important for the UX.
@caesar

> As for pagination, sure, it can be hard, but why would it be harder in this case than in the cases where Yarn already does it?

It's done in the background as a background job. See this Dashbaord for a visuaul:



> (As for infinite scroll, if you have pagination on the server side already, it’s trivial on the client side. Yes you need JS of course, but not a lot)

Remember the builtin Web Interface (an SSR) is designed to be able to used without Javascript (graceful degradation).
@caesar

> As for pagination, sure, it can be hard, but why would it be harder in this case than in the cases where Yarn already does it?

It's done in the background as a background job. See this Dashbaord for a visuaul:



> (As for infinite scroll, if you have pagination on the server side already, it’s trivial on the client side. Yes you need JS of course, but not a lot)

Remember the builtin Web Interface (an SSR) is designed to be able to used without Javascript (graceful degradation).
@caesar

> Particularly with the apparent convention of the newest posts being at the bottom of the file.

This is generally the convenatio, yes. And folks like @lyse @xuu @movq and I have considered and talked about formalizing the "direction" of a feed including supporting "Range" requests. These are both things that I will likely do myself at some point, because it further helps with optimizing the traffic/bandwidth used and helps keeps things running smoothly as the network scales over time.
@caesar

> Particularly with the apparent convention of the newest posts being at the bottom of the file.

This is generally the convenatio, yes. And folks like @lyse @xuu @movq and I have considered and talked about formalizing the "direction" of a feed including supporting "Range" requests. These are both things that I will likely do myself at some point, because it further helps with optimizing the traffic/bandwidth used and helps keeps things running smoothly as the network scales over time.
@caesar

> but I’m a little puzzled why the same issues with a feed being huge don’t present an issue every time you want to poll for updates?

They do! As I said in af4el2q Pods will refuse to fetch feeds over the --max-fetch-limit in size. Feeds are also rotated on Pods. There is also a soec for this.
@caesar

> but I’m a little puzzled why the same issues with a feed being huge don’t present an issue every time you want to poll for updates?

They do! As I said in af4el2q Pods will refuse to fetch feeds over the --max-fetch-limit in size. Feeds are also rotated on Pods. There is also a soec for this.
@caesar Major downside hwo? I don't see it as a downside at all, it's the same idea as a logfile, and generally you have (or should have) rotation policies around it. How you manage this is up to you. yarnd for example has a builtin weekly job that ensures feeds on a pod are kept under 1MB in size (configurable) and are rotated when they exceed this. Pods will also refuse to fetch feeds over this size too.
@caesar Major downside hwo? I don't see it as a downside at all, it's the same idea as a logfile, and generally you have (or should have) rotation policies around it. How you manage this is up to you. yarnd for example has a builtin weekly job that ensures feeds on a pod are kept under 1MB in size (configurable) and are rotated when they exceed this. Pods will also refuse to fetch feeds over this size too.
@tkanos Hmmm? πŸ€” What error? What's the confusion? Can I help? πŸ€—
@tkanos Hmmm? πŸ€” What error? What's the confusion? Can I help? πŸ€—
@tkanos Hmmm it would appear this feed of yours using Github is missing some important Metadta? πŸ€”
@tkanos Hmmm it would appear this feed of yours using Github is missing some important Metadta? πŸ€”
@thecanine Because @novaburst said so? πŸ˜‚

> (this is on falkon, based on qtwebengine/chromium)
@thecanine Because @novaburst said so? πŸ˜‚

> (this is on falkon, based on qtwebengine/chromium)
@tkanos On Yarn.social? πŸ€”
@tkanos On Yarn.social? πŸ€”
@carsten Happy weekend! πŸ€— Any interesting plans? πŸ€”
@carsten Happy weekend! πŸ€— Any interesting plans? πŸ€”
@justamoment Knew there was a reason I like you πŸ˜… CSS is NOT something I'm any good at, @ullarah is orders of magnitude better than me at this πŸ˜‚ I _think_ between the two of you, we'll continue ti improve the UI/UX immensely πŸ€—
@justamoment Knew there was a reason I like you πŸ˜… CSS is NOT something I'm any good at, @ullarah is orders of magnitude better than me at this πŸ˜‚ I _think_ between the two of you, we'll continue ti improve the UI/UX immensely πŸ€—
@tkanos Roughly documented in a blog post at Future of Yarn.social