$ inspect-db yarns.db | jq -r '.Value.URL' | awk '{ total += length; count++ } END { if (count > 0) print total / count }'
40.3387
Given an RFC3339 UTC timestamp has a length of 20 characters with seconds precision. We're talking about Twt Subject taking up ~63 characters/bytes on average._~
$ inspect-db yarns.db | jq -r '.Value.URL' | awk '{ total += length; count++ } END { if (count > 0) print total / count }'
40.3387
Given an RFC3339 UTC timestamp has a length of 20 characters with seconds precision. We're talking about Twt Subject taking up ~63 characters/bytes on average._~
- @xuu would see an increase of ~20%
- @falsifian would see an increase of ~8%
- @bender would see an increase of ~20%
- @lyse would see an increase of ~15%
- @aelaraji would see an increase of ~13%
- @sorenpeter would see an increase of ~8%
- @movq would see an increase of ~9%
Just from a scalability standpoint along I'm not seeing a switch to location-based Twt ids to support threading a good idea here. This is what I meant when I said to @david in a recent call that we open up a new can of worms (_or new set of problems_) by drastically changing the approach, rather than incrementally improving the existing approach we have today (_which has served us well for the past 4 years already_0.~_
- @xuu would see an increase of ~20%
- @falsifian would see an increase of ~8%
- @bender would see an increase of ~20%
- @lyse would see an increase of ~15%
- @aelaraji would see an increase of ~13%
- @sorenpeter would see an increase of ~8%
- @movq would see an increase of ~9%
Just from a scalability standpoint along I'm not seeing a switch to location-based Twt ids to support threading a good idea here. This is what I meant when I said to @david in a recent call that we open up a new can of worms (_or new set of problems_) by drastically changing the approach, rather than incrementally improving the existing approach we have today (_which has served us well for the past 4 years already_0._~
Apologies, I can't edit the poll once it's live, so the suggestion on feedback for supporting Markdown will have to be discussed at another time.
Apologies, I can't edit the poll once it's live, so the suggestion on feedback for supporting Markdown will have to be discussed at another time.
$ ./compare.sh
Original file size: 28145 bytes
Modified file size: 70672 bytes
Percentage increase in file size: 151.10%
...
$ ./compare.sh
Original file size: 28145 bytes
Modified file size: 70672 bytes
Percentage increase in file size: 151.10%
...
With the proposal to switch to location based addressing using a pointer to a feed and a timestamp in that feed you're looking at roughly 2025 characters long because both the HTTP and HTML and even URI specifications do not specify maximum length for URI(s) AFAIK only recommendations.
With the proposal to switch to location based addressing using a pointer to a feed and a timestamp in that feed you're looking at roughly 2025 characters long because both the HTTP and HTML and even URI specifications do not specify maximum length for URI(s) AFAIK only recommendations.
http://polljunkie.com/poll/xdgjib/twtxt-v2
http://polljunkie.com/poll/xdgjib/twtxt-v2
And yea since yarnd has a store it's a bit easier to support edit / delete actions 😅
And yea since yarnd has a store it's a bit easier to support edit / delete actions 😅
2024-09-07T12:55:56Z 🥳 NEW FEED: @<twtxt http://edsu.github.io/twtxt/twtxt.txt>
2024-09-07T12:55:56Z 🥳 NEW FEED: @<kdy https://twtxt.kdy.ch/twtxt.txt>
2024-09-07T12:55:56Z\t🥳 NEW FEED: @<twtxt http://edsu.github.io/twtxt/twtxt.txt>
2024-09-07T12:55:56Z\t🥳 NEW FEED: @<kdy https://twtxt.kdy.ch/twtxt.txt>
2024-09-07T12:55:56Z 🥳 NEW FEED: @<twtxt http://edsu.github.io/twtxt/twtxt.txt>
2024-09-07T12:55:56Z 🥳 NEW FEED: @<kdy https://twtxt.kdy.ch/twtxt.txt>
yarnd and I suspect jenny too. and as indicated in here quite easy to have a reference implementation written in Bash with standard UNIX tools.~_
yarnd and I suspect jenny too. and as indicated in here quite easy to have a reference implementation written in Bash with standard UNIX tools._~
$ ./twtxt-v2.sh reply 242561ce02d "Cool! 👌"
Posted twt with hash: b2c938f9838
...
$ ./twtxt-v2.sh timeline
...
prologic@twtxt.net [2024-09-22T07:26:37Z] <242561ce02d> Okay folks, I've spent all day on this today, and I _think_ its in "good enough"™ shape to share:
**Twtxt v2**:
- Specification: https://docs.mills.io/uJXuisaYTRWYDrl8A2jADg?both
- implementation: https://gist.mills.io/prologic/afdec15443da4d7aa898f383f171ec1b
prologic@localhost [2024-09-22T07:51:16Z] <b2c938f9838> Cool! 👌 (reply-to:242561ce02d)
$ ./twtxt-v2.sh reply 242561ce02d "Cool! 👌"
Posted twt with hash: b2c938f9838
...
$ ./twtxt-v2.sh timeline
...
prologic@twtxt.net [2024-09-22T07:26:37Z] <242561ce02d> Okay folks, I've spent all day on this today, and I _think_ its in "good enough"™ shape to share:
**Twtxt v2**:
- Specification: https://docs.mills.io/uJXuisaYTRWYDrl8A2jADg?both
- implementation: https://gist.mills.io/prologic/afdec15443da4d7aa898f383f171ec1b
prologic@localhost [2024-09-22T07:51:16Z] <b2c938f9838> Cool! 👌 (reply-to:242561ce02d)
Twtxt v2:
- Specification: https://docs.mills.io/uJXuisaYTRWYDrl8A2jADg?both
- implementation: https://gist.mills.io/prologic/afdec15443da4d7aa898f383f171ec1b
Twtxt v2:
- Specification: https://docs.mills.io/uJXuisaYTRWYDrl8A2jADg?both
- implementation: https://gist.mills.io/prologic/afdec15443da4d7aa898f383f171ec1b
yarnd, hosted it temporarily (_I think locally_) and used it to poison the caches of a few production pods.Thankfully the gossip protocol used by
yarnd as part of its "peering" between pods isn't fully trusted, twts are not archived for example into permanent storage. So the moment my pod re-fetched my own feed, the spoofed Twt was obliterated 😅Eventual consistency 🤣
yarnd, hosted it temporarily (_I think locally_) and used it to poison the caches of a few production pods.Thankfully the gossip protocol used by
yarnd as part of its "peering" between pods isn't fully trusted, twts are not archived for example into permanent storage. So the moment my pod re-fetched my own feed, the spoofed Twt was obliterated 😅Eventual consistency 🤣
I hope to see @david @movq @lyse @xuu @sorenpeter and hopefully others too @aelaraji @falsifian and anyone else that sees this! 🙏 We're _hopefully_ going to primarily discuss the future of Twtxt and the last few weeks of discussions 🤣
- Event: Yarn.social Online Meetup
- When: 28th September 2024 at 12:00pm UTC (midday)
- Where: Mills Meet : Yarn.social
- Cadence: 4th Saturday of every Month
Agenda:
- Let's talk about the upcoming changes to the Twtxt spec(s)
- See #xgghhnq
#Yarn.social #Meetup
I hope to see @david @movq @lyse @xuu @sorenpeter and hopefully others too @aelaraji @falsifian and anyone else that sees this! 🙏 We're _hopefully_ going to primarily discuss the future of Twtxt and the last few weeks of discussions 🤣
- Event: Yarn.social Online Meetup
- When: 28th September 2024 at 12:00pm UTC (midday)
- Where: Mills Meet : Yarn.social
- Cadence: 4th Saturday of every Month
Agenda:
- Let's talk about the upcoming changes to the Twtxt spec(s)
- See #xgghhnq
#Yarn.social #Meetup
- We increase the Hash length from
7 to 11.- We formalise the Update Commands extension.
- We amend the Twt Hash and Metadata extension to state:
> Feed authors that wish to change the location of their feed (_once Twts have been published_) must append a new
# url = comment to their feed to indicate the new location and thus change the "Hashing URI" used for Twts from _that_ point onward.This has implications of the "order" of a feed, and we should either do one of two things, either:
- Mandate that feeds are append-only.
- Or amend the Metadata spec with a new field that denotes the order of the feed so clients can make sense of "inline" comments in the feed. -- This would also imply that the default order is (_of course_) append-only. Suggestion:
# direction = [append|prepend]
- We increase the Hash length from
7 to 11.- We formalise the Update Commands extension.
- We amend the Twt Hash and Metadata extension to state:
> Feed authors that wish to change the location of their feed (_once Twts have been published_) must append a new
# url = comment to their feed to indicate the new location and thus change the "Hashing URI" used for Twts from _that_ point onward.This has implications of the "order" of a feed, and we should either do one of two things, either:
- Mandate that feeds are append-only.
- Or amend the Metadata spec with a new field that denotes the order of the feed so clients can make sense of "inline" comments in the feed. -- This would also imply that the default order is (_of course_) append-only. Suggestion:
# direction = [append|prepend]
yarnd to see how many things would break and how many assumptions there are around the idea of "Content Addressing"; here's where I'm at so far:- What breaks
Basically I'm at a point where spending time on this is going to provide very little value, there are assumptions made in the lextwt parser, assumptions made in yarnd, assumptions in the way storage is done and the way threading works and things are looked up. There are far reaching implications to changing the way Twts are identified here to be "location addressed" that I'm quite worried about the amount of effort would be required to change
yarnd here.
yarnd to see how many things would break and how many assumptions there are around the idea of "Content Addressing"; here's where I'm at so far:- What breaks
Basically I'm at a point where spending time on this is going to provide very little value, there are assumptions made in the lextwt parser, assumptions made in yarnd, assumptions in the way storage is done and the way threading works and things are looked up. There are far reaching implications to changing the way Twts are identified here to be "location addressed" that I'm quite worried about the amount of effort would be required to change
yarnd here.
> What if someone completely changes the content of the root of the thread?
Does the Subject reference the feed and timestamp only or the intent too?
> What if someone completely changes the content of the root of the thread?
Does the Subject reference the feed and timestamp only or the intent too?
- Twt Subjects lose their meaning.
- Twt Subjects cannot be verified without looking up the feed.
- Which may or may not exist anymore or may change.
- Two persons cannot reply to a Twt independently of each other anymore.
_and probably some other properties we'd stand to lose that I'm forgetting about..._
- Twt Subjects lose their meaning.
- Twt Subjects cannot be verified without looking up the feed.
- Which may or may not exist anymore or may change.
- Two persons cannot reply to a Twt independently of each other anymore.
_and probably some other properties we'd stand to lose that I'm forgetting about..._
(replyto:…) proposal (_location addressing vs. content addressing_) is that you just introduce a similar problem down the track, albeit rarer where if a feed changes its location, your thread's "identifiers" are no longer valid, unless those feed authors maintain strict URL redirects, etc. This potentially has the long-term effect of being rather fragile, as opposed to what we have now where an Edit just really causes a natural fork in the thread, which is how "forking" works in the first place.I realise this is a bit pret here, and it probably doesn't matter a whole lot at our size. But I'm trying to think way ahead, to a point where Twtxt as a "thing" can continue to work and function decades from now, even with the extensions we've built. We've already proven for example that Twts and threads from ~4 years ago still work and are easily looked up haha 😝~
(replyto:…) proposal (_location addressing vs. content addressing_) is that you just introduce a similar problem down the track, albeit rarer where if a feed changes its location, your thread's "identifiers" are no longer valid, unless those feed authors maintain strict URL redirects, etc. This potentially has the long-term effect of being rather fragile, as opposed to what we have now where an Edit just really causes a natural fork in the thread, which is how "forking" works in the first place.I realise this is a bit pret here, and it probably doesn't matter a whole lot at our size. But I'm trying to think way ahead, to a point where Twtxt as a "thing" can continue to work and function decades from now, even with the extensions we've built. We've already proven for example that Twts and threads from ~4 years ago still work and are easily looked up haha 😝~