# I am the Watcher. I am your guide through this vast new twtiverse.
# 
# Usage:
#     https://watcher.sour.is/api/plain/users              View list of users and latest twt date.
#     https://watcher.sour.is/api/plain/twt                View all twts.
#     https://watcher.sour.is/api/plain/mentions?uri=:uri  View all mentions for uri.
#     https://watcher.sour.is/api/plain/conv/:hash         View all twts for a conversation subject.
# 
# Options:
#     uri     Filter to show a specific users twts.
#     offset  Start index for quey.
#     limit   Count of items to return (going back in time).
# 
# twt range = 1 21
# self = https://watcher.sour.is/conv/envp4vq
@lyse Hey, it's nice to hear that :D

I guess there are already many tools to do that (my brother uses one when he's riding a bike, with fancy animations and a 3D map), but when you make your own tool, it tastes differently.

Share with us if you revive it!
@eaplmx I wrote a few requirements for the rewrite and then got sick of it, so I started writing some code. It doesn't do much (just user CRUD), but I'm quite proud of the test coverage. :-) Comments of all sorts very welcome!
@lyse

> The thing yarnd uses seems a bit odd to Lyse.

I didn't choose it 😅 I have no experience in this kind of thing 😂
@lyse

> The thing yarnd uses seems a bit odd to Lyse.

I didn't choose it 😅 I have no experience in this kind of thing 😂
@lyse I also argue you can use a KV store here, but abstracting out the interface is a good idea anyway. The types of data you're wanting to store (exercises) are more like time-series (which most use KV stores anyway).

Recommend Bitcask of course 😅 Or Badger or Bolt
@lyse I also argue you can use a KV store here, but abstracting out the interface is a good idea anyway. The types of data you're wanting to store (exercises) are more like time-series (which most use KV stores anyway).

Recommend Bitcask of course 😅 Or Badger or Bolt
@lyse

> Maybe long-term: OAuth/etc. But that goes against the rule of simplicity and
puts a big hole in the privacy part.

I'd recommend implementing IndieAuth. You can even test against _any_ yarnd instance out there as they're all valid IndieAuth providers (or one locally even)
@lyse

> Maybe long-term: OAuth/etc. But that goes against the rule of simplicity and
puts a big hole in the privacy part.

I'd recommend implementing IndieAuth. You can even test against _any_ yarnd instance out there as they're all valid IndieAuth providers (or one locally even)
@prologic I'd probably go with gettext here. That's what I know and also seems to work quite well. Once I come to the web UI part. At the moment I started out with a REST API only.

We're using Badger at work and unfortunately it's the wrong database for us. Badger is well suited for high write workloads but we're mostly only reading. It's constantly reorganizing the database and wasting a lot of resources. Currently, we're trying to understand the details to reconfigure it, but we could not achieve a lot so far. Tomorrow, I have to continue research on that topic. :-( Also with all the other trouble it had caused us so far, I'll not use Badger in my hobby projects. We also noticed ristretto, the cache, somehow accesses /etc/passwd. No reaction from the developers. Development stalled and then lately continued in a new fork by the same guys but it's already discontinued for another fork and a lot of stuff they don't need anymore like vlogs, write transactions, etc. have been removed. A lot of stuff is changing at the moment.

Bolt is a candidate we even did a crude storage implementation with. The idle performance is great, basically zero, but when work has to happen, we obviously don't hit as good numbers as with Badger. Still good enough from what we've seen so far without spending time analyzing it in any detail. We would also have to bolt the data encryption to Bolt ourselves. That's where Badger really shines. But if you en-/decrypt data in the application, you're still be able to get some metadata from Bolt, like how many records are there, how big are they etc. From my understanding Badger encrypts the whole database and derives dedicated encryption keys from the master key which are then even rotated automatically.

So Bolt could be something to try. I'll probably also have a deeper look into Bitcask some day this week. However, with these more "esoteric" databases debugging probably gets harder. For well-hung, more traditional solutions like SQLite etc. there are lots of viewers and tools out there. Not sure if that is the case for the others, too.

Ok, I have to check out IndieAuth. Actually, I have no idea about it.
@lyse Use Bitcask 😆 Designed for high write and read 😂
@lyse Use Bitcask 😆 Designed for high write and read 😂
@lyse If you're going to seriously look at Bitcask (which I use in all my projects, including yarnd) you _should_ know some things up front:

- Be aware of a design flaw / bug #238 whereby your program will deadlock if you attempt to do .Get()(s) inside of a .Scan() or .Fold(). This is a known problem, so avoid doing this. The work-around is to copy the keys you want to work with, then perform your operations on them separately.

With that out of the way:

Hopefully most of the information on the README will be enough to get you started and understand performance characteristics and the overall API.

Couple of additional things to note:

- There export/dump commands to help you debug your database. As well as import/restore the other way around.

For example, I use this frequently:

h
$ bitcask -p ./dinofs.db dump | jq '. | map_values(@base64d)'


...continued...
@lyse If you're going to seriously look at Bitcask (which I use in all my projects, including yarnd) you _should_ know some things up front:

- Be aware of a design flaw / bug #238 whereby your program will deadlock if you attempt to do .Get()(s) inside of a .Scan() or .Fold(). This is a known problem, so avoid doing this. The work-around is to copy the keys you want to work with, then perform your operations on them separately.

With that out of the way:

Hopefully most of the information on the README will be enough to get you started and understand performance characteristics and the overall API.

Couple of additional things to note:

- There export/dump commands to help you debug your database. As well as import/restore the other way around.

For example, I use this frequently:

h
$ bitcask -p ./dinofs.db dump | jq '. | map_values(@base64d)'


...continued...
- There is also FUSE filesystem called bitcaskfs you can use to mount your database and treat it like a filesystem for debugging purposes or whatever.

- And finally if you need High Availability or a Server<->Client model there is the builtin bitcaskd which provides a Redis-compatible API for a server/client model or the bitraft which provides Raft-based HA with a Redis-compatible API.
- There is also FUSE filesystem called bitcaskfs you can use to mount your database and treat it like a filesystem for debugging purposes or whatever.

- And finally if you need High Availability or a Server<->Client model there is the builtin bitcaskd which provides a Redis-compatible API for a server/client model or the bitraft which provides Raft-based HA with a Redis-compatible API.
@lyse Wow, that's a good definition for the problem to solve. Was a pleasant surprise.
(Random story ahead) The name is pretty relatable for me since I heard that word in a Popular Mechanics magazine a few years ago about the band.

Do you have some screenshots or a quick video to watch how is it going? I know programmers are more text-based but an appealing image is always useful to share the visual idea behind it.

Something that has happened to me with FitBot is that if I leave the project in a theoretical way, only writing about it, and thinking about how to create it, I get sick of the idea. So I need some users (starting with me) to feel the product/software/service is alive.
@eaplmx Stay tuned but don't expect a lot. I'll upload some screenshots when I return from our short hike. If I will make it home, that is. I'm gonna melt when I just open the door for a fraction of a second.
@eaplmx Writing some explanations took forever. Also had to start it three times on my old machine where I had a working development setup. And still I missed a couple of things to truely showcase everything, but my bed is calling. There you go: https://lyse.isobeef.org/kraftwerk1/
@lyse Also curious how you plan to write a Web UI without any Javascript -- But you started off by writing a Rest API? Are those going to be two separate things?
@lyse Also curious how you plan to write a Web UI without any Javascript -- But you started off by writing a Rest API? Are those going to be two separate things?
@lyse Thanks dude, I'll take a look. If you need any specific feedback or to ping-pong ideas, throw me an @