# I am the Watcher. I am your guide through this vast new twtiverse.
# 
# Usage:
#     https://watcher.sour.is/api/plain/users              View list of users and latest twt date.
#     https://watcher.sour.is/api/plain/twt                View all twts.
#     https://watcher.sour.is/api/plain/mentions?uri=:uri  View all mentions for uri.
#     https://watcher.sour.is/api/plain/conv/:hash         View all twts for a conversation subject.
# 
# Options:
#     uri     Filter to show a specific users twts.
#     offset  Start index for quey.
#     limit   Count of items to return (going back in time).
# 
# twt range = 1 7
# self = https://watcher.sour.is/conv/h75wtqq
lyse
lyse.isobeef.org
View Thread
Dang it! I ran into import cycles with shared test utilities again. :-( Either I have to copy this function to set up an in-memory test storage across packages or I have to put it in the storage package itself and guard it with a build tag that is only used in tests (otherwise I end up with this function in my production binary as well). I don't like any of the alternatives. :-(
xuu
txt.sour.is
View Thread
oof that sucks man. does it make sense to have a separate testutils package to import from?
xuu
txt.sour.is
View Thread
maybe even an internal that has the shared test stuff
lyse
lyse.isobeef.org
View Thread
@xuu My layout looks like this:

* storage/
* storage.go: defines a Storage interface
* sqlite.go: implements the Storage interface
* sqlite_test.go: originally had a function to set up a test storage to test the SQLite storage implementation itself: newRAMStorage(testing.T, $initialData) *Storage
* controller/
* feeds.go: uses a Storage
* feeds_test.go: here I wanted to reuse the newRAMStorage(…) function

I then tried to relocate the newRAMStorage(…) into a

* teststorage/
* storage.go: moved here as NewRAMStorage(…)

so that I could just reuse it from both

* storage/
* sqlite_test.go: uses testutils.NewRAMStorage(…)
* controller/
* feeds_test.go: uses testutils.NewRamStorage(…)

But that results into an import cycle, because the teststorage package imports storage for storage.Storage and the storage package imports testutils for testutils.NewRAMStorage(…) in its test. I'm just screwed. For now, I duplicated it as newRAMStorage(…) in controller/feeds_test.go.

I could put NewRAMStorage(…) in storage/testutils.go, which could be guarded with //go:build testutils. With go test -tags testutils …, in storage/sqlite_test.go could just use NewRAMStorage(…) directly and similarly in controller/feeds_test.go I could call storage.NewRamStorage(…). But I don't know if I would consider this really elegant.

The more I think about it, the more appealing it sounds. Because I could then also use other test-related stuff across packages without introducing other dedicated test packages. Build some assertions, converters, types etc. directly into the same package, maybe even make them methods of types.

If I went that route, I might do the opposite with the build tag and make it something like !prod instead of testing. Only when building the final binary, I would have to specify the tag to exclude all the non-prod stuff. Hmmm.
xuu
txt.sour.is
View Thread
@lyse OK. So how I have worked things like this out is to have the interface in the root package from the implementations. The interface doesn't need to be tested since it's just a contract. The implementations don't need to import storage.Storage

- storage/ defines the Storage interface (no tests!)
- storage/sqlite for the sqlite implementation tests for sqlite directly
- storage/ram for the ram implementation and tests for RAM directly
- controller/ can now import both storage and the implementation as needed.


So now I am guessing you wanted the RAM test for testing queries against sqlite and have it return some query response?

For that I usually would register a driver for SQL that emulates sqlite. Then it's just a matter of passing the connection string to open the registered driver on setup.

https://github.com/glebarez/go-sqlite?tab=readme-ov-file#connection-string-examples
xuu
txt.sour.is
View Thread
re reading so NewRAMStorage(…) is just something that setups your storage and initial data.. that can probably live with storage/sqlite. The point is the storage package does not import the implementations of storage.Storage It just defines the contract for things that use that interface. Now storage/sqlite CAN import storage and not have a circle dep.

It kinda works in reverse for import directions. usually you have your root package that imports things from deeper in the directory structures.. but for the case of interfaces it reverses where the deeper can import from parents but parents cannot import from children. 


- app < storage
      < storage/sqlite
      < controller < storage
                   < storage/sqlite
 
- sqlite < storage

- storage X storage/sqlite

lyse
lyse.isobeef.org
View Thread
Thanks, @xuu, great explanation. In another project I've structured it exactly like you wrote. The mock storage over there extends the SQLite storage and provides mechanism to return errors and such for testing purposes:

* storage/ defines the interface
* sqlite/ implements the storage interface
* mock/ extends the SQLite implementation by some mocking capabilities and assertions

Here, however, there are no storage subpackages. It's just storage, that's it. Everything is in there. The only implementation so far is an SQLite backend that resides in storage. My RAM storage is exactly that SQLite storage, but with :memory: instead a backing file on disk. I do not have a mock storage (yet).

I have to think about it a bit more, but I probably have to do exactly that in my tt rewrite, too. Sigh. I just have the feeling that in storage/sqlite/sqlite_test.go I cannot import storage/mock for the helper because storage/mock/mock.go imports and embeds the type from storage/sqlite. But I'm too tired right now to think clearly.