# I am the Watcher. I am your guide through this vast new twtiverse.
# 
# Usage:
#     https://watcher.sour.is/api/plain/users              View list of users and latest twt date.
#     https://watcher.sour.is/api/plain/twt                View all twts.
#     https://watcher.sour.is/api/plain/mentions?uri=:uri  View all mentions for uri.
#     https://watcher.sour.is/api/plain/conv/:hash         View all twts for a conversation subject.
# 
# Options:
#     uri     Filter to show a specific users twts.
#     offset  Start index for quey.
#     limit   Count of items to return (going back in time).
# 
# twt range = 1 196320
# self = https://watcher.sour.is?offset=164899
# next = https://watcher.sour.is?offset=164999
# prev = https://watcher.sour.is?offset=164799
@eldersnake But not me against your pod, so that's even weirder right? 🤔
Test

EDIT: Okay, convo works properly now at least
@prologic It's weird though cos I could reproduce it on any of my browsers on either my laptop or phone 🤔
@movq You wouldn't believe it, but I'm pretty sure @eldersnake and I found a nasty little bug in Firefox 126.0.1 🤣
@movq You wouldn't believe it, but I'm pretty sure @eldersnake and I found a nasty little bug in Firefox 126.0.1 🤣
Lot of testing going on here today. 🤣
Lot of testing going on here today. 🤣
Lot of testing going on here today. 🤣
Lot of testing going on here today. 🤣
@eldersnake I just installed 127.0.1 (64-bit) tonight and tested and it worked just fine. Try upgrading and roll that commit back and see if it still repros? 🤔 I'm almost willing to bet this is a bug 🐛
@eldersnake I just installed 127.0.1 (64-bit) tonight and tested and it worked just fine. Try upgrading and roll that commit back and see if it still repros? 🤔 I'm almost willing to bet this is a bug 🐛
@prologic Firefox 126.0.1 is my primary
@eldersnake Test
@eldersnake Test
What browser and version is this btw? Not sure if you mentioned?
What browser and version is this btw? Not sure if you mentioned?
@eldersnake Fuck'n hell 🤣 This has to be a browser bug!
@eldersnake Fuck'n hell 🤣 This has to be a browser bug!
@prologic Fix works!
Test
@eldersnake Looks legit 👌
@eldersnake Looks legit 👌
Okay just pushed:


* aa2f3ae9 - (HEAD -> main, origin/main) Workaround for this invalid Referer BS (6 seconds ago) <James Mills>
Okay just pushed:


* aa2f3ae9 - (HEAD -> main, origin/main) Workaround for this invalid Referer BS (6 seconds ago) <James Mills>
@prologic Woops my blockquote got mangled but you get the gist haha
@prologic I was wondering if my reverse proxy could cause something but it's pretty standard...


>server {
listen 80;
server_name we.loveprivacy.club;

location / {
return 301 https://$host$request_uri;
#proxy_pass http://127.0.0.1:8000;
}
}
server {
listen 443 ssl http2;
server_name we.loveprivacy.club;

ssl_certificate /etc/letsencrypt/live/we.loveprivacy.club/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/we.loveprivacy.club/privkey.pem;

client_max_body_size 8M;

location / {
proxy_pass http://127.0.0.1:8000;
}
}


>
Should I just code in a work-around? If the Referer is /post then consider that total bullshit, and ignore? 🤔
Should I just code in a work-around? If the Referer is /post then consider that total bullshit, and ignore? 🤔
Why would a Web Browser set the Referer header incorrectly?! 🤔
Why would a Web Browser set the Referer header incorrectly?! 🤔
Can't reproduce with Iridium either.
Can't reproduce with Iridium either.
Can't reproduce with Firefox.
Can't reproduce with Firefox.
Test
A initial post, reply and conv -> post (_bottom of page_) and nothing 🤔 All 6 requests are POST /post followed by a GET <from> where ever I was coming from. Hmmm 🧐 This is nuts 🌰
A initial post, reply and conv -> post (_bottom of page_) and nothing 🤔 All 6 requests are POST /post followed by a GET <from> where ever I was coming from. Hmmm 🧐 This is nuts 🌰
Test
Test
Test
Test
Test
Test
@eldersnake Yeah no worries!

You can see here, at least, htmx knows what the current URL is:


HX-Current-URL: https://we.loveprivacy.club/conv/vcpt7gq
Referer: https://we.loveprivacy.club/post


But the freak'n browser is setting the wrong value for Referer. There is simply no way to be on the /post endpoint normally anyway.
@eldersnake Yeah no worries!

You can see here, at least, htmx knows what the current URL is:


HX-Current-URL: https://we.loveprivacy.club/conv/vcpt7gq
Referer: https://we.loveprivacy.club/post


But the freak'n browser is setting the wrong value for Referer. There is simply no way to be on the /post endpoint normally anyway.
@prologic Firefox but it happens on Brave and Chromium on my laptop, or any browser (Brave, Chromium, Vanadium) I try on my Android phone.
What Browser is this again? Does it happen with any other browser?
What Browser is this again? Does it happen with any other browser?
@prologic Sorry, my messages don't get included in the current convo unless I tag you. Guess something gets lossed in translation with this weird posting issue. ANYWAY, it is rather perplexing. Clearly only an issue on my Pod, but what could the source of it be 🤔
Easy: 5.00 miles, 00:09:14 average pace, 00:46:11 duration
kept it in zone 2 pretty well
#running #treadmill
Easy: 5.00 miles, 00:09:14 average pace, 00:46:11 duration
kept it in zone 2 pretty well
#running #treadmill
Easy: 5.00 miles, 00:09:14 average pace, 00:46:11 duration
kept it in zone 2 pretty well
#running #treadmill
@eldersnake Ahh missed this. Fuck, that's just silliness and not right. wtf?!
@eldersnake Ahh missed this. Fuck, that's just silliness and not right. wtf?!
The POST /post XHR (_that is being run by htmx_) should never, ever be Referer: .../post 🤦‍♂️
The POST /post XHR (_that is being run by htmx_) should never, ever be Referer: .../post 🤦‍♂️
For example, this is what you *should* see:
For example, this is what you *should* see:
Yeah sorry just realised, but just checked again and the referrer is the same (/post) on either the POST or the GET 🤔
@eldersnake I mean on the POST /post request itself. That's the Referer that's looked up and used as the redirect.
@eldersnake I mean on the POST /post request itself. That's the Referer that's looked up and used as the redirect.
Testing
Indeed /post is referrer...

>GET /post HTTP/2
Host: we.loveprivacy.club
[...]
HX-Request: true
HX-Target: content
HX-Current-URL: https://we.loveprivacy.club/conv/vcpt7gq
Referer: https://we.loveprivacy.club/post
>
@prologic Okay lemme see
The only way this is possible is if the Referrer is /post and not / or /discover like it should be. Can we confirm/deny this if you inspect the POST /post requests? 🤔

See this code

g
if htmx.IsHTMX(r) {
  htmx.NewResponse().
  Location(RedirectRefererURL(r, s.config, "/")).
  Write(w)
} else {
  http.Redirect(w, r, RedirectRefererURL(r, s.config, "/"), http.StatusFound)
}
The only way this is possible is if the Referrer is /post and not / or /discover like it should be. Can we confirm/deny this if you inspect the POST /post requests? 🤔

See this code

g
\t\tif htmx.IsHTMX(r) {
\t\t\thtmx.NewResponse().
\t\t\t\tLocation(RedirectRefererURL(r, s.config, "/")).
\t\t\t\tWrite(w)
\t\t} else {
\t\t\thttp.Redirect(w, r, RedirectRefererURL(r, s.config, "/"), http.StatusFound)
\t\t}
The only way this is possible is if the Referrer is /post and not / or /discover like it should be. Can we confirm/deny this if you inspect the POST /post requests? 🤔

See this code

g
if htmx.IsHTMX(r) {
  htmx.NewResponse().
  Location(RedirectRefererURL(r, s.config, "/")).
  Write(w)
} else {
  http.Redirect(w, r, RedirectRefererURL(r, s.config, "/"), http.StatusFound)
}
There is only one explanation for this... Lemme see...
There is only one explanation for this... Lemme see...
Let's focus on this ☝️ how in da fuq is this happenning?!
Let's focus on this ☝️ how in da fuq is this happenning?!
@eldersnake LOL all good. Feel free to use https://gist.mills.io 👌
@eldersnake LOL all good. Feel free to use https://gist.mills.io 👌
@eldersnake Holy fuck?! That's unpossible?!
@eldersnake Holy fuck?! That's unpossible?!
@prologic Yeah realised I hit the character limit lol
Oh you deleted that last Twt 🤣
Oh you deleted that last Twt 🤣
@eldersnake I don't see the subsequent GET /, You are right that a POST /post does an internal XHR redirect to whatever view you're on to refresh it.
@eldersnake I don't see the subsequent GET /, You are right that a POST /post does an internal XHR redirect to whatever view you're on to refresh it.
Hmm...


>Jun 19 23:31:38 yarn_init.sh[61567]: [yarnd] 2024/06/19 23:31:38 (127.0.0.1:40254) "POST /post HTTP/
1.0" 200 0 3.402208ms
[...]
>
> Jun 19 23:31:39 yarn_init.sh[61567]: [yarnd] 2024/06/19 23:31:39 (127.0.0.1:40262) "GET /post HTTP/1.0" 404 729 123.474001ms
@prologic The login issue I cant yet narrow down as to when it happens as sometimes I login fine. But it gives off a 401 forbidden error. Anyway I've been focusing on the posting error as I figured it must be related. Registering and logging in as a new user works every time, which is weird.
@eldersnake That one was 👌
@eldersnake That one was 👌
@bender hmm you may be right...
But there should *never ever* be a GET /post -- There is no handler/route for that, that *will* indeed 404, really strange you're still seeing this? And what's this login issue?
But there should *never ever* be a GET /post -- There is no handler/route for that, that *will* indeed 404, really strange you're still seeing this? And what's this login issue?
Upgraded my Go compiler and I'm still not able to reproduce those 404(s) you're seeing. For clarity, this is what you expect to see in the server logs:


DEBU[0163] no subscriptions found for http://10.0.0.164:8000/user/admin/twtxt.txt
[yarnd] 2024/06/19 23:13:23 (10.0.0.164:64888) "POST /post HTTP/1.1" 200 0 8.323709ms
DEBU[0163] reloading templates in debug mode...
[yarnd] 2024/06/19 23:13:23 (10.0.0.164:64888) "GET / HTTP/1.1" 200 18120 87.933208ms


for every post there should be a POST /post ... followed pretty quickly by a GET /
Upgraded my Go compiler and I'm still not able to reproduce those 404(s) you're seeing. For clarity, this is what you expect to see in the server logs:


DEBU[0163] no subscriptions found for http://10.0.0.164:8000/user/admin/twtxt.txt
[yarnd] 2024/06/19 23:13:23 (10.0.0.164:64888) "POST /post HTTP/1.1" 200 0 8.323709ms
DEBU[0163] reloading templates in debug mode...
[yarnd] 2024/06/19 23:13:23 (10.0.0.164:64888) "GET / HTTP/1.1" 200 18120 87.933208ms


for every post there should be a POST /post ... followed pretty quickly by a GET /
FWOW go v1.22.2 here, but I'll upgrade now just to be sure...
FWIW go v1.22.2 here, but I'll upgrade now just to be sure...
FWIW go v1.22.2 here, but I'll upgrade now just to be sure...
@eldersnake So you're still having the exact same problems?! 🤔 And no I don't think the version of the compiler has much to do with this 😢
@eldersnake So you're still having the exact same problems?! 🤔 And no I don't think the version of the compiler has much to do with this 😢

* 8eef4d5d - (HEAD -> main, origin/main) Add DB restore capability and tools/backu-_db.sh script (19 seconds ago) <James Mills>


Oh well 😅 It works wonderfully!

In the event of a database corruption or loss:


$ URL=http://10.0.0.164:8000 ./tools/backup_db.sh > db.json
mv db.json data/db.restore.json
yarnd ...

* 8eef4d5d - (HEAD -> main, origin/main) Add DB restore capability and tools/backu-_db.sh script (19 seconds ago) <James Mills>


Oh well 😅 It works wonderfully!

In the event of a database corruption or loss:


$ URL=http://10.0.0.164:8000 ./tools/backup_db.sh > db.json
mv db.json data/db.restore.json
yarnd ...
@eldersnake are your replies creating new posts? I know this is a reply to a question from @prologic, but it is not within the same thread.
Unfortunately not on that front. Still the same 404 posting errors and oddly occasional login errors.

That's why I was wondering if using Go 1.22.4 could be an issue. I don't know how exactly. Only way to test is to rebuild it with an older version I guess, which is why I did the make clean in the first place. Old habits die hard lol.
@eldersnake Oh well, is the pod at least behaving normally now, just like mine? 😅
@eldersnake Oh well, is the pod at least behaving normally now, just like mine? 😅