lexer.go and newparser.go resemble the parser combinators: https://git.isobeef.org/lyse/tt2/-/commit/4d481acad0213771fe5804917576388f51c340c0 It's far from finished yet.
The first attempt in parser.go doesn't work as my backtracking is not accounted for, I noticed only later, that I have to do that. With twt message texts there is no real error in parsing. Just regular text as a "fallback". So it works a bit differently than parsing a real language. No error reporting required, except maybe for debugging. My goal was to port my Python code as closely as possible. But then the runes in the string gave me a bit of a headache, so I thought I just build myself a nice reader abstraction. When I noticed the missing backtracking, I then decided to give parser combinators a try instead of improving on my look ahead reader. It only later occurred to me, that I could have just used a rune slice instead of a string. With that, porting the Python code should have been straightforward.
Yeah, all this doesn't probably make sense, unless you look at the code. And even then, you have to learn the ropes a bit. Sorry for the noise. :-)
⨁ Follow button on their profile page or use the Follow form and enter a Twtxt URL. You may also find other feeds of interest via Feeds. Welcome! 🤗
⨁ Follow button on their profile page or use the Follow form and enter a Twtxt URL. You may also find other feeds of interest via Feeds. Welcome! 🤗
This guy put a prompt into ChatGPT and recorded its output. Then he shuffled the words in the prompt text in such a way that the prompt was completely unintelligible to a human being, but still retained some of the correlational statistics of the words. He fed that as a prompt into ChatGPT. The result? Output that was almost identical to the original output.
Gibberish input into ChatGPT will produce coherent-sounding text just as well as a carefully-crafted prompt. This is such a nice and simple demonstration that ChatGPT has no "intelligence" of any kind built into it.
$ speedtest
Speedtest by Ookla
[error] Error: [8] nodename nor servname provided, or not known
Server: Foxtel Broadband - Brisbane (id: 8847)
ISP: Aussie Broadband
Idle Latency: 9.46 ms (jitter: 0.20ms, low: 9.25ms, high: 9.76ms)
Download: 93.39 Mbps (data used: 50.5 MB)
22.67 ms (jitter: 4.76ms, low: 10.01ms, high: 135.16ms)
Upload: 35.10 Mbps (data used: 58.7 MB)
880.05 ms (jitter: 95.73ms, low: 16.75ms, high: 1823.19ms)
Packet Loss: 0.0%
$ speedtest
Speedtest by Ookla
[error] Error: [8] nodename nor servname provided, or not known
Server: Foxtel Broadband - Brisbane (id: 8847)
ISP: Aussie Broadband
Idle Latency: 9.46 ms (jitter: 0.20ms, low: 9.25ms, high: 9.76ms)
Download: 93.39 Mbps (data used: 50.5 MB)
22.67 ms (jitter: 4.76ms, low: 10.01ms, high: 135.16ms)
Upload: 35.10 Mbps (data used: 58.7 MB)
880.05 ms (jitter: 95.73ms, low: 16.75ms, high: 1823.19ms)
Packet Loss: 0.0%
$ speedtest
Speedtest by Ookla
[error] Error: [8] nodename nor servname provided, or not known
Server: Foxtel Broadband - Brisbane (id: 8847)
ISP: Aussie Broadband
Idle Latency: 9.46 ms (jitter: 0.20ms, low: 9.25ms, high: 9.76ms)
Download: 93.39 Mbps (data used: 50.5 MB)
22.67 ms (jitter: 4.76ms, low: 10.01ms, high: 135.16ms)
Upload: 35.10 Mbps (data used: 58.7 MB)
880.05 ms (jitter: 95.73ms, low: 16.75ms, high: 1823.19ms)
Packet Loss: 0.0%
Trees on a meadow with the sun in their backs
If you hit 'reply' on a post it fills in the mentions as well in the status entry box.
#running
#running
#running
#running
Sasly I didn't come across RNNS though 😆 But yhay doesn't matter 🤔
Sasly I didn't come across RNNS though 😆 But yhay doesn't matter 🤔
Sasly I didn't come across RNNS though 😆 But yhay doesn't matter 🤔
Like, if you type "The dog is", autocomplete will suggest some words from you that are likely to come next. Maybe "barking", "wet", "hungry", ... It'll rank those by how high a probability it rates each follow-up word. It'll probably not suggest words like "uranium" or "quickly", because you very rarely if ever encounter those words after "The dog is" in English sentences so their probability is very low.
👆 That's the "autoregressive" part.
It gets these probabilities from a "language model", which is a fancy way of saying a table of probabilities. A literal lookup table of the probabilities would be wayyyyy too big to be practical, so neural networks are often used as a representation of the lookup table, and deep learning (many-layered neural networks + a learning algorithm) is the hotness lately so they use that.
👆 That's the "language model" part.
So, you enter a prompt for ChatGPT. It runs fancy autocorrect to pick a word that should come next. It runs fancy autocorrect again to see what word will come next *after the last word it predicted and some of your prompt words*. Repeat to generate as many words as needed. There's probably a heuristic or a special "END OF CHAT" token to indicate when it should stop generating and send its response to you. Uppercase and lowercase versions of the tokens are in there so it can generate those. Punctuation is in there so it can generate that. With a good enough "language model", it'll do nice stuff like close parens and quotes, correctly start a sentence with a capital letter, add paragraph breaks, and so on.
There's really not much more to it than that, aside from a crapton of engineering to make all that work at the scale they're doing it.