Thank you so much for the suggestion! I tried Q8 of the model you mentioned, and I am very impressed with the results! The output itself was exactly what I wanted, the speed was a little on the slower side. Loading my previous conversation with a context of over 15k tokens took about 10 minutes to get the first response, but the later messages were much faster. The web ui loses connection almost every time though, and I just manually copy the response from the terminal window in to the web ui to save it for future context. I am currently downloading the Q6 model, and might experiment with going even lower for faster speeds and more stability, if the quality of the output doesn't degrade too much.
Not a native English speaker, so I might be completely wrong, but doesn't the word 'queer' litteraly mean 'confused' / 'weird'? Not that it makes a difference in this context, I'm just wondering
I've encountered a problem, not sure if it's related to the update or not. On both Eternity and browser my Subscribed feed shows posts from All. From Eternity only upvoting posts fails with a 401 code, but I seem to be able to do it on the browser. Any one else encountered some like that?
Thank you so much for the suggestion! I tried Q8 of the model you mentioned, and I am very impressed with the results! The output itself was exactly what I wanted, the speed was a little on the slower side. Loading my previous conversation with a context of over 15k tokens took about 10 minutes to get the first response, but the later messages were much faster. The web ui loses connection almost every time though, and I just manually copy the response from the terminal window in to the web ui to save it for future context. I am currently downloading the Q6 model, and might experiment with going even lower for faster speeds and more stability, if the quality of the output doesn't degrade too much.