Surprise 🤭
Well, it started speaking German with me... but only a few words here and there ✌️😅
Played around a bit and found this current sweet spot for me... no missing spaces or clipped words so far:
Yeah I had one instance of chinese output. It seems PDQ makes it more sensitive to ablation artifacts but I only noticed it on unusually formatted prompts. Going lower than scale 1.2 results in refusals.
How would you say it compares to v1? I haven't run an extensive comparison yet
Me neither... I'll give it a spin, once I'm fit again, feeling a bit under the weather 🤧
get well soon sir 🤞
Giving this a try. To note i use it at Q3K so it's a decent speed on my 8Gb video card.
So far heavily inconsistent when temperature is too high (0.85). In a short story i had it going, would say someone is facing away, then they are on their back, then they are tidying up the room but got up from a nap, etc. But lowering it (0.7) and it seems to play quite nice. I'll give this a few hours of testing on how it's output is for creative and RPing tasks.
If you can tolerate a bit slower speed I'd recommend IQ4_XS. I run this size 8GB for 24B models and it's much smarter than Q3K in my tests.
It's still likely to get knowledge and physics messed up more than v1 because PDQ sort of warps its perceptions slightly in order to boost creativity (theoretically). But lower temps should help too.