Differences between the two heretic versions of Cydonia 24b v4.3
Hi @coder3101
What are the differences between the two heretic versions you made of Cydonia 24b v4.3 ?
Thank You
v1 is has almost zero refusal but at the cost of some divergence to original model.
v2 has slightly higher (still better than original) refusal but almost no divergence from original model.
If you feel, model quality is off the original model, use v2. If you need absolute best refusal, always use v1.
In model card you will find both refusal and KL divergence.
Where gguf? :)
Besides modelcard, under fine tune section:
These are both really good, but I think v1 is better.
Would you have the time to make a v3 that is like v1, but diverges even more from the original model in the direction it's already diverging ?
At that point you could probably even give it a different name, because it won't really be a Cydonia anymore.
I will try that with a new heretic version!
At that point you could probably even give it a different name, because it won't really be a Cydonia anymore.
Really, @SzilviaB ?
Would you have the time to make a v3 that is like v1, but diverges even more from the original model in the direction it's already diverging?
I've tried Cydonia Heretic v1 before and it's still Cydonia - just with a less positive, more willing tone. (And more mistakes because of the higher KL Divergence. That's the whole point of Heretic: minimize brain damage by keeping an eye on divergence.)
Abliteration is more of chipping away the bad parts rather than remolding the entire model.
I agree, I am not fine tuning the model like you do, so it likely won’t be a new name. If and when bi-projected norm preserving abliteration is implemented in heretic, I might make a new v3. This new method likely is more stable and somehow makes the model better.
Still, it will be just same Cydonia. I am not willing to give it a new name.
Really, @SzilviaB ?
@TheDrummer people would know it's coming from Cydonia, first and foremost from the description, so it's not like it would be hidden, and people do rename models on HF all the time.
Cydonia itself is coming from a Mistral.
It was a suggestion.
I am surprised that this is an issue, but it was not to take away from your work.
Cydonia, Magidonia and other models that you made, are probably some of the best ever.