submitted by hauhau901
Hey everyone, made an uncensored version of Qwen3.5-4B - one of the brand new small models Qwen dropped these days.
Quick specs: 4B dense params, 32 layers, hybrid Gated DeltaNet linear attention + full softmax (3:1 ratio), 262K native context. Natively multimodal (text, image, video). This thing is surprisingly capable for its size .
This is the aggressive variant - 0/465 refusals during testing. Fully uncensored with zero capability loss.
The model will answer everything , though it sometimes adds a small disclaimer at the end of responses (seems to be baked into base training and is not a refusal).
Link:
Available quants: Q4_K_M (2.6 GB), Q6_K (3.3 GB), Q8_0 (4.2 GB), BF16 (7.9 GB)
Sampling settings from Qwen authors:
- Thinking mode: --temp 0.6 --top-p 0.95 --top-k 20
- Non-thinking: --temp 0.7 --top-p 0.8 --top-k 20
Note: This is a brand new architecture (released today). Make sure you're on a recent llama.cpp build. Works with llama.cpp, LM Studio, Jan, koboldcpp, etc.
Currently working on uncensored versions of Qwen3.5-9B, 27B, and 35B as well - will post those as they're ready.
All my releases:
As always, the goal is lossless uncensoring with no dataset changes and no capability loss.
MrMrsPotts
How have you determined there is no capability loss?
sine120
That's what the LLM wrote in the write-up
Fit-Produce420
To be fair, he asked the question in such a way that prompted that answer.
If you say "Evaluate your abilities, but they aren't very good, anyway" and it will usually agree it is incompetent.
"It" has no capacity to determine that either way, it just goes with the flow.
tonyunreal
Tried it, seems to perform better than the other decensored 4b variants.
Under certain scenarios (which I assume the original model is aligned to avoid answering), it answers poorly then quickly decend into chaotic loops, just like the other variants of small 3.5 models.
But when it works it answers better.
hauhau901 [ ]
Yeah, unfortunately that's the only caveat currently with these (smaller) models. They are obviously having nowhere near enough parameters to still complete requests they haven't been (almost) directly trained for. So sometimes although it's now fully uncensored, it can flop if it's something completely left-field for it.
Thanks for the feedback!
P.S. Once I finish the 9b/27b/35b, I expect those to be (inherently) a lot more consistent.
Fulminareverus
please let us know we 35b is done. A Q5 on 35B should be doable on a 5090, right?
hauhau901 [ ]
Wrapping up 9b now. Will move on to 27b and 35b. Hopefully up tomorrow!
PromiseMePls
I'm getting endless answers, chaotic loops and nonsensical responses with 9b lol
metigue
What's the KL divergence and PPL compared to the original?
nonerequired_
KL divergence might not be a good metric here
ArtfulGenie69
They use it on just about all the abliterated and heritic models. Luckily the model is posted and small so anyone could figure it out.
LoaderD
Not disagreeing with you, genuinely asking, why do you think that would be the case?
Major_Specific_23
Hello, thank you. It works great on my 4060ti. I just have one question, are the vision capabilities still intact with this gguf (I am using q8). Lmstudio doesnt allow me to upload images when i load your model. Thanks
hauhau901 [ ]
Hey, everything should be intact, make sure you put the mmproj files as well, like with lm studio :)
ZookeepergameNovel18
Do I just use the mmproj file from the original Qwen3.5 4B?
hauhau901 [ ]
Yes :) should be fine
hauhau901 [ ]
Hey , I've updated the repo now with the mmproj file :) Download it and put it next to the gguf
Major_Specific_23
thank you. it works now
[deleted]
[deleted]
hauhau901 [ ]
9b is up!
PromiseMePls
I tired to get it to work, but it just doesn't work lol
It goes kinda nuts when I try to speak to it.
Ok-Internal9317
I'm still waiting on huihui_ai
Fault23
Remindme! 5 days
RemindMeBot
I will be messaging you in 5 days on to remind you of
to send a PM to also be reminded and to reduce spam.
Parent commenter can
seymores
I am a noob -- how do you create uncensored model?
peva3
KGB prison cell.
ArtfulGenie69
Heritic or abliteration
ZootAllures9111
Heretic is the only thing I've ever seen actually work in the context of uncensored models that haven't been finetuned on additional data. All the ones labeled "abliterated" are useless I find.
ArtfulGenie69
Not my experience. huihui releases are great and I have had no issues even with ablits back when we had 70b llama models like deepseek r1 distilled. Also lots of other models. I don't usually use heretic because just about anyone does them, I trust that huihui on huggingface does a pretty good job usually. Also if you notice some of the kind authors of the ablits and heretics give you a kl distance from the original source model. That should help you tell how bad they are.
Fit-Produce420
There are many ways but essentially you might try to trigger many different refusal conditions on purpose, noting where the activations for refusal are located, and then remove them from the model.
You could also try post-training to undo refusals by training them positively on the negative data set.
You could also attempt to do this selectively, for instance allowing discussion of drugs but maintaining refusal for sex or murder.
ArtfulGenie69
You don't have to remove you just lessen the power of that layer that is activating when you receive the refusal.
catplusplusok
I made this one for consumer / unified memory Blackwell users, enjoy! . Anyone in the position to quantize 120B-A10B one? I might eventually, need to figure out runpod setup as I can't load it full locally.
Temporary-Mix8022
Can I ask - what do you use for these in terms of compute?
Do you run locally or use some kind of cloud based compute?
hauhau901 [ ]
Hello, I currently have 3 blackwell rtx 6000 cards as my workstation :)
I_Downvote_Cunts
God and I thought I was pretty well off with a single 5090.
teleprint-me
There's always a bigger fish.
Glittering-Call8746
U used the base 4b ? Or another aligned model based of it..
hauhau901 [ ]
Instruct model with reasoning turned off
Glittering-Call8746
Which hf repo is the instruct model ?
hauhau901 [ ]
The standard one :) scroll down on it to see how to make it instruct and not thinking
pet3121
Hey guys I am new here. So what's the deal with other people creating all this Qwen3.5 with different variations? If I want to try the original one like from Alibaba where can I find the author in hugginggace?
Feztopia
"So what's the deal" It's called open source. Or in this case it's open weight.
EpsilonAnura
Would you mind converting it to MLX or have the safetensors published as well?
Informal-Till-6227
I would also be interested in fine tuning it further. The HF checkpoint would be really great. Great work either way.
JinPing89
How can you guys let 4b models run on your android phone? I know google has this AI edge gallery app, which can run Gemma 3 4b very well, but it didn't support guff models, and the app is in beta that it doesn't even have a chat history.
Qwen3.5 4b on pockets would be a game changer.
bcell4u
Why do I get 20+ tp/s on this model vs ~11 on the non abliterated model of the same unsloth version?
isigeda
How to disable thinking mode in lm studio?
isigeda
oh found it.
{% set enable_thinking = false %}
OrneryMammoth2686
Nice work! That took no time at all :)
PS: which method did you use?
AbbreviationsOk6975
My ollama run: Error: 500 Internal Server Error: unable to load model: C:\Users\....\.ollama\models\blobs\sha256-3aa21dfc185595bb9f3f8f98f08325fcecfdf446c510ce2b03fe19104e700c16