qwen3-4b-unslop-good-lora-v1

A larger pilot fine-tune of Qwen3-4B for unslop rewriting: taking AI-sounding passages and attempting to rewrite them into cleaner, more natural prose while preserving meaning.

This run sits between the weaker 1.7B pilot and the later 30B replication-style attempt.

How it was trained

• Base model: unsloth/Qwen3-4B-unsloth-bnb-4bit
• Training path: Unsloth fine-tuning on Hugging Face Jobs
• Dataset: N8Programs/unslop-good
• Rows used: 1000 (full training split)
• Objective: conversational rewrite / style cleanup

Intended use

Use this model as a pipeline stage for:

• rewriting AI-sounding prose into more natural text
• reducing cliché-heavy or overblown style
• testing whether moderate scale helps the unslop task before moving to much larger models

Limitations

• pilot-sized dataset
• still prone to tonal drift and over-dramatization
• still may paraphrase too freely instead of doing high-fidelity polish
• should be reviewed by a human or used as one stage in a larger editing pipeline

Example usage

from transformers import AutoTokenizer, AutoModelForCausalLM

repo = "Oysiyl/qwen3-4b-unslop-good-lora-v1"
messages = [
    {"role": "user", "content": "Polish this AI passage to feel more human while preserving meaning:\n[TEXT HERE]"}
]

tokenizer = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, device_map="auto")
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=400, temperature=0.8, repetition_penalty=1.1, do_sample=True)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Training loss vs training progress

Training loss is shown on normalized training progress (0–100%) for optimization visibility only. It helps show whether the run trained smoothly, but it is not by itself evidence of generalization or overfitting. Held-out rewrite fidelity remains the real decision metric.

Recommended evaluation sample

Use this full fiction passage for held-out testing:

The wind rose all at once, as if the mountain itself had inhaled. She stood at the edge of the pass, cloak snapping like a torn banner behind her, staring down at the valley where the last lights were going out one by one. Each flame extinguished felt deliberate—an answer to a question she had been too afraid to ask. Somewhere below, bells began to ring, not in warning, but in mourning. They already knew. “Don’t,” he said, breathless, reaching for her arm. His fingers brushed fabric, then air. “If you cross that line, there’s no coming back.” She didn’t turn. The cold burned her lungs, sharpened her thoughts. “There was never a way back,” she replied softly. “Only a way forward you were willing to pretend didn’t exist.” Thunder rolled, low and ancient. For a moment, the sky fractured into white fire, and in that instant he saw her face—resolved, terrified, incandescent with purpose. Not the girl he had sworn to protect, but the woman the world had demanded she become. “You’ll die,” he said. She smiled then, finally facing him, eyes bright with unshed tears. “So will everyone,” she answered. “The difference is what we leave burning when we’re gone.” She stepped forward. The mountain exhaled.

Full observed output on that sample

A gust rose suddenly from the mountainside. At the mouth of the gorge, she stopped, the folds of her robe fluttering violently against the wind; looking down toward the valley where the last embers of light flickered out, one after another. Each fading glimmer suggested an answer to a question she feared to voice. At the bottom, bells began to sound, not for warnings, but for sorrowful farewells. They already knew. "Don't." He reached for her hand. But his fingers only encountered fabric before vanishing into space. "If you cross that line, it won't be easy to return." She did not turn. The cold gnawed hungrily at her lungs, forcing sharp clarity into her mind. “There was never any chance of returning,” she murmured quietly. “You simply refused to see that possibility.” The thunder rumbled low and distant. Then came the flash of lightning across the sky, followed almost immediately by the crackle of fire. In that brief moment, he caught sight of her face—resolute, trembling, radiant with determination. Not the girl who had vowed to protect him, but the woman the world had compelled her to become. “You’ll die,” he whispered. “So will everyone else,” she replied, turning to meet his gaze. Tears glistened in her dark eyes, shining through the gloom. “The difference lies in what remains to burn when our time is done.” She took a step toward him. The mountain sighed slowly.

Judgment

This is clearly better than the 0.6B and 1.7B pilots, but it is still not as strong as the 30B run.

Why it is better:

• it preserves the scene and basic structure
• it does not collapse into a totally different story
• it reads like a rewrite rather than a random continuation

Why it still falls short:

• it over-dramatizes some phrasing
• it introduces wording changes that feel less natural or less precise than the source
• the ending drifts a bit (“She took a step toward him” / “The mountain sighed slowly”) instead of staying maximally faithful

Comparison vs pilot series

• 0.6B: failed badly; became a different story
• 1.7B: more fluent than 0.6B, but still invented scenes and structure
• 4B: first clearly improved text-only model in the series; mostly keeps the scene intact, but still drifts and over-shapes the prose
• 2B VL Instruct: despite the stronger instruct/VL backbone, still drifted hard and was not trustworthy
• 30B-A3B VL Instruct: first model in the series that looks plausibly faithful on held-out evaluation

So 4B is a real improvement, but not yet the breakthrough. The real jump appears at 30B scale.

Conclusion

This 4B pilot is a meaningful step up from the 0.6B and 1.7B models and is worth keeping as an intermediate result. But it still does not fully solve the fidelity problem. In this pilot family, it looks like moderate scale helps, but large-scale models are where the behavior starts to become genuinely convincing.