Fine-Tuning Qwen 3.5 for Text-to-SQL on a Mac

I fine-tuned Qwen 3.5 on a text-to-SQL task on my Mac with MLX. It took 15 minutes.

Qwen 3.5 Small dropped this week (0.8B, 2B, 4B, 9B). Ultra-efficient models where the 9B can compete with last year's 70B+.

To celebrate, I fine-tuned three of them on a text-to-SQL task. Here's what I found:

I took the 0.8B, 2B and 4B, added Mistral-Nemo 12B as a baseline, and ran the exact same experiment on all four: teach them to generate SQL from natural language. Same dataset (5,000 examples from gretelai/synthetic_text_to_sql), same settings, same Mac (64 GB), no cloud GPU.

Starting point: the 0.8B can't write SQL at all. I ask "Who earns more than 100k in Engineering?" and it answers "There is no salary column." The column is right there in the schema. 0% valid SQL. It just doesn't know.

Adding "Only answer in SQL" to the prompt? 1.5%. The model reads the data, computes the answer in its head, and outputs "1" or "400". Prompt engineering doesn't work here.

So I used LoRA fine-tuning. The idea is simple: instead of retraining all 752M parameters (expensive), you freeze the entire model and only train tiny adapter matrices. For the 0.8B that's 3.6M parameters, 0.48% of the total. Combined with 4-bit quantization, the whole thing fits in under 4 GB of RAM.

600 iterations. ~15 min per model. Results:

➡️ 0.8B: 0% → 86.5% valid SQL ➡️ 2B: 3.5% → 95.0% ➡️ 4B: 8.5% → 71.0% ➡️ 12B: 26.5% → 57.0%

The 2B won and the 12B finished last.

That surprised me, so I looked closer at the errors.

The 12B Mistral doesn't fail because it's bad at SQL. It fails because it's too good at math. When it sees data in the schema, it computes the answer mentally and outputs "42" instead of writing SELECT COUNT(*) FROM...

81% of its errors were just numbers. Correct answers, wrong format.

It's a competition between two forces: LoRA says "write SQL" while the model's instruct training says "answer helpfully." At 12B the instruct training wins, at 2B LoRA wins. That's the whole story.

The other thing I didn't expect: when measuring semantic accuracy, all 4 sizes landed around ~54%. The bottleneck isn't capability, it's following the format. This probably applies to any structured output task with LoRA on instruction-tuned models.

📌 A few things I'll remember:

Val loss is misleading: the 4B had the best validation loss (0.596) but finished 3rd in accuracy. Always evaluate on the real task, not the training curve. The 2B fine-tuned on my Mac in 8 min beats GPT-4o zero-shot on the same benchmark (50% vs 45%).

🍎 Total cost: $0, everything runs on Apple Silicon with MLX.

🔧 What I'd try next: more training data (used 5K out of 100K available), chat template formatting for larger models, and execution-based evaluation by running the generated SQL against a real database instead of string matching.