Training a Small Multilingual LLM for Structured Order Extraction (English, German, Arabic) on Google Colab, for CPU Deployment

Abstract

Online stores receive free-text order requests in several languages ("I want a red T-shert, size m, to be delivered to mainstreet 34, 23456 Hamburg, I would like to pay in mastercard/paypal"). This article describes a complete, reproducible recipe for turning such messages into a fixed JSON object (brand, product, color, size, quantity, deliver_address, payment_method) with a model that is small enough to run on a 2–4 core CPU server. We fine-tune Qwen3-0.6B, a 0.6-billion-parameter open-weight model with Apache-2.0 licence and 100+ language coverage, using Low-Rank Adaptation (LoRA) on the free Google Colab T4 GPU. Because no labelled data exists for a specific store, we generate training data synthetically from a catalogue configuration: a structured order is sampled first, then rendered into English, German or Arabic text with language-specific grammar (German adjective declension, Arabic gender agreement, Arabic-Indic digits) and realistic noise (typos, casing, missing fields, chit-chat). Labels are therefore correct by construction. The model is trained with a completion-only loss on 6,000 examples, evaluated with field-level exact match on a hand-written 45-message gold set, merged, converted to GGUF and quantized for llama.cpp. The pipeline was validated end-to-end in a 2-vCPU sandbox; the GPU training run itself is executed by the reader on Colab, and the article explains how to read its results. Article 2 covers serving the exported model on CPU with grammar-constrained JSON decoding.

Keywords: instruction fine-tuning, LoRA, synthetic data, information extraction, multilingual NLP, Qwen3, GGUF, llama.cpp, Google Colab

1. Introduction

1.1 The problem

A customer writes a message in their own words and language; the shop system needs a machine-readable order. The classical approach — regular expressions and keyword lists per language — breaks on the first typo ("T-shert"), on word order ("zwei schwarze Hoodies von Nordwind in XL"), on grammatical agreement ("سترة حمراء" versus "تيشيرت أحمر") and on mixed scripts (an Arabic sentence containing a German address in Latin letters).

1.2 Goals and constraints

The requirements set for this study are:

1. Three input languages: English, German, Arabic (including dialectal phrasing such as "أبغى"
2. One fixed output schema: Exactly seven keys, returned strictly formatted as valid JSON.
3. Store-specific behaviour: Product, colour and payment names normalised to the store's catalogue vocabulary.
4. Small hardware: CPU-only server with 2–4 cores & 4–8 GB RAM; training fits in free Google Colab.
5. A learning objective: The recipe teaches how to fine-tune and ship a model for a specific domain, justifying every design choice.

1.3 Contributions

The article contributes:

• (i) Schema design with explicit normalisation rules.
• (ii) Configuration-driven synthetic-data generator that produces grammatically plausible messages in three languages with correct labels.
• (iii) LoRA fine-tuning recipe for Qwen3-0.6B running on a T4 GPU in well under an hour.
• (iv) Evaluation protocol measuring what is actually shipped (quantized GGUF on CPU).
• (v) Tested notebook and script set validated end-to-end.

2. Background and Related Work

Instruction fine-tuning

Fine-tuning a pre-trained language model on (instruction, answer) pairs makes it follow the instruction format reliably, including for unseen inputs of the same kind [1]. For a single narrow task, a few thousand high-quality examples are sufficient; Zhou et al. showed that 1,000 carefully curated examples can align a 65B model [2], and the task here is far narrower than open-ended chat.

Parameter-efficient fine-tuning

LoRA freezes pre-trained weights and learns a low-rank update ΔW = BA for selected weight matrices, cutting trainable parameters by orders of magnitude without quality loss [3]. QLoRA extends this to 4-bit quantized base models [4]; for a 0.6B model this is unnecessary as full-precision weights fit comfortably.

Synthetic training data

Self-Instruct demonstrated instruction data can be generated rather than annotated [5]. We use a stricter variant: the label (structured order) is sampled first and text is generated from it, so labels are exact by construction with zero annotator error.

Constrained decoding

A fine-tuned model usually emits valid JSON, but "usually" is not a guarantee. Grammar-constrained decoding masks every token violating a formal grammar [6] and works even without fine-tuning [7]. llama.cpp converts a JSON Schema into such a grammar [8]. We combine both: fine-tuning gives correct content, the grammar gives correct structure.

Small multilingual models

Qwen3 family covers 119 languages with an Apache-2.0 0.6B dense model [9][10]. Gemma 3 270M is positioned by Google for task-specific fine-tuning [11]. Qwen3.5 adds 0.8B–9B hybrid Gated-DeltaNet/attention models [12]. Section 4.1 compares them for this task.

Quantization

Running on CPU requires integer quantization. For Qwen3, 8-bit quantization is reported as near-lossless, while 4-bit shows noticeable degradation, and smaller models are less stable under quantization than larger ones [13]. This drives the Q8_0 recommendation in Section 4.6.

Alternative: token classification

An encoder such as XLM-R [14] fine-tuned for span tagging would be faster, but extracts spans verbatim and cannot normalise "rot" to red or "Nachnahme" to cash_on_delivery without a second rule layer. A generative model does extraction and normalisation in one step.

3. Task Definition

3.1 Input and output

Input: one customer message (plain text, any of the three languages, mixed scripts allowed).
Output: one JSON object with exactly the seven keys below, always in this order.

{"brand": null, "product": "t-shirt", "color": "red", "size": "M", "quantity": 1,
 "deliver_address": "mainstreet 34, 23456 Hamburg", "payment_method": ["mastercard", "paypal"]}

3.2 Design decisions