Training a Small Multilingual LLM for Structured Order Extraction (English, German, Arabic) on Google Colab, for CPU Deployment

Abstract

Online stores receive free-text order requests in several languages ("I want a red T-shert, size m, to be delivered to mainstreet 34, 23456 Hamburg, I would like to pay in mastercard/paypal"). This article describes a complete, reproducible recipe for turning such messages into a fixed JSON object (brand, product, color, size, quantity, deliver_address, payment_method) with a model that is small enough to run on a 2–4 core CPU server. We fine-tune Qwen3-0.6B, a 0.6-billion-parameter open-weight model with Apache-2.0 licence and 100+ language coverage, using Low-Rank Adaptation (LoRA) on the free Google Colab T4 GPU. Because no labelled data exists for a specific store, we generate training data synthetically from a catalogue configuration: a structured order is sampled first, then rendered into English, German or Arabic text with language-specific grammar (German adjective declension, Arabic gender agreement, Arabic-Indic digits) and realistic noise (typos, casing, missing fields, chit-chat). Labels are therefore correct by construction. The model is trained with a completion-only loss on 6,000 examples, evaluated with field-level exact match on a hand-written 45-message gold set, merged, converted to GGUF and quantized for llama.cpp. The pipeline was validated end-to-end in a 2-vCPU sandbox; the GPU training run itself is executed by the reader on Colab, and the article explains how to read its results. Article 2 covers serving the exported model on CPU with grammar-constrained JSON decoding.

Table of Contents

1. Introduction
   1. The Problem
   2. Goals and constraints
   3. Contributions
2. Background and related work
3. Introduction

1. Introduction

1.1 The Problem

A customer writes a message in their own words and language; the shop system needs a machine-readable order. The classical approach — regular expressions and keyword lists per language — breaks on the first typo ("T-shert"), on word order ("zwei schwarze Hoodies von Nordwind in XL"), on grammatical agreement ("سترة حمراء" versus "تيشيرت أحمر") and on mixed scripts (an Arabic sentence containing a German address in Latin letters). A general-purpose large language model handles all of that, but calling a hosted API for every message costs money, sends personal data (a delivery address) to a third party, and is not needed for a task this narrow.

1.2 Goals and constraints

The requirements set for this study are:

1.3 Contributions

The article contributes:

2. Background and related work

Instruction fine-tuning. Fine-tuning a pre-trained language model on (instruction, answer) pairs makes it follow the instruction format reliably, including for unseen inputs of the same kind [1]. For a single narrow task, a few thousand high-quality examples are sufficient; Zhou et al. showed that 1,000 carefully curated examples can align a 65B model [2], and the task here is far narrower than open-ended chat.

Parameter-efficient fine-tuning. LoRA freezes the pre-trained weights and learns a low-rank update ΔW = BA for selected weight matrices, which cuts trainable parameters by orders of magnitude without a quality loss at this scale [3]. QLoRA extends the idea to 4-bit quantized base models so that very large models fit a single GPU [4]; for a 0.6B model this is unnecessary, because the full-precision weights already fit comfortably.

Synthetic training data. Self-Instruct demonstrated that instruction data can be generated rather than annotated [5]. We use a stricter variant: the label (a structured order) is sampled first and the text is generated from it, so labels are exact by construction and no model is needed to produce the data.

Constrained decoding. A fine-tuned model usually emits valid JSON, but "usually" is not a guarantee. Grammar-constrained decoding masks every token that would violate a formal grammar [6] and works even without fine-tuning for structured tasks [7]. llama.cpp converts a JSON Schema into such a grammar [8]. We combine both: fine-tuning gives correct content, the grammar gives correct structure.

Small multilingual models. The Qwen3 family covers 119 languages and ships a 0.6B dense model under Apache-2.0 [9][10]. Gemma 3 270M is explicitly positioned by Google for task-specific fine-tuning such as "unstructured to structured text processing" [11]. Qwen3.5 adds 0.8B–9B models with a hybrid Gated-DeltaNet/attention architecture and native vision input [12]. Section 4.1 compares them for this task.

Quantization. Running on CPU requires integer quantization of the weights. For Qwen3, 8-bit quantization is reported as near-lossless, while 4-bit shows noticeable degradation, and smaller models are less stable under quantization than larger ones [13]. This drives the Q8_0 recommendation in Section 4.6.

Alternative: token classification. An encoder such as XLM-R [14] fine-tuned for span tagging would be faster, but it extracts spans verbatim and cannot normalise "rot" to red or "Nachnahme" to cash_on_delivery without a second rule layer. A generative model does extraction and normalisation in one step, which is why it was chosen.

3. Task definition

3.1 Input and output

Input: one customer message (plain text, any of the three languages, mixed scripts allowed). Output: one JSON object with exactly the seven keys below, always in this order.

The worked example from the requirements therefore maps to:

{"brand": null, "product": "t-shirt", "color": "red", "size": "M", "quantity": 1,
 "deliver_address": "mainstreet 34, 23456 Hamburg", "payment_method": ["mastercard", "paypal"]}