defuss-lid tells you which language a text is written in, right where the text is. Pretrained fastText and Google CLD3 models, compressed into small language packs, run in your process or browser tab: no inference service, no API key, no text sent anywhere.
78 kB is the der_3 pack: German, English and Russian in 78.0 kB. Twenty languages fit in 110.4 kB, all 109 CLD3 labels in 130.4 kB.
der_3 (de, en, ru), plus the small fastText runtimeder_3 top-1 on the first 20 code points of FLORES sentencesfull), plus its WebAssembly runtimePick a model: this tab downloads it once from Hugging Face (a pinned revision) and runs it locally. Your text never leaves the browser.
Results update while you type. Short snippets are harder than full sentences.
Loading the runtime...
Type or paste some text to identify its language.
Scores are the models' native scores, not calibrated probabilities (fastText scores can slightly exceed 1). fastText models show their top 3; CLD3 returns only its top language with its own reliable flag. A pack chooses among its own languages and never answers "none of these": Ukrainian text in der_3 comes out as Russian.
Each pack is one file, shipped in the npm package as defuss-lid/packs/<name> and published on Hugging Face. What your users download is the pack plus its runtime: fastText needs only a small JavaScript runtime, while CLD3 also loads a WebAssembly core, so its runtime is much larger. Each card adds both up and compares the pack with the same languages on the other runtime.
der_3German, English and Russian. The smallest recommended pack, and the most accurate on short text in its set.
78.0 kB pack, 77.5 kB all-in gzipped
| Figure | der_3 | CLD3 alternative |
|---|---|---|
| All-in, gzipped | 77.5 kB (better) | 199.4 kB |
| Pack | 78.0 kB (better) | 106.6 kB |
| Runtime | 17.3 kB (better) | 173.6 kB |
| All-in | 95.4 kB (better) | 280.2 kB |
| FLORES sentences | 100.00% (better) | 99.97% |
| First 20 code points | 98.42% (better) | 94.99% |
| MASSIVE utterances | 98.54% (better) | 97.19% |
deferi_6Six major European languages: German, English, French, Spanish, Russian and Italian. A larger file than its CLD3 counterpart, yet more accurate, and with the smaller runtime smaller all-in.
127.5 kB pack, 119.8 kB all-in gzipped
| Figure | deferi_6 | CLD3 alternative |
|---|---|---|
| All-in, gzipped | 119.8 kB (better) | 199.9 kB |
| Pack | 127.5 kB | 107.3 kB (better) |
| Runtime | 17.3 kB (better) | 173.6 kB |
| All-in | 144.8 kB (better) | 280.9 kB |
| FLORES sentences | 99.97% (better) | 99.93% |
| First 20 code points | 93.74% (better) | 89.25% |
| MASSIVE utterances | 96.92% (better) | 94.84% |
major_20Twenty widely used languages across Latin, Cyrillic, Arabic, Devanagari, Bengali, Thai, Hangul and CJK scripts. A close call: fastText wins on FLORES and uncompressed size, CLD3 on MASSIVE and gzipped size.
110.4 kB pack, 202.6 kB all-in gzipped
| Figure | major_20 | fastText alternative |
|---|---|---|
| All-in, gzipped | 202.6 kB (better) | 214.1 kB |
| Pack | 110.4 kB (better) | 241.8 kB |
| Runtime | 173.6 kB | 17.3 kB (better) |
| All-in | 284.0 kB | 259.1 kB (better) |
| FLORES sentences | 99.65% | 99.70% (better) |
| First 20 code points | 91.39% | 92.82% (better) |
| MASSIVE utterances | 95.03% (better) | 91.94% |
fullEvery CLD3 output label, including romanized variants such as ru-Latn and ja-Latn: for open-ended input. Smaller and more accurate than the compressed full fastText model, even with its larger runtime.
130.4 kB pack, 218.7 kB all-in gzipped
| Figure | full | fastText alternative |
|---|---|---|
| All-in, gzipped | 218.7 kB (better) | 581.1 kB |
| Pack | 130.4 kB (better) | 708.1 kB |
| Runtime | 173.6 kB | 17.3 kB (better) |
| All-in | 304.0 kB (better) | 725.4 kB |
| FLORES sentences | 97.77% (better) | 93.14% |
| First 20 code points | 78.80% (better) | 73.61% |
| MASSIVE utterances | 88.60% (better) | 87.17% |
kB = 1,000 bytes. All-in counts each runtime once: a second pack on the same runtime adds only its own bytes. All-in, gzipped is what a page downloads from a server that compresses JavaScript. Accuracy is top-1 on held-out FLORES devtest sentences, their first 20 code points and MASSIVE test utterances. In each pair the better value is the darker one. The README size table lists every runtime file.
No model was trained from scratch. defuss-lid reuses the released fastText lid.176 and Google CLD3 weights, keeps what a set of languages needs, and quantizes the rest.
Pure JavaScript, no WebAssembly: typed arrays and arithmetic.
</s>. No case folding or Unicode normalization, exactly as the released model expects.< > word boundaries go through fastText's signed-byte FNV-1a hash into 2,000,000 buckets; a binary search finds the rows the pack kept.Google's original network and feature extraction, in WebAssembly.
.cld3q file holds the output columns of its languages, product-quantized embeddings and int8 dense layers.cld3-core.bin.gz is the CLD3 WebAssembly with its weights cut out (85.6 kB). The runtime checks the SHA-256 of the download and of the unpacked core before using it.A pack keeps only what its languages need. For fastText these are the hierarchical-softmax decisions on the paths to the chosen labels, so each kept language keeps its original score. For CLD3 they are the chosen softmax columns.
fastText packs reuse the released 8-group norm product quantizer unchanged and store classifier rows as int8. CLD3 packs product-quantize the three large embedding tables (4 groups × 256 centroids) and store dense layers as int8 with one scale per output neuron.
fastText subsets keep only the word and n-gram rows seen in 100 FLORES dev sentences per language. Accuracy is measured on separate FLORES devtest and MASSIVE test data; the pruning API rejects test and devtest records.
Pack sizes exclude the runtime. Every CLD3 pack shares one core, so a second CLD3 pack costs only its own bytes.
CLD3 packs compress storage and transfer only. They rebuild the original WebAssembly with its dense kernels, so inference memory does not shrink.
Chinese is the largest observed failure: the pruned 20-language fastText pack falls from 88.70%/85.24% to 60.49%/44.92% on MASSIVE zh-CN/zh-TW. The recommended major_20 is CLD3, which keeps its shared feature tables.
der_3 gets 100.00% of FLORES sentences right but 98.42% of their first 20 code points; for full the drop is 97.77% to 78.80%.
An allowlisted pack picks among its languages and does not reject others. To abstain, set a threshold you have measured on your own data.
From the v0.3 report, measured on 2026-11-02: FLORES devtest sentences, their first 20 code points, and MASSIVE test utterances. Each set is its own closed-set task, so numbers are comparable within a set, not across sets. kB = 1,000 bytes.
The four shipped packs, v0.3. All-in adds the runtime the pack needs. Scroll the table sideways for all columns.
| Pack | Languages | Method | Pack kB | All-in kB | FLORES sentences % | FLORES first 20 code points % | MASSIVE utterances % |
|---|---|---|---|---|---|---|---|
der_3 | 3 | fastText native PQ + int8 | 78.0 | 95.4 | 100.00 | 98.42 | 98.54 |
deferi_6 | 6 | fastText native PQ + int8 | 127.5 | 144.8 | 99.97 | 93.74 | 96.92 |
major_20 | 20 | CLD3 PQ + int8 | 110.4 | 284.0 | 99.65 | 91.39 | 95.03 |
full | 109 | CLD3 PQ + int8 | 130.4 | 304.0 | 97.77 | 78.80 | 88.60 |
Full means the full native label inventory, not full benchmark coverage: FLORES evaluates 78 common labels and MASSIVE 51. Small differences, such as 99.97 against 99.93 on FLORES sentences for deferi_6 and its CLD3 alternative, are not a robust quality advantage; the report gives paired bootstrap intervals. The report also lists the configurations that are not shipped, among them the unmodified fastText lid.176 model: 93.11% (FLORES) and 87.17% (MASSIVE) on the full set.
45 unit tests of src/ run on Node.js and on Bun; V8 measures the coverage on Node.js, and the build fails below 90 % of lines, functions and statements or 80 % of branches.
Every example on this page is executed by the project's end-to-end tests; the comment after each console.log is the exact output.
The package carries both runtimes and the four recommended packs. No runtime dependencies, no native addons.
npm install defuss-lidbun add defuss-lidNode.js 20.6 or newer resolves the node export condition, which reads the pack from the installed package. Run it with node detect.mjs.
import {load} from 'defuss-lid/packs/der_3';
const lid = await load();
console.log(lid.predict('Guten Morgen!')[0].language); // de
const top3 = lid.predict('Good morning, everyone!', {k: 3});
console.log(top3.map(p => p.language).join(',')); // en,de,ruBun uses the same disk loader as Node.js. Run it with bun detect.mjs.
import {load} from 'defuss-lid/packs/der_3';
const lid = await load();
console.log(lid.predict('Guten Morgen!')[0].language); // de
const top3 = lid.predict('Good morning, everyone!', {k: 3});
console.log(top3.map(p => p.language).join(',')); // en,de,ruIn the browser the same import resolves a fetch-based loader. It requests the pack from the package's models/ folder, relative to the module's URL, so your bundler or server has to serve models/ (and, for CLD3 packs, runtime/cld3-core.bin.gz) as assets. Serve the .gz file as plain bytes, without Content-Encoding: gzip: the runtime checks its hash and unpacks it itself.
import {load} from 'defuss-lid/packs/der_3';
const lid = await load();
console.log(lid.predict('Guten Morgen!')[0].language); // deNo install and no build: import the runtime from jsDelivr and load a model from the pinned Hugging Face revision.
<script type="module">
import {loadModel} from 'https://cdn.jsdelivr.net/npm/defuss-lid@0.3.0/dist/index.js';
const lid = await loadModel('https://huggingface.co/kyr0/defuss-lid-1.0/resolve/24d27e5b42d002d69fded6301607160919151370/der_3.dlid');
console.log(lid.predict('Guten Morgen!')[0].language); // de
</script>CLD3 packs run in WebAssembly and return only their top language, with CLD3's reliable flag. Call dispose() when you are done.
import {load} from 'defuss-lid/packs/full';
const lid = await load();
const [top] = lid.predict('Gallia est omnis divisa in partes tres.');
console.log(top.language); // la
console.log(top.reliable); // true
console.log(lid.predict('Guten Morgen!')[0].reliable); // false
lid.dispose();fastText models can rank only the languages your app supports. A restriction still picks one of them, so add a threshold to abstain.
import {load} from 'defuss-lid/packs/deferi_6';
const lid = await load();
const allowed = {languages: ['de', 'en', 'fr']};
console.log(lid.predict('Guten Tag!', allowed)[0].language); // de
console.log(lid.predict('Buongiorno a tutti!', allowed)[0].language); // de
console.log(lid.predict('Buongiorno a tutti!', {...allowed, threshold: 0.5}).length); // 0| Call | Returns |
|---|---|
load(options?)from defuss-lid/ | The pack, loaded: a fastText model or a CLD3 pack |
loadModel(url)from defuss-lid | A fastText LidModel from any .dlid URL |
model.predict(text, {k, threshold, languages})fastText | [{language, score}], best first; [] for blank text |
loadCld3Pack(url, {coreURL})from defuss-lid/ | {predict, dispose, meta} for any .cld3q URL |
pack.predict(text, {threshold})CLD3 | [{language, score, reliable}], top-1 only |
PACKSfrom defuss-lid/ | Names, languages, bytes and SHA-256 of the four packs |
Download the packs directly, or pin the exact revision this page uses: 24d27e5. The npm package ships byte-identical copies.