Slava Dubrov Edge of Context
Search

NER model selection: spaCy, GLiNER, Transformers, or LLMs?

Viacheslav Dubrov

· Updated · 5 min · Retrieval and Language Systems

Named entity recognition (NER) finds and labels spans of text, such as people, companies, and product codes. This page is for engineers choosing an extraction path for a production workload. It gives you a starting point that you can test against your schema, documents, and operating constraints.

Use spaCy for stable labels and fast pipelines. Choose GLiNER when labels change often. Test a bi-encoder when many labels can reuse cached embeddings, or a fine-tuned Transformer token classifier when you have labeled examples. Use LLM-based structured extraction when you need a complex record rather than a simple list of spans.

Choose by the required labels, language, output shape, labeled data, and latency budget.

Decision table

NeedStarting pointWhy
Known labels and CPU deploymentspaCy baselinePretrained pipelines and rules provide a baseline; check language and domain accuracy.
New labels without full trainingGLiNERLabel-conditioned extraction works well when the label set changes. Start with clear type descriptions, not bare names.
Large, reusable label inventoryGLiNER bi-encoderIt can pre-compute and cache label embeddings instead of encoding labels with every document.
Entities plus relations in one passGLiNER-Relex candidateIt jointly extracts entity and relation types. Benchmark it against separate stages on the target relation schema.
Entities plus document fields or classificationGLiNER2Its schema interface combines entity recognition, classification, and structured extraction tasks.
Nested or overlapping entitiesNested-capable decoder, including GLiNER with flat_ner=FalseFlat decoding suppresses overlaps. Test nested spans on annotations that contain the overlap patterns you expect.
Multilingual or multi-script extractionGLiNER-X or XLM-R baselineTest each target language, domain, and the wording of its type descriptions.
Stable domain labels with representative labeled spansFine-tuned token classifier or spaCy pipelineEstablish a supervised baseline, then compare exact-span F1, latency, and cost on your domain set.
Implicit, cross-sentence, or structured fieldsLLM or hybrid candidateStructured outputs constrain response shape, not extraction correctness. Validate field accuracy on a domain test set.
High-stakes PII or regulated workflowPresidio plus GLiNER2-PII candidateCombine model output with deterministic checks, redaction controls, and review.
Exact IDs, product codes, or known termsRules and dictionariesDeterministic matching is useful for fixed patterns; test missed variants and false matches.

How to choose

Write the entity types and a few labeled examples before choosing a model. Include ambiguous terms and spans inside other spans if the task needs them.

If labels are stable, test an existing pipeline first. Fine-tune a token classifier when its domain errors justify training and representative labeled examples are available. Compare it on a held-out domain set. It predicts token labels without generating the answer token by token.

If labels change often, test GLiNER or an LLM extractor. Write a short description for each ambiguous type. ZeroNER evaluates type descriptions with entity types unseen during training. Descriptions clarify the label; a test of unseen types must also keep those types out of training.

If you reuse a large label inventory, evaluate a bi-encoder before a joint label-text encoder. The GLiNER bi-encoder paper reports about 130 times higher throughput for gliner-bi-edge-v2.0 with cached label embeddings than gliner_small-v2.5 at 1,024 labels. That comparison used one H100 GPU and batch size 1; the speed tests covered 64-, 256-, and 512-token inputs. It shows why label reuse is worth testing, not the speedup to expect on another workload.

If documents span several languages, benchmark every target language and script. OpenNER 1.0 found no single model best across its 52-language collection. Test both English and localized type descriptions because the label text is part of the model input.

If the output is a structured record rather than spans, use an LLM with structured outputs or a hybrid pipeline. LangExtract is useful when each extraction must map back to its source location. NuExtract is a self-hosted option for multimodal document extraction. Both still need field-level evaluation on the target documents.

Many business extraction tasks are not pure NER. “Find the parties, obligations, effective date, termination clause, and governing law” is document understanding with entity fields.

For PII, use a PII framework such as Presidio as the integration and redaction layer. Treat GLiNER2-PII as a model candidate, not a compliance decision. Measure recall on the sensitive formats, languages, and documents that the system will process.

Production pipeline

A production extraction pipeline often adds stages around the model:

  1. Candidate extraction: spaCy, GLiNER, Transformer model, LLM, rules, or a combination.
  2. Optional normalization or entity linking: map spans to canonical IDs, product codes, users, companies, or entries in your domain vocabulary.
  3. Validation and review: reject impossible labels, enforce schema constraints, deduplicate spans, and route uncertain cases to review.

Normalization maps extracted text to a stable ID in your data. For the span “Apple”, first decide whether it means the company, the fruit, a product family, or a stock ticker. Then map that meaning to the corresponding ID.

Evaluation checklist

Measure more than entity-level F1:

Exact-span F1 counts a prediction as correct only when both the span and label match the annotation. A relaxed score may accept overlapping spans, so record its matching rule. Neither score checks whether the extracted name maps to the correct database ID.

  • exact span F1
  • relaxed span F1
  • label confusion matrix
  • nested entity handling
  • entity normalization accuracy
  • type-description and label-language sensitivity
  • language, script, and document-format slices
  • false positives by label
  • confidence calibration
  • latency and cost per document
  • human correction rate

Keep the test set separate from schema design and threshold tuning. Report the number of documents and entity mentions per label. Use document-level confidence intervals when repeated mentions in one document would otherwise inflate certainty.

Deeper reading

References