Benchmark
How we measured search quality
The numbers on the home page come from one benchmark run on 2026-10-12. This page explains what was measured and how, so you can judge the numbers yourself.
Results
| Setup | nDCG@10 | No results |
|---|---|---|
| Store's built-in search | 0.291 | 52.9% |
| SeekGist Arabic layer | 0.837 | 0.9% |
| SeekGist + AI rewrite | 0.857 | 0.0% |
Full requests like “laptop for programming under 700”: 0.637 → 0.930 with the AI rewrite.
| Test set | Search quality (nDCG@10) | Searches with no results |
|---|---|---|
| The same 228 queries (search rules were tuned with these in view) | 0.905 | 0.0% |
| 387 new queries the tuning team never saw | 0.804 | 0.8% |
Catalog
A real electronics store catalog with 9,428 products, in English only (no Arabic names). It is private, so the store is not named and the data is not published.
Queries
228 real-style searches in 8 groups. 225 of them have at least one relevant product, and only those count. Group sizes:
| Group | Queries |
|---|---|
| Arabic | 67 |
| English | 62 |
| Arabic + English mixed | 10 |
| Typos | 16 |
| Arabizi | 10 |
| Model numbers | 18 |
| Specs (16gb, 144hz…) | 22 |
| Full requests | 20 |
Relevance labels
Every product got a grade for every query: 2 = fully matches, 1 = right kind but a limit is missing, 0 = not relevant. Grades come from rules on category, name, brand and specs, never from search results. The rules were checked by hand twice.
Metrics
- nDCG@10: how good the top 10 results are, using the grades. 1.0 is perfect.
- No-result rate: the share of queries that return nothing.
Setups
- Store's built-in search: a copy of the stock OpenCart 2.1 search (every word must appear in the name).
- SeekGist Arabic layer: Arabic normalization, light stemming, units and a dictionary of Arabic, Arabizi and English words, on Meilisearch 1.55.
- SeekGist + AI rewrite: the same, plus an AI model (Claude Haiku 5.5) that rewrites the query and finds price and spec limits. Used only when the shopper presses Enter.
Limits
One catalog, one store type (electronics), one run. Labels are rule-based, so they can miss some relevant products. The table “SeekGist today” shows the product itself, from its own test report.