Why Cross-Lingual Search Matters for a Language Like Taita
Most search tools are built for a handful of high resource languages. If you speak Kidaw'ida (Taita), a Bantu language spoken in Kenya's Taita Hills, there is almost no way to search for content in your own language, even when that content already exists in a language you also speak, like English or Kiswahili.
Cross-lingual information retrieval, letting a query typed in one language find results written in another, is one way to close that gap. TaitaCLIR is my attempt at a small, working version of that for Taita.
1. The First Version: Built During the Princeton in Kenya Global Seminar
TaitaCLIR started in 2026 as a project for the Princeton in Kenya Global Seminar, GLS 347: Technology for African Languages in the Digital Age. I built it as part of a small team of Princeton students.
That first version is a semantic search demo over a small, hand built product catalog written in English, Kiswahili and Taita. A multilingual sentence transformer (paraphrase-multilingual-MiniLM-L12-v2) encodes queries and catalog items, FAISS indexes the embeddings for cosine similarity search, and a Streamlit interface shows results across all three languages.
It works, but it's a small prototype. The catalog is limited and we did not run a formal retrieval evaluation on it.
Repository: github.com/jomboi8/TaitaCLIR
2. Continuing the Work
After the seminar, I kept building on the idea, together with a Princeton faculty mentor and a fellow student researcher. What started as a Streamlit demo grew into a proper three part system: a Flutter mobile app, a Go API and a Python service handling retrieval, backed by Postgres.
This time we actually measured how well it works. On a labeled set of 36 queries, covering names, phrases, attribute style descriptions and typos, across English, Kiswahili and Taita, over a knowledge base of 57 concepts, the system reached a Recall@1 of 80.6 percent, a Recall@5 of 94.4 percent and a mean reciprocal rank of 0.86.
Exact name queries retrieve well in all three languages. Attribute style, descriptive queries, which is the harder and more central case this project is actually about, are weaker, especially in Taita.
3. What the Evaluation Actually Showed
We also compared embedding models directly rather than assuming one was better. LaBSE, the model we adopted, was tested against Cohere's multilingual embeddings on the same queries. The result was mixed rather than a clean win: Cohere showed a real advantage aligning Kiswahili and Taita, but underperformed LaBSE specifically on the attribute style queries that matter most here, so we kept LaBSE.
4. Where It's Headed
The honest read on the evaluation is that embeddings alone aren't handling the harder, descriptive queries well enough, particularly in Taita. Because of that, we're now moving toward a retrieval augmented generation (RAG) architecture rather than relying on embedding similarity alone. That work is ongoing.
This second version is still a private repository while we keep building it with collaborators, so there's no public link yet. We'll share it once it's in a state I'm comfortable putting in front of people.
5. What I Took From This
Working on TaitaCLIR taught me that most of low resource NLP work is data curation and honest evaluation, not just picking a model and hoping it generalizes.
