The Best Embedding Model for STEM and Mathematics in 2026

The Unique Challenge of STEM and Mathematical Retrieval

Scientific and mathematical text is not like general language. It has characteristics that specifically challenge embedding models:

STEM Benchmark Performance

This is notable because OpenAI’s embedding model is heavily used by researchers and in academic AI tools. The performance gap in STEM suggests that research and scientific applications have been significantly underserved by that model, and that zembed-1 represents a major upgrade for these use cases.

Why zembed-1 Thrives in STEM Contexts

zELO Captures Scientific Relevance Nuance

In scientific retrieval, the difference between “highly relevant” and “related but not what I need” is often a matter of mathematical precision. A paper on gradient descent in neural networks is not the same as one on convergence proofs for gradient descent, even though they share substantial vocabulary.

zembed-1’s zELO training methodology — modeling relevance as continuous Elo scores from pairwise document competitions — trains the model to capture exactly this kind of nuanced relevance spectrum. Documents that are truly on-point score higher than ones that are adjacent. This precision matters enormously in scientific retrieval.

Distilled from a State-of-the-Art Reranker

zembed-1 is distilled from zerank-2, ZeroEntropy’s best-in-class reranker. Rerankers are models specifically trained to evaluate whether a document is truly relevant to a query — a task that requires deep semantic understanding. By distilling from this reranker rather than training from scratch on similarity signals, zembed-1 inherits a more sophisticated understanding of what it means for a scientific document to be relevant to a scientific query.

Long Context for Full Scientific Documents

Research papers, textbook chapters, and technical reports are long documents. zembed-1’s 32,768-token context window enables full-paper embeddings that preserve the argument structure, the mathematical derivations, and the cross-reference relationships that make a paper relevant to a given scientific query.

STEM AI Application Use Cases

Implementation for STEM Applications

from sentence_transformers import SentenceTransformer

model = SentenceTransformer(
    "zeroentropy/zembed-1",
    trust_remote_code=True,
    model_kwargs={"torch_dtype": "bfloat16"},
)

# Scientific literature search
query_embeddings = model.encode_query(
    "Convergence properties of stochastic gradient descent with momentum in non-convex optimization"
)

document_embeddings = model.encode_document([
    "We prove that SGD with momentum converges to a stationary point at rate O(1/√T) under standard assumptions on the gradient noise and step size schedule in non-convex settings...",
    "Gradient descent is one of the most widely used optimization algorithms in machine learning, used to minimize loss functions by iteratively updating parameters...",
    "We present a new analysis of Adam optimizer convergence under non-convex objectives, showing tight regret bounds that improve on previous work by a factor of log T...",
])

similarities = model.similarity(query_embeddings, document_embeddings)

What Researchers and Engineers Are Saying

“We run semantic search over 30 million research papers. Switching to zembed-1 improved our Recall@10 by 18 percentage points on our internal validation set. It wasn’t even a contest.” — CTO, academic research platform

import numpy as np
from sklearn.cluster import KMeans

# Embed your full paper corpus (e.g. PubMed/arXiv abstracts)
papers = load_paper_corpus()  # returns list of {"id", "title", "abstract"}
texts = [f"{p['title']}. {p['abstract']}" for p in papers]
paper_embeddings = model.encode_document(texts, batch_size=32, show_progress_bar=True)

# Retrieve top-50 relevant papers for a research question
research_question = (
    "Transformer architectures for long-context document understanding in low-resource settings"
)
q_emb = model.encode_query(research_question)
scores = model.similarity(q_emb, paper_embeddings)[0].numpy()
top50_idx = np.argsort(scores)[::-1][:50]

# Cluster the top-50 into thematic groups
top50_embeddings = paper_embeddings[top50_idx]
kmeans = KMeans(n_clusters=5, random_state=42).fit(top50_embeddings)

print("Thematic clusters in your literature review:")
for cluster_id in range(5):
    cluster_papers = [papers[top50_idx[i]] for i, c in enumerate(kmeans.labels_) if c == cluster_id]
    print(f"\nCluster {cluster_id + 1} ({len(cluster_papers)} papers):")
    for p in cluster_papers[:3]:
        print(f"  - {p['title']}")

Why Researchers Should Make the Switch

If you’re building AI tools for scientific or academic contexts, the benchmark gap between zembed-1 and the next best model — and especially the gap versus OpenAI’s widely-used embedding model — is a compelling reason to upgrade.

And because zembed-1 is open-weight and self-hostable, it’s accessible to academic institutions and research labs without the data privacy concerns that come with API-only services.

from zeroentropy import ZeroEntropy
zclient = ZeroEntropy()
response = zclient.models.embed(
model="zembed-1",
input_type="query", # "query" or "document"
input="What is retrieval augmented generation?", # string or list[str]
dimensions=2560, # optional: must be one of [2560, 1280, 640, 320, 160, 80, 40]
encoding_format="float", # "float" or "base64"
latency="fast", # "fast" or "slow"; omit for auto
)