2026
SciEmbed
Embedding model for scientific documents, trained with the sentences in which papers cite each other as contrastive signal. ModernBERT-base with up to 8K context; six checkpoints released under MIT.
Up to ~30M training pairs on 4Ă—H200, SciRepEval 66.85 (EMNLP 2026)
PyTorch, ModernBERT, Contrastive learning, DuckDB