Laya is a non-autoregressive decision engine for text: one call returns one typed answer, a yes or no, a choice from a fixed list, or a score, with no tokens generated along the way. It is for developers and machine-learning engineers who need a verdict inside a request path (moderation, triage, routing, ranking, filling a field) and who currently get it by asking a chat model for a one-word reply. The repository holds the Python package; the checkpoints live on Hugging Face under the convaiinnovations account, and there is a hosted demo if you only want to see the shape of the output.
What it does
You hand Laya plain text and a decision type. The project supports three: a typed choice among options you supply, a score, and a yes/no. The answer comes back typed, so your code reads a field instead of parsing prose. According to the project, it reads and decides in over 100 languages, and a router picks the right checkpoint for each request, so a single entry point covers both the English-focused and the multilingual weights.
The headline numbers in the README are the authors' own, from their own setup and their own benchmark, not an independent measurement:
- one call in 33 ms, which is also what the project's demo shows
- a context of 8,192 tokens, with a 4,000-token document answered in about 1.7 seconds
- 0.362 on their benchmark for the base checkpoint used zero-shot
- 0.766 after fine-tuning on your own decisions
That last pair is the honest story of this repository: out of the box the base model is mediocre on the project's own benchmark, and the good figure is a fine-tuned one.
How it works
Laya is an encoder-style model rather than a generator. Nothing is sampled: the text goes through the network once and the decision is read out of that single forward pass, which is where the latency figures come from. The repository's topics name ModernBERT and PyTorch, so this is a modern encoder backbone with decision heads on top rather than a small chat model talked into behaving.
Training is the unusual part. The authors train with reinforcement learning against strictly proper scoring rules, a method they call RLCD. A proper scoring rule only pays off when the reported probability is honest, so the stated aim is calibration: a score you can threshold, not just a label you hope is right. The copy of the README saved here is cut off before it explains how the router chooses a checkpoint, so treat that piece as described by the project rather than documented in what we have.
Getting started
The code is Apache-2.0 and the weights are published openly, so there is no API key and no paid service in the loop: you pay only for the hardware you run it on, and nothing leaves your machine. Installation is a package manager away, with the package published on PyPI:
pip install laya
The saved README does not list hardware requirements, so read the quoted latencies as coming from a machine the project has not described here. PyTorch will use an NVIDIA GPU if you have one, and an 8,192-token encoder pass is far cheaper than generating text either way. If you want to try it before installing anything, the README links a Colab notebook, a Hugging Face Space called laya-demo, and documentation at nandhakishorm.github.io/laya. Fine-tuning, which is where the 0.766 figure comes from, needs labeled examples of your own decisions plus the compute to train on them. Everything starts from the repository itself at github.com/NandhaKishorM/laya.
When to use it / when not
The main catch is the gap between 0.362 and 0.766: the appealing number is not what you get on day one, and closing the gap is a labeling and training project, not a configuration change. The second thing to weigh is age. The first commits are from September 2026 and the most recent push is from October 2026, so this is a very young repository, popular fast but with little time behind its interface.
Use it instead of asking Claude or GPT for a one-word JSON answer when the decision runs at volume or inside a tight latency budget, when you want a calibrated score you can threshold, and when the inputs arrive in many languages. Don't use it if you need the reasoning behind a verdict, free-form output, or anything that cannot be reduced to a choice, a score or a yes/no: this model has no way to explain itself, because it never writes.
Take Laya seriously if you have a classification or scoring step that a general chat model is handling badly or expensively, and you have, or can collect, your own labeled decisions to fine-tune on: that is the case the repository is built for, and the case its best number reflects. If what you want is a drop-in zero-shot replacement with no training work attached, the project's own benchmark score is the argument for waiting and watching this one instead.