Skip to content
OmniJevPublic

About

๐Ÿš€๐Ÿš€ A multimodal System One decision model that gives calibrated answers to typed questions about screens, photos, video and text in one forward pass.

Topics

Resources

Stars

191 stars

Watchers

0 watching

Forks

Latest commit

ย 

History

69 Commits

Folders and files

NameName
Last commit message
Last commit date
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 

Repository files navigation

OneJev, a Multimodal System One Decision Model

English ยท ็ฎ€ไฝ“ไธญๆ–‡ ยท ๆ—ฅๆœฌ่ชž

Hugging Face ModelScope Demo GitHub Website

Data License Python PyTorch Transformers

OneJev is a multimodal System One decision model based on Qwen. It returns calibrated probabilities for typed questions about screenshots, photos, videos and text in a single forward pass. Available in 0.8B, 4B, 9B and 27B.

๐Ÿ”ฅ News

Decision Index (public part)

Block column chart of public Decision Index 0.3 scores, OneJev 1.6 against OneJev 1.5: 0.8B 48.82 vs 42.71 (+6.1), 4B 62.01 vs 54.89 (+7.1), 9B 65.31 vs 59.03 (+6.3), 27B 69.49 vs 63.64 (+5.8)

Four leaderboards of the public Decision Index 0.3 by model size, top ten each, OneJev1.6 highlighted: OneJev1.6-0.8B first of 51 models up to 2.5B parameters, OneJev1.6-4B first of 38 models of 3B to 6B, OneJev1.6-9B first of 26 models of 6B to 12B, OneJev1.6-27B second of 59 models of 25B to 36B

Other benchmarks

Block bar charts: the four OneJev1.6 sizes against Jev 1.13, Jev-Omni 12B and Qwen3.8-27B thinking on the OneJev test set, DecisionBench hard, TypeSafe and MMStar

Accuracy (%). The OneJev test set is our own held-out set; the others are public.

Speed

Latency on one H200 for one 1280x720 screenshot: 0.8B 31 ms for 1 question, 51 ms for 10, 5.1 ms per question; 4B 64 ms for 1 question, 104 ms for 10, 10.4 ms per question; 9B 81 ms for 1 question, 131 ms for 10, 13.1 ms per question; 27B 189 ms for 1 question, 324 ms for 10, 32.4 ms per question

Quick start

Choose one backend, then run the Python example below.

Option A: Flide

For NVIDIA GPUs. Supports text, images and video. Flide is our inference engine for decision models.

uv venv -p 3.12 && source .venv/bin/activate
uv pip install flide torchvision
flide serve --model OmniJev/OneJev1.6-0.8B --port 8000

Option B: llama.cpp

For GGUF models. Supports text and images; use Flide for video. Install llama.cpp first (brew install llama.cpp on macOS).

pip install git+https://github.com/OmniJev/OneJev.git
qev serve --gguf mradermacher/OneJev1.6-0.8B-GGUF:Q8_0 --port 8000

Option C: Docker

The repo ships a Compose stack that detects your hardware and starts one backend. The llama.cpp profiles download mradermacher/OneJev1.6-0.8B-GGUF:Q8_0 on first start; set QEV_GGUF in docker/.env for another size or quant.

cp docker/.env.example docker/.env   # set HF_TOKEN for gated/private repos
./docker/start.sh                    # auto-detect (Windows: .\docker\start.ps1)
./docker/start.sh logs               # follow logs
./docker/start.sh down               # stop every profile

Force a backend with ./docker/start.sh <torch|rocm|vulkan|gguf|gguf-rocm|cpu> (add --build to rebuild). Every profile serves the API and the playground on :8000. On Windows only torch (NVIDIA) and cpu are available.

Send a request

The server answers at http://localhost:8000. In another terminal, run this example with your own screenshot.png:

IMG=$(base64 -w0 screenshot.png)
curl http://localhost:8000/v1/systemone -H "Content-Type: application/json" -d @- <<EOF
{
  "state": {"task": "Pay the open invoice from ACME", "screen": "<image:1>"},
  "media": [{"type": "image", "data": "data:image/png;base64,$IMG"}],
  "questions": {
    "done": {"type": "noul", "instructions": "The invoice has been paid"},
    "next": {"type": "choice", "instructions": "What should the agent do next?",
             "criteria": {"click": "click an element", "type": "type text", "scroll": "scroll", "stop": "stop"}},
    "progress": {"type": "score", "instructions": "How far along is the task?",
                 "criteria": ["not started", "halfway", "almost done", "done"]}
  }
}
EOF

answers.done.noul is the probability of yes, answers.next.probabilities gives one probability per option and answers.progress.score is the expected level.

The API is compatible with TypeSafe System One. More examples: video, curl, official TypeSafe SDK.

Training

git clone https://github.com/OmniJev/OneJev.git
cd OneJev
pip install -e ".[train]"

# Choose A or B. A is enabled below.
# A. Demo: 100 examples with images (23 MB)
hf download OmniJev/OneJev-Data sample/sample-100.parquet --repo-type dataset --local-dir data/onejev
python -m train.unpack data/onejev --sample

# B. Full dataset: 94,707 examples (17.7 GB). Uncomment these two lines instead of A.
# hf download OmniJev/OneJev-Data --repo-type dataset --include "data/*.parquet" --local-dir data/onejev
# python -m train.unpack data/onejev

Either option creates data/onejev/train.jsonl and extracts images and video frames into data/onejev/media/.

OneJev-Data releases 94,707 of the original 99,193 training questions; the remaining sources do not permit redistribution.

Fine-tune

The four models fine-tune Qwen3.5-0.8B, Qwen3.5-4B, Qwen3.5-9B and Qwen3.8-27B for one epoch with the vision tower frozen. The configs use a learning rate of 5e-6, a 16,384-token limit, and cross-entropy plus Brier loss over answer probabilities. To train the 4B model on four GPUs:

torchrun --nproc-per-node 4 -m train.sft --config train/configs/onejev_4b_full.yaml

Configs: 0.8B ยท 4B ยท 9B ยท 27B. Set --nproc-per-node to your GPU count; the 9B and 27B configs enable FSDP to shard model and optimizer state. For your own data, change train and media_root in the config.

The final 4B checkpoint is saved to train/runs/onejev_4b/final. Start it with:

flide serve --model train/runs/onejev_4b/final --format onejev --port 8000

Friendly Links

License

Apache 2.0

Stars and downloads

OneJev stars on GitHub OneJev downloads on Hugging Face

Citation

@misc{onejev2026,
  title        = {{OneJev}: A Multimodal System One Decision Model},
  author       = {{OmniJev Team}},
  year         = {2026},
  howpublished = {\url{https://github.com/OmniJev/OneJev}}
}

About

๐Ÿš€๐Ÿš€ A multimodal System One decision model that gives calibrated answers to typed questions about screens, photos, video and text in one forward pass.

Topics

Resources

Stars

191 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages