You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit e361256
Browse filesBrowse the repository at this point in the historyBrowse files
- 🚀 **Run a 120B model at 28 tokens/s on an 8 GB laptop GPU:** a llama.cpp extension that keeps hot MoE experts in VRAM, up to **2.1x faster** than the best layer split.
28
-
- 📈 **Raised model accuracy by +8.14 points on average** across 15 benchmark runs by cleaning training data, and cut training time by **~49%**.
29
26
- 🧠 **Gave AI coding agents a memory over MCP** that answers a question in ~168 tokens instead of reading 38,921 tokens of source (**~231x cheaper**).
30
-
- 🖥️ **Built a desktop app for local and cloud models** with an approval-gated agent mode and a Rust addon that made audit logging ~60x faster.
31
-
- 🔐 **Wrote a fail-closed Rust MCP gateway** with OIDC, RBAC, signed approval tickets and a tamper-evident audit log.
27
+
- 🎓 **Turned any LLM into a calibrated tutor** that remembers what you've mastered, shipped as a CLI on PyPI.
28
+
- 🧑 **Built a tester that uses your app like a real person:** typos, rage-clicks and locale mix-ups from 8 personas across web, CLI, API, chat and more, with zero runtime dependencies.
29
+
- 🚀 **Run a 120B model at 28 tokens/s on an 8 GB laptop GPU:** a llama.cpp extension that keeps hot MoE experts in VRAM, up to **2.1x faster** than the best layer split.
32
30
- 🧪 **Measured, tested and documented**: I don't claim a number until a benchmark backs it.
33
31
34
32
---
35
33
36
34
## 🛠️ ML and AI projects
37
35
38
-
### 🚀 [extended-strata-moe](https://github.com/voidstackloop/extended-strata-moe): big MoE models on a small GPU
39
-
40
-
Extends [Strata](https://github.com/Niko1221/Strata) from one model to **every Mixture-of-Experts architecture llama.cpp supports** (DeepSeek, Qwen3 MoE, gpt-oss, GLM, Llama 4, Kimi and more). The experts a conversation actually uses stay cached in VRAM, the CPU computes the misses from system RAM at the same time, and the two sync through a **3 µs CUDA mailbox**, so nothing is uploaded over PCIe during generation.
Verified on 53 MoE architectures with KL divergence under 0.002 against CPU-only evaluation. It serves OpenAI- and Anthropic-compatible APIs, so Claude Code or Cursor can use it directly.
48
-
49
-
`C++ / CUDA``Python``llama.cpp``LLM inference`
50
-
51
-
---
52
-
53
-
### 🧠 [modelforge](https://github.com/voidstackloop/modelforge): local and cloud AI models in one desktop app
54
-
55
-
An Electron app that puts local models (llama.cpp, vLLM) and cloud models (OpenAI, Anthropic, Gemini) in one streaming chat, now focused on clinical decision support and research.
56
-
57
-
-**Agent mode with guardrails:** the model can read and write files and run commands in a folder you choose, and every action needs your approval.
58
-
-**Safety by design:** emergency red-flag detection, allergy and medication conflict warnings, and a consent preview before patient data goes to a cloud provider.
59
-
-**Rust where it earns its place:** a napi-rs addon cut 5,000 audit-log writes from about 20 s to about 0.3 s, with a TypeScript fallback.
60
-
-**Honest scope:** documented as decision support, not a diagnostician or a certified medical product.
61
-
62
-
<palign="center"><imgsrc="https://raw.githubusercontent.com/voidstackloop/modelforge/main/docs/screenshots/chat.png"width="720"alt="modelforge chat with a local model"></p>
### 📊 [buffdata](https://github.com/voidstackloop/buffdata): better training data means better models
69
-
70
-
A Python CLI and SDK that validates, deduplicates, PII-redacts and LLM-scores datasets before training. It works with OpenAI, Anthropic, Gemini or a local model, and a **Rust worker** runs managed jobs from a queue.
71
-
72
-
| Benchmark (40% duplicated, 10% empty rows) | Before | After |
73
-
|---|---|---|
74
-
| DBpedia-14 | 86.97% |**96.68%**|
75
-
| Emotion | 71.23% |**83.77%**|
76
-
| AG News, 100k rows | 83.39% |**89.42%**|
77
-
| Training time, AG News 100k | 80.2 s |**42.3 s**|
78
-
79
-
On already-clean data, accuracy stays within ±0.10 points, so it doesn't damage good datasets.
80
-
81
-
`Python``Rust``LLM APIs``Data quality`
82
-
83
-
---
84
-
85
36
### 🗂️ [project-memory](https://github.com/voidstackloop/project-memory): long-term memory for AI coding agents
86
37
87
38
`pmem` stores a project's conventions, decisions and patterns in SQLite and serves them to any MCP client (Claude Code, Cursor, OpenCode), so an agent stops re-reading the codebase every session.
@@ -107,19 +58,44 @@ pipx install expert-mentor
107
58
108
59
---
109
60
110
-
### 🔐 [modelforge-clinical-mcp](https://github.com/voidstackloop/modelforge-clinical-mcp): a security-first MCP gateway for clinical AI
61
+
### 🧑 [human-like-ai-tester](https://github.com/voidstackloop/human-like-ai-tester): test your app like a real, messy human
62
+
63
+
A Python CLI and library on [PyPI](https://pypi.org/project/human-like-ai-tester/) that tests web apps, CLIs, APIs, chatbots, desktop and mobile apps and data pipelines the way people actually use them: adjacent-key typos, `(555) 234-5678` in digit-only fields, rage-clicks after 1.5 s of silence, Back-button navigation and `12/05/2026` read in the wrong locale.
111
64
112
-
A Rust gateway that decides what an AI model may read or change. It fails closed and enforces OIDC tokens, roles, per-field grants and kill switches. Writes need a signed single-use approval ticket and are deduplicated by an idempotency key, and every decision lands in an audit log without patient data.
65
+
-**8 personas**, from a rushed commuter to a screen-reader user, each with its own typo rate, patience and habits.
66
+
-**One merged audit:** a multi-persona sweep produces a single deduplicated Markdown and JSON report.
67
+
-**Agent-ready:** ships LLM prompts and an agent skill so an AI agent can run the same human-style test session.
68
+
- Zero runtime dependencies, 71 network-free tests and CI on Python 3.9 to 3.13.
69
+
70
+
```sh
71
+
pip install human-like-ai-tester
72
+
human-tester demo
73
+
```
74
+
75
+
`Python``AI agents``QA automation``Accessibility`
76
+
77
+
---
113
78
114
-
`Rust``MCP``OIDC / OAuth 2``Zero trust`
79
+
### 🚀 [extended-strata-moe](https://github.com/voidstackloop/extended-strata-moe): big MoE models on a small GPU
80
+
81
+
Extends [Strata](https://github.com/Niko1221/Strata) from one model to **every Mixture-of-Experts architecture llama.cpp supports** (DeepSeek, Qwen3 MoE, gpt-oss, GLM, Llama 4, Kimi and more). The experts a conversation actually uses stay cached in VRAM, the CPU computes the misses from system RAM at the same time, and the two sync through a **3 µs CUDA mailbox**, so nothing is uploaded over PCIe during generation.
Verified on 53 MoE architectures with KL divergence under 0.002 against CPU-only evaluation. It serves OpenAI- and Anthropic-compatible APIs, so Claude Code or Cursor can use it directly.
89
+
90
+
`C++ / CUDA``Python``llama.cpp``LLM inference`
115
91
116
92
---
117
93
118
94
## 🧭 How I work
119
95
120
96
I'm self-taught and learned by building real projects instead of tutorials. For every project, I:
121
97
122
-
1.**Start with the hard problem.** GPU memory scheduling, dataset contamination, agent permissions: the part most people would outsource.
98
+
1.**Start with the hard problem.** GPU memory scheduling, agent context, human error: the part most people would outsource.
123
99
2.**Measure it.** Every number above comes from a benchmark in the repo.
124
100
3.**Make it runnable.** One-command setup, CI, releases and docs, so someone else can use it without asking me.
0 commit comments