English | 简体中文

This directory contains runnable client examples for the local GameAssistantToolServer HTTP APIs.

• knowledge/: Knowledge service sample (insert/build/query text knowledge).
• memory/: Memory service sample (insert/build/search structured records with images).
• mmr/: MMR service sample (multimodal insert/query by text and image).
• vision/: Vision service sample (scene image insert/build/query).
• asr/: ASR service sample (query audio by raw samples, file, or OpenAI API).
• ocr/: OCR service sample (query text from an image, with or without boxes).
• bar/: BAR service sample (boss action recognition: insert/build/query).
• vlm/: VLM service sample (streamed text / text+image chat).
• requirements.txt: Python dependencies for all sample scripts.

1. Start GameAssistantToolServer first (default host: 127.0.0.1:9190).
2. Ensure required models are prepared according to the root README.
3. Use Python 3.10+ (recommended).

From this directory:

cd samples
pip install -r requirements.txt

Dependencies in requirements.txt:

• rich
• requests-toolbelt
• pillow

Run commands from samples:

cd knowledge
python run_knowledge_sample.py

What it does:

1. Uses sample_texts/sample_1.txt and sample_texts/sample_2.txt.
2. Initializes one knowledge instance.
3. Inserts text files and builds index.
4. Queries twice.
5. Cleans up instance.

cd memory
python run_memory_sample.py

What it does:

1. Generates demo images under memory/sample_images.
2. Initializes one memory instance.
3. Inserts records with properties/tags/images.
4. Builds index.
5. Runs condition, text, and image search.
6. Validates top-1 image result and cleans up.

cd mmr
python run_mmr_sample.py

What it does:

1. Generates demo images under mmr/sample_images.
2. Checks MMR enable state.
3. Initializes one MMR instance.
4. Inserts text-only, image-only, and hybrid records.
5. Builds index.
6. Queries by text and image.
7. Lists records and cleans up.

cd vision
python run_vision_sample.py

What it does:

1. Generates demo images under vision/sample_images.
2. Initializes one vision instance and inserts scene pictures.
3. Builds index.
4. Queries with the same images.
5. Verifies top-1 picture id matches expectation.
6. Cleans up instance.

cd asr
python run_asr_sample.py

What it does:

1. Uses the provided audio file asr/sample_audio/demo_speech.wav.
2. Checks the ASR service enable state.
3. Queries ASR by raw samples (JSON list decoded from the wav).
4. Queries ASR by uploading the audio file.
5. Queries ASR via the OpenAI-compatible /v1/audio/transcriptions endpoint.

cd ocr
python run_ocr_sample.py

What it does:

1. Generates a demo image with text lines under ocr/sample_images.
2. Checks the OCR service enable state.
3. Queries OCR and prints the recognized text.
4. Queries again with return_boxes=True to show bounding boxes.
5. Queries a blank image to show an empty result.

cd bar
python run_bar_sample.py

What it does:

1. Generates a demo boss action (a moving shape) under bar/sample_frames.
2. Initializes one BAR instance.
3. Inserts one boss action.
4. Adds a point annotation to frame 0.
5. Processes and builds the instance.
6. Queries with a frame and prints the recognized boss action.
7. Cleans up instance.

cd vlm
python run_vlm_sample.py

What it does:

1. Generates a demo image under vlm/sample_images.
2. Checks the VLM service enable state.
3. Queries the VLM with text only.
4. Queries with text + image.
5. Queries with text + two images.
6. Prints the streamed answers.

All APIs default to:

• HOST = "127.0.0.1:9190"

If your server runs elsewhere, edit these files:

• knowledge/knowledge_api.py
• memory/memory_api.py
• mmr/mmr_api.py
• vision/vision_api.py
• asr/asr_api.py
• ocr/ocr_api.py
• bar/bar_api.py
• vlm/vlm_api.py

A successful run usually shows:

• API responses with "code": "ok"
• progress output for build/query stages
• final cleanup request success

• Connection errors: confirm server is running and reachable on the configured host.
• Build/query failures: check server logs and model/config readiness.
• Missing packages: re-run pip install -r requirements.txt.
• Permission/file errors: run from the corresponding sample directory so relative paths resolve correctly.