Where the techniques for LLM inference are located.

Class of the request. Includes the information about a request.

Class of the scheduler. Manages iteration-level scheduling.

You can add your own scheduler here.

Memory model of LLMServingSim. Calculating KV cache sizes and weight sizes are located here.

Class that controls the flow between ASTRA-Sim and Scheduler.

Calls Chakra Graph Converter to convert trace into execution graph.

Lookup perf_model to generate model trace. Also uses memory_model for tensor sizes.

Generates network and memory config json file automatically. You can change it according to your needs.

Gets the pim trace and add the pim operator in the trace.

Where various helper functions are located. Especially model config and writing format of the trace file.