Repository navigation
[Bug]: Run cost is underestimated in the run list for runs with multiple job submissions #4448
Description
Steps to reproduce
- Submit a run whose job gets more than one submission that actually runs, e.g. a task retried after an interruption or error, or a service redeployed with a rolling deployment.
- Compare the run's cost in the UI run list and on the run details page.
Actual behaviour
The run list shows a lower cost than the run details page. The run list only accounts for the latest submission of each job.
The run cost is summed over the job submissions loaded for the response:
dstack/src/dstack/_internal/server/services/runs/__init__.py
Lines 1248 to 1261 in 3b13a7a
runs/list drops older submissions according to job_submissions_limit before the cost is calculated:
dstack/src/dstack/_internal/server/services/runs/__init__.py
Lines 1074 to 1080 in 3b13a7a
The UI run list requests job_submissions_limit: 1, so its cost column shows the cost of the latest submission only:
API clients that omit job_submissions_limit get the default of 10 submissions per job, so their cost is also underestimated for runs with more submissions. runs/get returns the full submission history, which is why the run details page is correct.
Expected behaviour
The run cost is the same in the run list, on the run details page, and in the API, regardless of job_submissions_limit.
Calculating costs from job submissions is not viable. A correct cost requires loading every submission of every listed run, which is what job_submissions_limit exists to avoid, and it makes cost aggregation over many runs (e.g. per project or user) impractical. Run costs should be calculated with a proper metering system that records usage and allows calculating costs over runs efficiently.
dstack version
master (3b13a7a)
Activity