Add DATASET_VERSION to params, upload, and download#103
Open
jmoraispk wants to merge 2 commits into
Open
Conversation
DATASET_VERSION ("4a") is sent to the server on download requests so it
can return the correct dataset build, independently of the software version.
Increment the letter suffix (4a → 4b) when the on-disk format changes.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
1. consts.py: add DATASET_VERSION_PARAM_NAME = "dataset_version" 2. All three converters: write DATASET_VERSION into params.json alongside VERSION so every converted scenario carries its dataset schema version on disk 3. upload.py: read dmVersion from the dedicated dataset_version field (falls back to DATASET_VERSION constant if field is absent) 4. download.py: add version= parameter (default DATASET_VERSION) to download() and _download_url(); appended as ?version=<v> so the server can return the matching dataset build Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
jmoraispk
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements the dataset versioning scheme discussed with @sohamd22.
Three changes:
consts.py— addsDATASET_VERSION_PARAM_NAME = "dataset_version"alongside the existingDATASET_VERSION = "4a"constantdataset_versionintoparams.jsonat conversion time, so every scenario carries its schema version on diskupload.py—dmVersionnow reads from the dedicateddataset_versionfield in params (falls back toDATASET_VERSIONconstant if absent, for backwards compatibility)download.py—download()gains aversion=keyword argument (defaultDATASET_VERSION) appended to the URL as?version=4a, so the server can route to the correct dataset buildOpen question for @sohamd22
Is including
dataset_versionin the upload call redundant given the server already reads it fromparams.json? Current thinking: keep it in both places — the params file is the canonical source, but passing it explicitly in the upload call gives the server a fast path without having to parse the params file first, and makes the intent obvious at the API boundary. Happy to drop it from the upload if you prefer simpler server-side logic.Test plan
params.jsoncontains"dataset_version": "4a"upload()sends"dmVersion": "4a"inadvanced_paramsdownload("scenario_name")hits?version=4aby defaultdownload("scenario_name", version="4b")hits?version=4b