This template ensures the reproducibility of your results through 3 artifacts:

1. The development environment
   • Recorded in the Docker images that you upload and described in Dockerfile and environment files that you keep up to date with the Docker installation.
   • (Less reliably) described in the environment file that you keep up to date for the conda installation.
2. The project code.
   • Recorded in the git repository that you keep up to date.
   • Made reproducible (to a desired degree) by you correctly seeding the random number generators and optionally removing non-deterministic operations, or replicable by running enough seeds.
3. The data, outputs, model weights and other artifacts.
   • Recorded and uploaded by you.
   • (Virtually) placed in the placeholder directories abstracting away the user storage system.

The template provides an automatic setup of the checkpointing directory for an experiment. The unique identifier for the directory is created by hashing the config used and optionally the git commit sha. Running the same experiment with the same config will thus set its working directory to the same checkpoint directory every time (if the resuming option is enabled).

To use this feature pass resuming.resume=True and resuming.use_commit=True to your script using a Hydra config that inherits form the setup.yaml config file, like the template_experiment.py script.

Even without using resuming.use_commit=True, the path to the checkpoint directory will be computed, and you could for example, read from it.

You can also force a resuming directory by passing resuming.resume_dir=<path> to your script.

For a non-sweep run, the run will have the id of the checkpoint directory as its wandb id, therefore your wandb run will stay the same and resume when your run is resumed. Make sure to use a custom step key when you log metrics so that you can have full control over when to start rewriting when you resume (E.g. if you checkpoint less often than you log, you may relog from the last checkpoint), otherwise the default step key of wandb will resume from the latest step and may be inconsistent with the checkpoint.

For a sweep run, it already has an id from the sweep, so to resume it you should manually get its id and restart the script with the same arguments the sweep agent started it, this way the config and the checkpoint directory will be the same (i.e. go to the wandb run UI, copy-paste the command it was run with and add wandb.run_id=<id-of-the-run>). This is a limitation of the wandb sweep system. See this issue.

A project started from a template is different from a fork in that it is not (necessarily) meant to be updated. The template is free to change and evolve, and it is not guaranteed that it will be compatible with your project.

Nevertheless, many changes are likely to be compatible with your project. In that case, there are two ways to incorporate them:

1. Manually copy the changes from the template to your project, adapting them when needed (different variable names especially).
2. Use git to merge the changes from the template to your project.

git remote add template https://github.com/CLAIRE-Labo/python-ml-research-template.git
git fetch template
# Cherry pick the commits you want to merge, making sure they are compatible.
# Add the option -n if you want to have the changes staged but not committed so you can edit them.
git cherry-pick -x <commit-hash>

Conda environments are not so self-contained, some packages can just work on your machine because they use some of your system libraries not recorded in the conda environment. An exhaustive and precise list of the system libraries outside conda is hard to record, and the environment will not run on another machine missing those libraries. Reinforcement learning (RL) environments usually require system libraries not recorded by conda, and RL is a big part of our work.

Moreover, the environment is specified as an environment.yml file description, and does not contain the dependencies themselves. Some dependencies may actually become unavailable when a user tries to download them later.

Docker images are self-contained and do contain the dependencies themselves.

Some reasons for many files and extra code is to be able to provide a generic template that can be configured, extended, or shortened depending on the needs of the project.

The other part of the apparent complexity probably comes from unfamiliarity with the tools and practices used in the template. These practices, however, (although usually not all combined in a research project, whence this template) are well established and have been proven to be very useful.

For example, the Dockerfile seems complex because it leverages multi-staging to be very time and cache-efficient. Different build stages can run in parallel, so changing your build dependencies, or installing something in the Dockerfile will cause very few rebuilds.

Using Docker Compose is also very convenient to define all the build arguments and multiple deployment options in a single file, avoiding long build and run commands.

This template is mainly addressed to students and researchers at CLAIRE. Frequently, students are not aware of the tools and practices that are available to them until they face the problems we've all faced at some point in our career (how do I manage my configs? How do conveniently log my metrics?, etc.). We chose to include these tools by default to help students and researchers avoid these problems from the start, and to encourage them to use them.

Yes, it seems like filling the template/template-variables.env file with your new project name and running ./template/change-project-name.sh would work.

The template is mainly designed to start new projects, as it's hard to make assumptions on the structure of an existing project. However, it is possible to use it to transfer from an existing project.

It's likely that your project is a bunch of Python files and a requirements.txt or environment.yml file. You can copy those files and potentially refactor the package structures and put all of them under src. You will also have to transfer the dependencies to the environment.yml file and identify the system dependencies.

If your project provides a Docker image, the Docker installation method allows extending it, assuming it has a well-configured Python environment.

In the worst case, you can keep the installation/ directory if that's useful to you and replace all the rest with your project and adapt the installation as needed. You could also just get some inspiration from the template and do your own thing.