For example, if I want to run the script
python baselines/her/experiment/train.py --num_cpu 19 --env_name HandManipulateBlock-v0 --n_epochs 200 --replay_strategy future

Will each mpi worker hold an individual replay buffer?

Within each mpi worker, I calculated the sum of the actions in the replay buffer and found them to be different.