The updated example looks good. I have made some cmake changes.

Cool tks. Feel free to merge/modify your way.

I ran it on multiple(two: GTX 1060, AMD Spectre R7) devices, works without any issues. This change demonstrates how to use multiple devices but not so much on how to distribute training data across multiple devices. How difficult would be such an example to implement ?

how to distribute training data across multiple devices. How difficult would be such an example to implement ?
That will be worth it to do as soon as
#2463
done.
That example at least shows how to do multigpu multimodels training.

That item would allow to also compare speed of mutiple vs single gpus execution.
Distruibuted single model training is out of scope here.
Review please ?