Use yaml file instead of CLI overrides

[jonasteuwen]

I am using hydra as follows:

@hydra.main(
    config_path="../config",
    config_name="train.yaml",
    version_base="1.2",
)
def main(config: DictConfig):
    train(config)

Subsequently I overwrite (set) some of the parameters on the command line:

python train.py \
    datamodule.num_workers=16 \
    datamodule.batch_size=32 \
    data_description=brca_tissue_segmentation_more_classes \
    transforms=segmentation \
    losses=segmentation \
    lit_module=new_unet

Is there a way to do this with a config? For instance we could run

python train.py --config experiment_config.yaml

which would, in this case have as settings:

datamodule:
    num_workers: 16
    batch_size: 32
data_description: brca_tissue_segmentation_more_classes
transforms: segmentation
losses: segmentation
lit_module: new_unet

[Jasha10]

Hi @jonasteuwen,

Have you seen the Hydra documentation on Configuring Experiments?

Make sure to check out the Example Application demonstrating the pattern. Using that example application, you can run each of the following in your shell to get a different result:

python my_app.py  # run config.yaml
python my_app.py +experiment=aplite  # run config.yaml merged with experiment/aplite.yaml
python my_app.py +experiment=nglite  # run config.yaml merged with experiment/nglite.yaml

[alexcoca]

@Jasha10 I was wondering what are the options for placing the experiments directory. Currently, I have a structure as follows:

evaluation
├── data_preparation
├── inference
│   ├── custom_evaluation_experiments
│   ├── model
│   ├── dataset
│   └── inference.yaml
└── metrics

Everything works fine following the docs you link,, but would it be possible to move the contents of the custom_evaluation_experiments out of evaluation/inference to evaluation_experiments group located at the same level with evaluation?

The command I am using to run my experiments is python scripts/run_inference.py --multirun '+ custom_evaluation_experiments =glob(*)'