Add scale map to apply_max_norm_regularization

[rockerBOO]

Allows setting the scale_weight_norms via

• --scale_weight_norms_map "{'proj': 100.0}"
• toml scale_weight_norms_map = { proj = 100.0 }

Set --scale_weight_norms 5.0 to some value for the other values and the map would set those values for matching module keys.

This is is just an idea to allow us to scale the weight norms differently for different modules. The implementation details are up in the air because I'm not sure what would be best here.

Could be that we want to use regex for the module name matching. Or make the map work exclusively to scale_weight_norms.

See https://github.com/rockerBOO/sd-ext/blob/main/scale_norms.py which emulates the same behavior but on the static weights. You can try different strategies with the map.

[recris]

This is something I wanted to implement for some time.

Maybe match the names using fnmatch style patterns? Slightly simpler than regex.

[rockerBOO]

Yeah fnmatch seems to be a good fit. I will give it a try.