Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] support requires_grad key in paramwise_cfg #767

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
+32 −11
Open

[Feature] support requires_grad key in paramwise_cfg #767

Changes from 3 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
e0a9753
add requires_grad feat
cxiang26 Nov 25, 2022
5f6f18d
Merge branch 'open-mmlab:main' into add_requires_grad_feat
cxiang26 Nov 25, 2022
022ef78
replace inplace op
cxiang26 Nov 26, 2022
effc23f
add requires_grad in custom_keys unit test
cxiang26 Nov 26, 2022
66be294
fix test logic
cxiang26 Nov 26, 2022
ab2f6f4
supplement the usage of required_grad
cxiang26 Nov 27, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions mmengine/optim/optimizer/default_constructor.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -217,6 +217,9 @@ def add_params(self,
if self.base_wd is not None:
decay_mult = custom_keys[key].get('decay_mult', 1.)
param_group['weight_decay'] = self.base_wd * decay_mult
requires_grad = custom_keys[key].get('requires_grad', True)
if not requires_grad:
param_group['params'][0].requires_grad_(requires_grad)
# add custom settings to param_group
for k, v in custom_keys[key].items():
param_group[k] = v
Expand Down