Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

stop_target_gradients default should be True in GAE function #28

Open
zhongwen opened this issue Sep 10, 2021 · 1 comment
Open

stop_target_gradients default should be True in GAE function #28

zhongwen opened this issue Sep 10, 2021 · 1 comment

Comments

@zhongwen
Copy link

truncated_generalized_advantage_estimation should have the stop_target_gradients defaulted to True

https://github.com/deepmind/rlax/blob/383f93bc8b33c3d1bc28f15e1e07fc5104c790ea/rlax/_src/multistep.py#L279

The False case applies only to meta-gradients use case which is rare in the common agents. We should mark this option as defaulted to be True to avoid usage bugs.

WDYT?

@zhongwen
Copy link
Author

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant