Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

scaler #10

Open
Benbie opened this issue May 24, 2024 · 6 comments
Open

scaler #10

Benbie opened this issue May 24, 2024 · 6 comments

Comments

@Benbie
Copy link

Benbie commented May 24, 2024

assert len(optimizer_state["found_inf_per_device"]) > 0, "No inf checks were recorded for this optimizer."
作者大大您好,我在三维医学图像上用您的方法,scaler报错这个,去掉scaler后loss完全不变,参数没有更新,请问可能是什么问题

@Haochen-Wang409
Copy link
Owner

您具体是怎么“去掉”scaler的?修改了哪些代码?

@Benbie
Copy link
Author

Benbie commented May 24, 2024

image
我将这部分换成了传统的 loss.backward() optimizer.step()

@Benbie
Copy link
Author

Benbie commented May 24, 2024

然后loss_learn保持不变,可能是参数完全没有任何更新

@Haochen-Wang409
Copy link
Owner

在其他的数据集,如ImageNet上,可以正常运行吗?

@Benbie
Copy link
Author

Benbie commented May 24, 2024

我将试一下其他的三维数据集,谢谢您

@Haochen-Wang409
Copy link
Owner

建议您在ImageNet数据集先跑通,因为我们并没有在其他数据集上进行预训练。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants