Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Loss, Logits error while training #53

Open
niranjanakella opened this issue Jul 13, 2023 · 1 comment
Open

Loss, Logits error while training #53

niranjanakella opened this issue Jul 13, 2023 · 1 comment

Comments

@niranjanakella
Copy link

niranjanakella commented Jul 13, 2023

There is a assignment error in the train.py script where in the loss and logits are considered to be 'str' type after the assignment and hence have to be updated.

Line: 108 and 146

loss, logits = model(texts, attention_mask=masks, labels=labels)

Here the loss variable is assigned as a 'str' type hence the following loss.backward() would fail stating that a 'str' type doesn't have a backward method.

@niranjanakella niranjanakella changed the title Inconsistency between labels Loss, Logits error while training Jul 14, 2023
@niranjanakella
Copy link
Author

niranjanakella commented Jul 14, 2023

This can be corrected by re-assigning the loss and logits to the corresponding model output values.

Before

loss, logits = model(texts, attention_mask=masks, labels=labels)

After

model_out = model(texts, attention_mask=masks, labels=labels)
loss, logits = model_out.loss, model_out.logits

NOTE: PR #54 has been raised

aenriquez27 added a commit to aenriquez27/gpt-2-output-dataset that referenced this issue Feb 26, 2024
Loss, Logits error while training
openai#53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant