Skip to content

训练自己数据集的问题 #5

@Zzhuoyuu

Description

@Zzhuoyuu

我使用我找到的数据集,由于它里面图片的大小和你的不一致(大部分大于256*256),所以我先压缩了大小,然后训练时出现了loss一直为nan的情况,请问你知道如何解决吗,感谢!

(pytorch) PS E:\QRCodeDetection-master\QRCodeDetection-master> python train.py -d ./resized_images -b 64 -e 80
2024-05-12 22:12:03.203769: E tensorflow/core/profiler/internal/gpu/cupti_tracer.cc:1666] function cupti_interface_->Subscribe( &subscriber_, (CUpti_CallbackFunc)ApiCallback, this)failed with error CUPTI could not be loaded or symbol could not be found.
2024-05-12 22:12:03.204291: E tensorflow/core/profiler/internal/gpu/cupti_tracer.cc:1757] function cupti_interface_->Finalize()failed with error CUPTI could not be loaded or symbol could not be found.
Epoch 1/80
E:\QRCodeDetection-master\QRCodeDetection-master\utils\util.py:134: RuntimeWarning: divide by zero encountered in log
tw = np.log(w / anchor_w)
E:\QRCodeDetection-master\QRCodeDetection-master\utils\util.py:135: RuntimeWarning: divide by zero encountered in log
th = np.log(h / anchor_h)
1/22 [>.............................] - ETA: 6:23 - loss: 29.2871 - score_acc: 0.5300 - cls_acc: 0.47972024-05-12 22:12:23.009745: E tensorflow/core/profiler/internal/gpu/cupti_tracer.cc:1666] function cupti_interface_->Subscribe( &subscriber_, (CUpti_CallbackFunc)ApiCallback, this)failed with error CUPTI could not be loaded or symbol could not be found.
2/22 [=>............................] - ETA: 4:12 - loss: 17.0141 - score_acc: 0.7628 - cls_acc: 0.39232024-05-12 22:12:34.730575: E tensorflow/core/profiler/internal/gpu/cupti_tracer.cc:1757] function cupti_interface_->Finalize()failed with error CUPTI could not be loaded or symbol could not be found.
22/22 [==============================] - 255s 11s/step - loss: nan - score_acc: 0.9753 - cls_acc: 0.4812
Epoch 2/80
22/22 [==============================] - 248s 11s/step - loss: nan - score_acc: 0.9966 - cls_acc: 0.9834
Epoch 3/80
22/22 [==============================] - 255s 11s/step - loss: nan - score_acc: 0.9966 - cls_acc: 0.9834
Epoch 4/80
22/22 [==============================] - 271s 12s/step - loss: nan - score_acc: 0.9966 - cls_acc: 0.9834
Epoch 5/80
22/22 [==============================] - 304s 14s/step - loss: nan - score_acc: 0.9966 - cls_acc: 0.9834 - val_loss: nan - val_score_acc: 0.9961 - val_cls_acc: 0.9774

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions