AlexeyAB / Yolo_mark

GUI for marking bounded boxes of objects in images for training neural network Yolo v3 and v2

Home Page:https://github.com/AlexeyAB/darknet

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

getting Nan Value while training yolov4 on custom data

arpitvw opened this issue · comments

Hello Team ,
I am using yolov4 for object detection and training it on custom dataset
after 300 iteration i am getting Nan value for weights
weights used:

  1. yolov4.conv.137 (after 6000 iteration also not able to detect object
  2. Tried to train it from scratch with below command but getting Nan value
    !./darknet detector train data/obj.data cfg/yolo-obj.cfg yolov4.weights -clear -dont_show -map

image

Note: Training data is related to crops

solutions:
1.Decrease the learning rate suppose, for example, if you are using learning rate as 0.00001, use 0.0000001.
2.adjust the learning_decay
3.add clipnorm value (the last option but it is optional)
4. Increase the validation data