-
-
Notifications
You must be signed in to change notification settings - Fork 16.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
half inference bug while using detect.py #3280
Comments
@edificewang thanks for the bug report! This seems to be a silent error since no error message is printed to screen right? I'd thought all CUDA devices supported FP16 ops. Do you have any source for a list of unsupported devices, or do you know a way to check in realtime for FP16 support? |
@edificewang I have an idea. Can you test inference on YOLOv5 PyTorch Hub models instead of using detect.py? The Hub models are loaded as FP32 and are run using AMP in a completely different inference pathway rather than converted to FP16. The command is:
Correct output is:
|
hello, @glenn-jocher |
@edificewang great thanks! I also opened up PR #3314 to modify detect.py for autocast usage, but unfortunately there seems to be a GPU speed penalty. You can test this branch also with:
|
python hubconf.py
Fusing layers...
Model Summary: 224 layers, 7266973 parameters, 0 gradients
Adding AutoShape...
YOLOv5 v5.0-106-g683cefe torch 1.8.1+cu111 CUDA:0 (Quadro T1000, 4096.0MB)
image 1/5: 720x1280
image 2/5: 720x1280
image 3/5: 1080x810
image 4/5: 1080x810
image 5/5: 320x640
Speed: 4599.0ms pre-process, 234.9ms inference, 0.5ms NMS per image at shape (5, 3, 640, 640)
Saved zidane.jpg, zidane.jpg, image2.jpg, bus.jpg, image4.jpg to runs\hub\exp It also output nothing! |
@edificewang ah I see, thanks! So autocast doesn't solve the problem either. |
@edificewang good news 😃! Your original issue may now be fixed ✅ in PR #3423. To receive this update:
Thank you for spotting this issue and informing us of the problem. Please let us know if this update resolves the issue for you, and feel free to inform us of any other issues you discover or feature requests that come to mind. Happy trainings with YOLOv5 🚀! |
🐛 Bug
When using detecto.py to make predictions, if the GPU does not support fp16 operations, an error occurs in the inference and no results can be obtained; when fp16 is turned off, the results are correct
Description
case1:
usage: python.exe .\detect.py --weights .\yolov5s.pt --source .\data\images
output
Namespace(agnostic_nms=False, augment=False, classes=None, conf_thres=0.25, device='', exist_ok=False, hide_conf=False, hide_labels=False, img_size=640, iou_thres=0.45, line_thickness=3, max_det=1000, name='exp', nosave=False, project='runs/detect', save_conf=False, save_crop=False, save_txt=False, source='.\data\images', update=False, view_img=False, weights=['.\yolov5s.pt'])
YOLOv5 v5.0-106-g683cefe torch 1.8.1+cu111 CUDA:0 (Quadro T1000, 4096.0MB)
Fusing layers...
Model Summary: 224 layers, 7266973 parameters, 0 gradients
image 1/2 D:\github\yolov5\data\images\bus.jpg: 640x480 Done. (0.051s)
image 2/2 D:\github\yolov5\data\images\zidane.jpg: 384x640 Done. (0.050s)
Results saved to runs\detect\exp3
Done. (0.172s)
case2:
modify:
change ./detect.py files
line 30 :# half = device.type != 'cpu' # half precision only supported on CUDA
line 30 : half = False # half precision only supported on CUDA
usage: python.exe .\detect.py --weights .\yolov5s.pt --source .\data\images
output
Namespace(agnostic_nms=False, augment=False, classes=None, conf_thres=0.25, device='', exist_ok=False, hide_conf=False, hide_labels=False, img_size=640, iou_thres=0.45, line_thickness=3, max_det=1000, name='exp', nosave=False, project='runs/detect', save_conf=False, save_crop=False, save_txt=False, source='.\data\images', update=False, view_img=False, weights=['.\yolov5s.pt'])
YOLOv5 v5.0-106-g683cefe torch 1.8.1+cu111 CUDA:0 (Quadro T1000, 4096.0MB)
Fusing layers...
Model Summary: 224 layers, 7266973 parameters, 0 gradients
image 1/2 D:\github\yolov5\data\images\bus.jpg: 640x480 4 persons, 1 bus, 1 fire hydrant, Done. (0.044s)
image 2/2 D:\github\yolov5\data\images\zidane.jpg: 384x640 2 persons, 2 ties, Done. (0.027s)
Results saved to runs\detect\exp4
Done. (0.132s)
Expected behavior
support an automatic judgment of whether the GPU supports fp16 operation,
The text was updated successfully, but these errors were encountered: