From dda993f3dd3c753dfd580d485b39c1001830fee4 Mon Sep 17 00:00:00 2001 From: AlexeyAB <alexeyab84@gmail.com> Date: Thu, 22 Feb 2018 19:54:40 +0000 Subject: [PATCH] Use half_float16 instead of float32 if defined both CUDNN and CUDNN_HALF. Use Tensor Cores. --- README.md | 54 +++++++++++++++++++++++++++++++++++++++--------------- 1 files changed, 39 insertions(+), 15 deletions(-) diff --git a/README.md b/README.md index 94fbdb9..ec8c19a 100644 --- a/README.md +++ b/README.md @@ -8,10 +8,11 @@ 4. [How to train (Pascal VOC Data)](#how-to-train-pascal-voc-data) 5. [How to train (to detect your custom objects)](#how-to-train-to-detect-your-custom-objects) 6. [When should I stop training](#when-should-i-stop-training) -7. [How to improve object detection](#how-to-improve-object-detection) -8. [How to mark bounded boxes of objects and create annotation files](#how-to-mark-bounded-boxes-of-objects-and-create-annotation-files) -9. [Using Yolo9000](#using-yolo9000) -10. [How to use Yolo as DLL](#how-to-use-yolo-as-dll) +7. [How to calculate mAP on PascalVOC 2007](#how-to-calculate-map-on-pascalvoc-2007) +8. [How to improve object detection](#how-to-improve-object-detection) +9. [How to mark bounded boxes of objects and create annotation files](#how-to-mark-bounded-boxes-of-objects-and-create-annotation-files) +10. [Using Yolo9000](#using-yolo9000) +11. [How to use Yolo as DLL](#how-to-use-yolo-as-dll) |  |  https://arxiv.org/abs/1612.08242 | |---|---| @@ -131,7 +132,7 @@ 2. If you have other version of **CUDA (not 8.0)** then open `build\darknet\darknet.vcxproj` by using Notepad, find 2 places with "CUDA 8.0" and change it to your CUDA-version, then do step 1 -3. If you **don't have GPU**, but have **MSVS 2015 and OpenCV 3.0** (with paths: `C:\opencv_3.0\opencv\build\include` & `C:\opencv_3.0\opencv\build\x64\vc14\lib`), then start MSVS, open `build\darknet\darknet_no_gpu.sln`, set **x64** and **Release**, and do the: Build -> Build darknet +3. If you **don't have GPU**, but have **MSVS 2015 and OpenCV 3.0** (with paths: `C:\opencv_3.0\opencv\build\include` & `C:\opencv_3.0\opencv\build\x64\vc14\lib`), then start MSVS, open `build\darknet\darknet_no_gpu.sln`, set **x64** and **Release**, and do the: Build -> Build darknet_no_gpu 4. If you have **OpenCV 2.4.13** instead of 3.0 then you should change pathes after `\darknet.sln` is opened @@ -195,7 +196,7 @@ 6. Set `batch=64` and `subdivisions=8` in the file `yolo-voc.2.0.cfg`: [link](https://github.com/AlexeyAB/darknet/blob/master/build/darknet/x64/yolo-voc.2.0.cfg#L2) -7. Start training by using `train_voc.cmd` or by using the command line: `darknet.exe detector train data/voc.data yolo-voc.2.0.cfg darknet19_448.conv.23` +7. Start training by using `train_voc.cmd` or by using the command line: `darknet.exe detector train data/voc.data yolo-voc.2.0.cfg darknet19_448.conv.23` (**Note:** If you are using CPU, try `darknet_no_gpu.exe` instead of `darknet.exe`.) If required change pathes in the file `build\darknet\x64\data\voc.data` @@ -272,7 +273,7 @@ 8. Start training by using the command line: `darknet.exe detector train data/obj.data yolo-obj.cfg darknet19_448.conv.23` - (file `yolo-obj_xxx.weights` will be saved to the `build\darknet\x64\backup\` for each 100 iterations until 1000 iterations has been reached, and after for each 1000 iterations) + (file `yolo-obj_xxx.weights` will be saved to the `build\darknet\x64\backup\` for each 100 iterations) 9. After training is complete - get result `yolo-obj_final.weights` from path `build\darknet\x64\backup\` @@ -309,23 +310,42 @@ 2.2 If training is stopped after 9000 iterations, to validate some of previous weights use this commands: -* `darknet.exe detector recall data/obj.data yolo-obj.cfg backup\yolo-obj_7000.weights` -* `darknet.exe detector recall data/obj.data yolo-obj.cfg backup\yolo-obj_8000.weights` -* `darknet.exe detector recall data/obj.data yolo-obj.cfg backup\yolo-obj_9000.weights` +(If you use another GitHub repository, then use `darknet.exe detector recall`... instead of `darknet.exe detector map`...) + +* `darknet.exe detector map data/obj.data yolo-obj.cfg backup\yolo-obj_7000.weights` +* `darknet.exe detector map data/obj.data yolo-obj.cfg backup\yolo-obj_8000.weights` +* `darknet.exe detector map data/obj.data yolo-obj.cfg backup\yolo-obj_9000.weights` And comapre last output lines for each weights (7000, 8000, 9000): -> 7586 7612 7689 RPs/Img: 68.23 **IOU: 77.86%** Recall:99.00% - -* **IOU** - the bigger, the better (says about accuracy) - **better to use** -* **Recall** - the bigger, the better (says about accuracy) - actually Yolo calculates true positives, so it shouldn't be used +Choose weights-file **with the highest IoU** (intersect of union) and mAP (mean average precision) For example, **bigger IOU** gives weights `yolo-obj_8000.weights` - then **use this weights for detection**. +Example of custom object detection: `darknet.exe detector test data/obj.data yolo-obj.cfg yolo-obj_8000.weights` + +* **IoU** (intersect of union) - average instersect of union of objects and detections for a certain threshold = 0.24 + +* **mAP** (mean average precision) - mean value of `average precisions` for each class, where `average precision` is average value of 11 points on PR-curve for each possible threshold (each probability of detection) for the same class (Precision-Recall in terms of PascalVOC, where Precision=TP/(TP+FP) and Recall=TP/(TP+FN) ), page-11: http://homepages.inf.ed.ac.uk/ckiw/postscript/ijcv_voc09.pdf + +In terms of Wiki, indicators Precision and Recall have a slightly different meaning than in the PascalVOC competition, but **IoU always has the same meaning**.  -How to calculate **mAP** [voc_eval.py](https://github.com/AlexeyAB/darknet/blob/master/scripts/voc_eval.py) or [datascience.stackexchange link](https://datascience.stackexchange.com/questions/16797/what-does-the-notation-map-5-95-mean) +### How to calculate mAP on PascalVOC 2007: + +1. To calculate mAP (mean average precision) on PascalVOC-2007-test: +* Download PascalVOC dataset, install Python 3.x and get file `2007_test.txt` as described here: https://github.com/AlexeyAB/darknet#how-to-train-pascal-voc-data +* Then download file https://raw.githubusercontent.com/AlexeyAB/darknet/master/scripts/voc_label_difficult.py to the dir `build\darknet\x64\data\voc` then run `voc_label_difficult.py` to get the file `difficult_2007_test.txt` +* Remove symbol `#` from this line to un-comment it: https://github.com/AlexeyAB/darknet/blob/master/build/darknet/x64/data/voc.data#L4 +* Then there are 2 ways to get mAP: + 1. Using Darknet + Python: run the file `build/darknet/x64/calc_mAP_voc_py.cmd` - you will get mAP for `yolo-voc.cfg` model, mAP = 75.9% + 2. Using this fork of Darknet: run the file `build/darknet/x64/calc_mAP.cmd` - you will get mAP for `yolo-voc.cfg` model, mAP = 75.8% + + (The article specifies the value of mAP = 76.8% for YOLOv2 416×416, page-4 table-3: https://arxiv.org/pdf/1612.08242v1.pdf. We get values lower - perhaps due to the fact that the model was trained on a slightly different source code than the code on which the detection is was done) + +* if you want to get mAP for `tiny-yolo-voc.cfg` model, then un-comment line for tiny-yolo-voc.cfg and comment line for yolo-voc.cfg in the .cmd-file +* if you have Python 2.x instead of Python 3.x, and if you use Darknet+Python-way to get mAP, then in your cmd-file use `reval_voc.py` and `voc_eval.py` instead of `reval_voc_py3.py` and `voc_eval_py3.py` from this directory: https://github.com/AlexeyAB/darknet/tree/master/scripts ### Custom object detection: @@ -341,6 +361,10 @@ * desirable that your training dataset include images with objects at diffrent: scales, rotations, lightings, from different sides + * for training on small objects, add the parameter `small_object=1` in the last layer [region] in your cfg-file + + * for training with a large number of objects in each image, add the parameter `max=200` or higher value in the last layer [region] in your cfg-file + 2. After training - for detection: * Increase network-resolution by set in your `.cfg`-file (`height=608` and `width=608`) or (`height=832` and `width=832`) or (any value multiple of 32) - this increases the precision and makes it possible to detect small objects: [link](https://github.com/AlexeyAB/darknet/blob/master/cfg/yolo-voc.2.0.cfg#L4) -- Gitblit v1.10.0