Page 1 of 3
YOLO v5 recognition
Posted: Sun 30 Aug 30 2020 1:52 pm
by Daniel Wee
Installation for Windows 10 requires torch 1.6.0 for python 3.8 and CUDA 10.2:-
Code: Select all
pip install torch===1.6.0 torchvision===0.7.0 -f https://download.pytorch.org/whl/torch_stable.html
for CUDA 10.1 use:-
Code: Select all
pip install torch==1.6.0+cu101 torchvision==0.7.0+cu101 -f https://download.pytorch.org/whl/torch_stable.html
Apart from this, the standard procedure is as follows:-
Code: Select all
cd Nextcloud/python
git clone https://github.com/ultralytics/yolov5
cd yolov5
sudo -H pip3 install -U -r requirements.txt
python3 detect.py
In Windows 10 use:-
Code: Select all
pip install -U -r requirements.txt
python detect.py
For more pytorch options, see:-
https://pytorch.org/
It is also necessary to install git:-
There is a problem that can arise with xcb and the qt.qpa plugin. This can be solved by setting the environment in ~/.profile (at the very end usually):-
Re: YoloV5 recognition
Posted: Sun 30 Aug 30 2020 2:04 pm
by Daniel Wee
At the time of writing, CUDA 11 is out but I'm using CUDA 10.1 because Tensorflow 2.2.0 is only compatible with 10.1 (officially 10.1 but 10.2 is supposedly backward compatible.)
https://github.com/tensorflow/tensorflow/issues/38194
https://www.tensorflow.org/install/gpu
To download CUDA 10.2, use this link:-
https://developer.nvidia.com/cuda-10.2- ... e=exelocal
Re: YoloV5 recognition
Posted: Sun 30 Aug 30 2020 7:07 pm
by Daniel Wee
https://lionbridge.ai/articles/create-a ... ng-yolov5/
https://mlwhiz.com/blog/2020/08/08/yolov5/
Using Ubuntu to manage images from Open Images:-
Code: Select all
git clone https://github.com/EscVM/OIDv4_ToolKit
cd OIDv4_ToolKit
pip install -r requirements.txt
Download 1000 images of lizard class:-
Code: Select all
python3 main.py downloader --classes Lizard --type_csv all -y --limit 1000
This will create a directory OID and placed the downloaded files inside.
Open Images database appears to contain about 600 classes of objects.
https://storage.googleapis.com/openimag ... nload.html
After downloading, you will need a label manipulation tool such as HyperLabel which is only available for Mac and Windows. Unfortunately there seems to be a problem downloading this at the moment. The Microsoft Store App seems to be only accessible when using VPN.
Re: YoloV5 recognition
Posted: Sun 30 Aug 30 2020 8:51 pm
by Daniel Wee
Turns out that the lizards in Open Images are not house lizards so we will need to scrape this from Google Image search. To do this we have to rely on selenium which only supports Chrome up to version 76 but current version is up to version 84. This means we will have to rely on FireFox.
In order for this to work, we will need the geckodriver.exe from:-
https://github.com/mozilla/geckodriver/releases
This is placed in the same folder as the script, in this case Nextcloud/python
https://stackoverflow.com/questions/433 ... -in-python
Code is located in Nextcloud/python/googleimagescrape and images is inside the houselizard subfolder.
Re: YoloV5 recognition
Posted: Sun 30 Aug 30 2020 9:54 pm
by Daniel Wee
After using HyperLabel (on Windows) to label the images, export it in YOLO format to a directory. Inside that directory there is a "data" directory and inside that will be an "img" directory that contains the images and the labels. Copy this "img" directory to yolov5/training/
Inside training there is a script that will randomly separate the labelled data into train and test sets. Running this will copy images and labels from the "img" directory and place them inside of the yolov5/training/data/ training directory in the respective subdirectories.
Create the dataset.yaml according to the instructions, changing the classes to 1 and the name of the class to
"Lizard" (or whatever you used in HyperLabel).
Copy the model over as per instructions and edit the number of categories to 1. (see instructions)
Run the training as follows from the yolov5 directory:-
Code: Select all
python train.py --img 640 --batch 16 --epochs 300 --data training/dataset.yaml --cfg training/yolov5l.yaml --weights ''
Note that this is different from the instructions which contains an error in spacing (300--data) and has a batch size that is too large for the GPU. This has been reduced to batch size of 2 in order for it to run in some cases. It seems that if that happens,
you need to restart the machine to allow a batch size of 16.
Re: YoloV5 recognition
Posted: Sun 30 Aug 30 2020 10:25 pm
by Daniel Wee
To monitor progress, open a second terminal and navigate to the yolov5 directory and run:-
Open a browser to
https://localhost:6006 to view the progress.
WARNING: DO NOT RUN TENSORBOARD WHILE TRAINING.
- the training results could not be written to file because tensorboard had a lock on it and this only happened after a 45-minute training session!!!
Re: YoloV5 recognition
Posted: Mon 31 Aug 31 2020 10:57 am
by Daniel Wee
After successful training - the results seem to be stored in yolov5/runs/exp
nn/weights/best.pt
I would copy this result to the yolov5/weights directory for use in inference testing for convenience. Unfortunately at the time of writing, although the training seems to complete, the best.pt file does not appear to be working or producing results. This might be due to an error in dataset.yaml where I used "lizard" rather than "Lizard" as the class name as per HyperLabel annotation. I will try to correct this and try again.
Changes made but still no result. When the detection is run, yolov5s.pt provides some detection but not best.pt from the runs/exp10/weights directory. I'm not sure if this is the correct way to get the resultant training weights but there does not appear to be other alternatives at the moment.
I think we have to look deeper into the training scheme for YOLOv5:-
https://blog.roboflow.com/how-to-train- ... m-dataset/
This suggests other labelling utilities apart from HyperLabel, namely LabelImg (Mac) and CVAT (http hosted).
https://blog.roboflow.com/getting-start ... tion-data/
https://blog.roboflow.com/getting-started-with-cvat/
Re: YoloV5 recognition
Posted: Mon 31 Aug 31 2020 2:14 pm
by Daniel Wee
Detection on video is performed with the following command:-
Code: Select all
python detect.py --weights runs/exp13/weights/best.pt --source inference/videos/lizard1.mp4 --view-img --output inference/output
At this point we have completed training successfully with batchsize 16 and 300 epochs but cannot seem to get detection working properly compared to using yolov5x.pt as weight, for example:-
Code: Select all
python detect.py --weights weights/yolov5x.pt --source inference/videos/lizard1.mp4 --view-img --output inference/output
Single image detection (by placing source image in inference/images) does not work either with the weights generated in runs/exp13/weights/best.pt
Re: YoloV5 recognition
Posted: Wed 02 Sep 02 2020 12:31 pm
by Daniel Wee
Comparing our training images to the Open Images downloads, the OI images all seem to have a width of 1024-pixels whereas the ones we downloaded from Google appear to be much smaller. I'm not sure if this is impacting the training. Perhaps I should try narrowing it down to higher resolution images?
The training command includes an --img parameter that specifies image size. It is unclear how this exactly works but the example uses 640 as the size. In reducing this to 480 the latest BingGoogle training set came down to 34 images and 35 targets. This might be a discarding of images below specified size. Many of Google's images are at 220 so that might be worth trying out.
Code: Select all
python train.py --img 480 --batch 16 --epochs 300 --data training/dataset.yaml --cfg training/yolov5l.yaml --weights ''
Reducing --img to 220 does generate more training images, to 59 and targets to 61.
In either case, the detection/inference is still not working with the resultant best.pt
Re: YoloV5 recognition
Posted: Wed 02 Sep 02 2020 6:42 pm
by Daniel Wee
More explanation of the training parameters:-
https://blog.francium.tech/custom-objec ... db513ecefb
Code: Select all
python train.py --img 480 --batch 16 --epochs 300 --data training/dataset.yaml --cfg training/yolov5l.yaml --weights ''
The --multi-scale option results in GPU errors.
More training details here:-
https://github.com/ultralytics/yolov5/w ... ustom-Data
The --weights allow you to specify a pre-trained model to start from or use '' for randomly initialized weights.
We are now using Colab to train using the local set of data but placed into a different configuration to match what Roboflow generated. The best.pt from the previous training was exported to Google drive and downloaded and work fairly well but only with large images of lizards. We want to be able to detect small images of lizards.