Improve a CMS’s photos library qualification with AI, Object Detection from images in Python with OpenCv and YOLO

by Posted on

Still digging, in Python, on more versatile usage in object detection from images to improve a house-made CMS multimedia searches. What a sentence! So, maybe, you have heard about the acronym YOLO one stands for “You Only Live Once”, a kind of modern version of the Latin “Carpe Diem”, the other YOLO stands for “You Only Look Once”.

So, let’s give a quick definition of the second YOLO thing and it’s all about image recognition.

All the files can be found on my github account at https://github.com/bflaven/BlogArticlesExamples/tree/master/python_playing_with_object_detection_opencv_yolo

What is this other YOLO thing?

YOLO is an extremely fast multi object detection algorithm which uses convolutional neural network (CNN) to detect and identify objects. Maybe less sexy than the first definition but pretty useful when like me you try to increase the images library relevance categorization…

At least, I also like the name of the library YOLO. I was figured out a funny dialog between developers mimicking pseudo rappers “What kinda python lib do you like, Bro!”, “YOLO, man! Totally. Need to check it out NOW!” 🙂

Anyway, let’s expose my objective.

To make better image categorization for a CMS

I have already exposed the issue, I am tackling with it already in a previous post dedicated to facial recognition.

If you ever read that post’s blog, I made few attempts on facial recognition with people faces mostly presidents. On a very European centric point of view, presidents can be range from known, less know to totally unknow!

Unfortunately, it is OK for human faces but not for any other element inside pictures to be recognized e.g. cars, buildings, dog, horse, human, tree…. whatever.

Well, as I am consistent or stubborn, my objective is still the same as a Backoffice’s PO: the multimedia library carries thousand of images that are uncorrectly tagged: no caption, no keywords… etc So, performing searches on that dataset is problematic. To be even more straightforward, I will answer “GIGO”, the famous “Garbage In, Garbage Out”. GIGO produces search results that suck! So how can the situation be improved, one possibility is to enhance image search with a fine-grained keywords attached to each picture but without making the job by humans. This is where you’d better call IA to the rescue.

Like always that is very practical issue, I won’t give you any explanations about the YOLO algorithm and what is “behind the curtain” with matrix and formula stuff both because it does not really interest me, I am user first and also because I am just no able to do so! No, I will focus on results !

Set-up your environnement

At the very beginning, I was looking for an example using TensorFlow Image Recognition but I decide to go for YOLO. I found it much easier to implement. Let’s see quickly what you need to play with YOLO.

Python librairies
First, I am using my mac and a remote disk to install all the required libraries with the help of anaconda, to save space disk. In addition to the principal Python libraries used in the scripts that are easy to install with pip or anaconda.

# YOLO object detection requirements
import cv2 as cv
import numpy as np
import time
 
# to parse the directory
import os
 
# to import in mysql
import mysql.connector as mysql

I just advise to use this command to install opencv.

# to install cv2 or opencv use this command
pip install opencv-python
 
# this command is not working
pip install opencv

You will have to download the YOLO files.

Got the look YOLO!

So easy to get YOLO, it requires 3 files to download:

  1. File_1 : In order to run the network, you will have to download the pre-trained YOLO weight file (237 MB). Download yolov3.weights
  2. File_2 : You have also download the the YOLO configuration file: yolov3.cfg – Download yolov3.cfg
  3. File_3 : Get the 80 COCO class names: coco.names – Download coco.names

You are done! You have the requirements to use YOLO.

Source: https://opencv-tutorial.readthedocs.io/en/latest/yolo/yolo.html

Few commands to use MySQL

I suppose that you have installed MySQL on your computer. I always forget the main commands when I am in the console so here are quick reminders.
For more explanations, you can check the previous post on facial recognition.

# to connect to the db
mysql -u root -p
# commands SELECT AND USE db
SHOW DATABASES;
USE yolo_images;
 
# commands SELECT AND USE db
SHOW TABLES;
# DESCRIBE yolo_advanced_images
DESCRIBE yolo_advanced_images;
 
 
# SELECT a dump .sql
SOURCE /Users/brunoflaven/Documents/01_work/blog_articles/tensorflow_image_recognition/mysql_yolo_advanced_images_1.sql
 
# SELECT ALL records
SELECT COUNT(*) FROM yolo_advanced_images;
 
# SELECT ALL records
SELECT * FROM yolo_advanced_images;
 
# DELETE single image WITH the help OF the id
DELETE FROM yolo_advanced_images WHERE id = '96';	
DELETE FROM yolo_advanced_images WHERE id = '97';
 
 
# empty TABLE
TRUNCATE TABLE yolo_advanced_images;

The table structure `yolo_advanced_images` for databas yolo_images

DROP TABLE IF EXISTS `yolo_advanced_images`;
 
CREATE TABLE `yolo_advanced_images` (
  `id` BIGINT UNSIGNED NOT NULL AUTO_INCREMENT,
  `file` VARCHAR(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_520_ci NOT NULL DEFAULT '',
`keywords` VARCHAR
(255) CHARACTER
SET utf8mb4
COLLATE utf8mb4_unicode_520_ci NOT NULL DEFAULT '',
  PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci;
 
LOCK TABLES `yolo_advanced_images` WRITE;
 
INSERT INTO `yolo_advanced_images` (`id`, `file`)
VALUES
	(NULL,'image1.jpg', 'keyword_1_img_1'),
	(NULL,'image1.jpg', 'keyword_2_img_1'),
	(NULL,'image2.jpg', 'keyword_1_img_2'),
	(NULL,'image3.jpg', 'keyword_1_img_3' );

Conclusion: I have made also a translation in Spanish of the coco.names. You can just download the file coco_es.names from my github account. Meaning that I can tag in any language (English, Spanish, Russian, Portuguese…) images with the help of YOLO. So, for instance, I can provide a better image search in any language by providing a native language tagging system for the same dataset of images.
After, that, you can even improve the tagging system by coupling it with facial recognition. Using the principles detailed in my previous post on facial recognition, you’ll be able to precise who are the persons on pictures if it is a president, an NBA star… whatever the known faces images you drop in your sample aka the directory of know faces! So, that can inject proper noun like Macron, Trump, Bryan, Biden… in addition to the keyword created by YOLO for “person”. Then you are good! By injecting these new tags into the real multimedia database used in the CMS, you may considerably improve users’ images searches and never felt anymore so miserable!

Image copyright is www.vexels.com

Read more

Published by Bruno Flaven

By now, Bruno Flaven works as AI Coordinator. He has been a project manager in a wide variety of Internet business applications both in Mobile and in Desktop for 20 years now. You can find more information about his professional life on his personal website (www.flaven.fr) or his linkedin profile: https://www.linkedin.com/in/brunoflaven. He is currently working In France Media Monde (FMM) mostly on mobile applications (iOS and Android). He has also worked as P.O for a Backoffice project made with Symfony. He also made few trainings to facilitate the handling of the tools that he helps to make.

Related Posts