PersonDetectionNopMicroPython < IoT_Course

Tags: view all tags

---+ Person detection with!MicroPython [[MicroPython][<br />]]

Even though the final goal of this demo is to get images from a camera and to decide on the spot if there is a person in sight, the first step is to test the model with well known test images. We use a pre-trained and quantized model. Re-creating the model and training it is non-trivial because the model provided has been built with an outdated version of !TensorFlow. It would be an interesting task to build a new Keras model and train it.

You can find an example of the person detection demo in the sources of TFLM and a different version for the esp-idf and the Arduino SDK. For !MicroPython I tried to port both versions and compared the results with their C++ originals.
---++ The TFLM version

Here only 2 test images are used: a person and a giraffe:
| <img alt="person.png" height="96" src="%ATTACHURL%/person.png" title="person.png" width="96" /> | <img alt="no_person.png" height="96" src="%ATTACHURL%/no_person.png" title="no_person.png" width="96" /> |
| person | no person |
The result from the invocation of the model returns a score value of -128 .. 127, which can be converted to a probability value through de-quantization. The microlite module provides the quantizeInt8ToFloat which accomplishes this task.

Here is the result from the !MicroPython program:

<img alt="mp_person_detect.png" height="253" src="%ATTACHURL%/mp_person_detect.png" title="mp_person_detect.png" width="661" />

As you can see, the results are correct and the confidence levels are pretty high.
---++ The esp-idf and Arduino version of the person detection model

These versions are pretty complex and they incorporate
   * inference on test images
   * camera readout
   * display on a TFT screen
all in a single multi-tasking program. As the hardware used is different from my boards I decided to disentangle these components and to provide a separate program for the use of test images and the use of the camera. The treatment of the test images can be done on both CPU boards, while the camera version is of course on possible with the !FreeNove board incorporating the OV2640 camera.

Here are the test images used in the example:
| <img alt="image0.png" height="96" src="%ATTACHURL%/image0.png" title="image0.png" width="96" /> | <img alt="image1.png" height="96" src="%ATTACHURL%/image1.png" title="image1.png" width="96" /> | <img alt="image2.png" height="96" src="%ATTACHURL%/image2.png" title="image2.png" width="96" /> | <img alt="image3.png" height="96" src="%ATTACHURL%/image3.png" title="image3.png" width="96" /> | <img alt="image4.png" height="96" src="%ATTACHURL%/image4.png" title="image4.png" width="96" /> |
| image 0 | image 1 | image 2 | image 3 | image 4 |
| <img alt="image5.png" height="96" src="%ATTACHURL%/image5.png" title="image5.png" width="96" /> | <img alt="image6.png" height="96" src="%ATTACHURL%/image6.png" title="image6.png" width="96" /> | <img alt="image7.png" height="96" src="%ATTACHURL%/image7.png" title="image7.png" width="96" /> | <img alt="image8.png" height="96" src="%ATTACHURL%/image8.png" title="image8.png" width="96" /> | <img alt="image9.png" height="96" src="%ATTACHURL%/image9.png" title="image9.png" width="96" /> |
| image 5 | image 6 | image 7 | image 8 | image 9 |
The images are provided as binary files with 96x96 uint8_t gray scale pixel values. Th visualize them I wrote a short Python script (show-test-image.py) and another one (pixel2png.py) to convert the binary files into png images.

Before passing the pixels into the input tensor for the model invocation the pixel values must be converted from uint8_t to int8_t expected be the model.

<img alt="person_detect_10imgs.png" height="270" src="%ATTACHURL%/person_detect_10imgs.png" title="person_detect_10imgs.png" width="1162" />

The program supplies its results in the above summary table. You may judge yourself, how well the person detection works. It seems that after all the difference of a person and a monkey is not that great!

-- %USERSIG{UliRaich - 2023-11-08}%

---++ Comments

<br />%COMMENT%

Attachments

Topic attachments
I	Attachment	History	Action	Size	Date	Who
png	image0.png	r1	manage	4.3 K	2023-11-08 - 20:52	UliRaich
png	image1.png	r1	manage	5.1 K	2023-11-08 - 20:52	UliRaich
png	image2.png	r1	manage	4.4 K	2023-11-08 - 20:52	UliRaich
png	image3.png	r1	manage	5.4 K	2023-11-08 - 20:52	UliRaich
png	image4.png	r1	manage	4.9 K	2023-11-08 - 20:52	UliRaich
png	image5.png	r1	manage	7.3 K	2023-11-08 - 20:53	UliRaich
png	image6.png	r1	manage	3.5 K	2023-11-08 - 20:53	UliRaich
png	image7.png	r1	manage	6.6 K	2023-11-08 - 20:53	UliRaich
png	image8.png	r1	manage	4.9 K	2023-11-08 - 20:53	UliRaich
png	image9.png	r1	manage	3.6 K	2023-11-08 - 20:53	UliRaich
png	mp_person_detect.png	r1	manage	38.6 K	2023-11-08 - 20:46	UliRaich
png	no_person.png	r1	manage	6.8 K	2023-11-08 - 20:26	UliRaich
png	person.png	r2 r1	manage	5.7 K	2023-11-08 - 20:26	UliRaich
png	person_detect_10imgs.png	r1	manage	45.1 K	2023-11-08 - 21:04	UliRaich

Topic revision: r1 - 2023-11-08 - UliRaich