This is an app that I have written for my AI & Robotics course to bridge the gap between the Convolutional Neural Network model generated by Teachable Machine with an Arduino microcontroller via Serial communication. Effectively, this add computer vision AI capability for the Arduino by using the laptop’s webcam and processing power to run the model.
- Develop a model in Teachable Machine (TM). Make sure to name your classes properly before you train / export. Else, the old name will stuck embedded in the model for some odd reason and you will have to start a new TM project for the new classs name to take effect.
- Download the Teachable Machine to Serial App, it is a zip folder, you must extract it.
- Export the TM Keras model to a file named converted_keras.zip, and place it inside the extracted folder
where the executable file is (ie TeachableMachine2Serial.exe). Very importantly, the name must be exact. If you retrain the model and download again, by default, it will name the file something else. You have to make sure the name is converted_keras.zip, the app is not smart enough to handle file selection yet. - Make sure no other software is using your webcam (ie Teachable Machine). Close it.
- Plug in your Arduino and upload the microcontroller code. In the demo, it is programmed to handle a model with labels Class 1, Class 2, and Class 3. Modify for your use case.
- Run TeachableMachine2Serial.exe, it will extract the zip folder, then ask you for a Serial port to send message to. This is the Serial port that your Arduino is connected to. After you select a port, the program will load the TensorFlow model, start capturing images from the webcam, run inference with the model and send the class name via Serial.
- You can debug by opening the preview window, it will show what class the model detected.
Since the software was intented to use internally, it currently only support Windows and lack a GUI. Maybe in the future if I have time, I will develop a GUI version to make it more user friendly.
The video above is in Cantonese, as I was pitching this app internally. I will record an English version eventually. Leave a comment and give a like if this would be helpful!