{"id":1622,"date":"2020-01-31T16:34:13","date_gmt":"2020-01-31T16:34:13","guid":{"rendered":"https:\/\/www.danielparente.net\/en\/2020\/01\/31\/create-a-real-time-object-tracking-camera-with-tensorflow-and-raspberry-pi\/"},"modified":"2020-02-01T22:16:43","modified_gmt":"2020-02-01T22:16:43","slug":"create-a-real-time-object-tracking-camera-with-tensorflow-and-raspberry-pi","status":"publish","type":"post","link":"https:\/\/www.danielparente.net\/en\/2020\/01\/31\/create-a-real-time-object-tracking-camera-with-tensorflow-and-raspberry-pi\/","title":{"rendered":"Create a real-time object tracking camera with TensorFlow and Raspberry Pi"},"content":{"rendered":"<p>[ad_1]<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/opensource.com\/sites\/default\/files\/lead-images\/raspberrypi_board_vector_red.png\" \/><\/p>\n<div>\n<p>Are you just getting started with machine\/deep learning, TensorFlow, or Raspberry Pi?<\/p>\n<p>I created <a href=\"https:\/\/github.com\/leigh-johnson\/rpi-deep-pantilt\" target=\"_blank\" rel=\"noopener noreferrer\">rpi-deep-pantilt<\/a> as an interactive demo of object detection in the wild, and in this article,\u00a0I&#8217;ll show you how to reproduce the video below, which depicts a camera panning and tilting to track my movement across a room.<\/p>\n<blockquote class=\"twitter-tweet\">\n<p dir=\"ltr\" lang=\"en\" xml:lang=\"en\">I&#8217;m just a girl, standing in front of a tiny computer, reminding you most computing problems can be solved by sheer force of will. \ud83d\udcaa<\/p>\n<p>MobileNetv3 + SSD <a href=\"https:\/\/twitter.com\/TensorFlow?ref_src=twsrc%5Etfw\" target=\"_blank\" rel=\"noopener\">@TensorFlow<\/a> model I converted <a href=\"https:\/\/twitter.com\/hashtag\/TFLite?src=hash&amp;ref_src=twsrc%5Etfw\" target=\"_blank\" rel=\"noopener\">#TFLite<\/a><a href=\"https:\/\/twitter.com\/hashtag\/RaspberryPi?src=hash&amp;ref_src=twsrc%5Etfw\" target=\"_blank\" rel=\"noopener\">#RaspberryPi<\/a> + <a href=\"https:\/\/twitter.com\/pimoroni?ref_src=twsrc%5Etfw\" target=\"_blank\" rel=\"noopener\">@pimoroni<\/a> pantilt hat, PID controller.<\/p>\n<p>Write-up soon! \u2728 <a href=\"https:\/\/t.co\/v63KSJtJHO\">https:\/\/t.co\/v63KSJtJHO<\/a> <a href=\"https:\/\/t.co\/dmyAlWCnWk\">pic.twitter.com\/dmyAlWCnWk<\/a><\/p>\n<p>\u2014 Leigh (@grepLeigh) <a href=\"https:\/\/twitter.com\/grepLeigh\/status\/1199910608474263552?ref_src=twsrc%5Etfw\" target=\"_blank\" rel=\"noopener\">November 28, 2019<\/a><\/p><\/blockquote>\n<p>This article will cover:<\/p>\n<ol>\n<li>Build materials and hardware assembly instructions.<\/li>\n<li>Deploying a <strong>TensorFlow Lite<\/strong> object-detection model (<strong>MobileNetV3-SSD<\/strong>) to a <strong>Raspberry Pi.<\/strong><\/li>\n<li>Sending tracking instructions to pan\/tilt servo motors using a <strong>proportional\u2013integral\u2013derivative (PID) controller.<\/strong><\/li>\n<li>Accelerating inferences of any <strong>TensorFlow Lite<\/strong> model with <strong>Coral&#8217;s USB Edge TPU Accelerator<\/strong> and <strong>Edge TPU Compiler<\/strong>.<\/li>\n<\/ol>\n<h2 id=\"terms-and-references\">Terms and references<\/h2>\n<ul>\n<li><strong><a href=\"https:\/\/www.raspberrypi.org\/\" target=\"_blank\" rel=\"noopener noreferrer\">Raspberry Pi<\/a>:<\/strong> A small, affordable computer popular with educators, hardware hobbyists, and robot enthusiasts.<\/li>\n<li><a href=\"https:\/\/www.raspberrypi.org\/downloads\/raspbian\/\" target=\"_blank\" rel=\"noopener noreferrer\"><strong>Raspbian<\/strong><\/a><strong>:<\/strong> The Raspberry Pi Foundation&#8217;s official operating system for the Pi.\u00a0Raspbian is derived from Debian Linux.<\/li>\n<li><a href=\"https:\/\/www.tensorflow.org\/\" target=\"_blank\" rel=\"noopener noreferrer\"><strong>TensorFlow<\/strong><\/a><strong>:<\/strong> An open source framework for <a href=\"https:\/\/en.wikipedia.org\/wiki\/Dataflow_programming\" target=\"_blank\" rel=\"noopener noreferrer\">dataflow<\/a> programming used for machine learning and deep neural learning.<\/li>\n<li><a href=\"https:\/\/www.tensorflow.org\/lite\" target=\"_blank\" rel=\"noopener noreferrer\"><strong>TensorFlow Lite<\/strong><\/a><strong>:<\/strong> An open source framework for deploying TensorFlow models on mobile and embedded devices.<\/li>\n<li><strong><a href=\"https:\/\/towardsdatascience.com\/portable-computer-vision-tensorflow-2-0-on-a-raspberry-pi-part-1-of-2-84e318798ce9\" target=\"_blank\" rel=\"noopener noreferrer\">Convolutional neural network<\/a>:<\/strong> CNN is a\u00a0type of neural network architecture that is well-suited for image classification and object detection tasks.<\/li>\n<li><strong><a href=\"https:\/\/towardsdatascience.com\/review-ssd-single-shot-detector-object-detection-851a94607d11\" target=\"_blank\" rel=\"noopener noreferrer\">Single-shot detector<\/a>:<\/strong> SSD is a type of CNN architecture\u00a0specialized for real-time object detection, classification, and bounding box localization.<\/li>\n<li><a href=\"https:\/\/ai.googleblog.com\/2018\/04\/mobilenetv2-next-generation-of-on.html\" target=\"_blank\" rel=\"noopener noreferrer\"><strong>MobileNetV3<\/strong><\/a><strong>:<\/strong> A state-of-the-art computer vision model optimized for performance on modest mobile phone processors.<\/li>\n<li><a href=\"https:\/\/github.com\/tensorflow\/models\/tree\/master\/research\/object_detection#nov-13th-2019\" target=\"_blank\" rel=\"noopener noreferrer\"><strong>MobileNetV3-SSD<\/strong><\/a><strong>:<\/strong> An SSD based on MobileNet architecture. This tutorial will use MobileNetV3-SSD models available through <a href=\"https:\/\/github.com\/tensorflow\/models\/blob\/master\/research\/object_detection\/g3doc\/detection_model_zoo.md\" target=\"_blank\" rel=\"noopener noreferrer\">TensorFlow&#8217;s object-detection model zoo<\/a>.<\/li>\n<li><a href=\"https:\/\/cloud.google.com\/edge-tpu\/\" target=\"_blank\" rel=\"noopener noreferrer\"><strong>Edge TPU<\/strong><\/a><strong>:<\/strong> a tensor processing unit (TPU) is an integrated circuit for accelerating computations performed by TensorFlow. The Edge TPU was developed with a small footprint for mobile and embedded devices &#8220;at the edge.&#8221;<\/li>\n<\/ul>\n<p class=\"rtecenter\"><sup>Cloud TPUs (left and center) accelerate TensorFlow model training and inference. Edge TPUs (right) accelerate inferences in mobile devices.<\/sup><\/p>\n<h2 id=\"build-list\">Build list<\/h2>\n<h3 id=\"essential\">Essential<\/h3>\n<h3 id=\"optional\">Optional<\/h3>\n<blockquote><p><em>Looking for a project with fewer moving pieces? Check out <a href=\"https:\/\/towardsdatascience.com\/portable-computer-vision-tensorflow-2-0-on-a-raspberry-pi-part-1-of-2-84e318798ce9\" target=\"_blank\" rel=\"noopener noreferrer\">Portable Computer Vision: TensorFlow 2.0 on a Raspberry Pi<\/a> to create a hand-held image classifier.<\/em><\/p><\/blockquote>\n<h2 id=\"set-up-the-raspberry-pi\">Set up the Raspberry Pi<\/h2>\n<p>There are two ways you can install Raspbian to your MicroSD card:<\/p>\n<ol>\n<li><a href=\"https:\/\/www.raspberrypi.org\/documentation\/installation\/noobs.md\" target=\"_blank\" rel=\"noopener noreferrer\">NOOBS<\/a> (&#8220;New Out Of Box Software&#8221;) is a GUI operating system installation manager. If this is your first Raspberry Pi project, I&#8217;d recommend starting here.<\/li>\n<li><a href=\"https:\/\/www.raspberrypi.org\/documentation\/installation\/installing-images\/README.md\" target=\"_blank\" rel=\"noopener noreferrer\">Write the Raspbian image to an SD card<\/a>.<\/li>\n<\/ol>\n<p>This tutorial and supporting software were written using <a href=\"https:\/\/www.raspberrypi.org\/documentation\/installation\/\" target=\"_blank\" rel=\"noopener noreferrer\">Raspbian (Buster)<\/a>. If you&#8217;re using a different version of Raspbian or another platform, you&#8217;ll probably experience some pains.<\/p>\n<p>Before proceeding, you&#8217;ll need to:<\/p>\n<h2 id=\"install-software\">Install software<\/h2>\n<ol>\n<li>Install system dependencies:\n<pre><span class=\"geshifilter\"><code class=\"bash geshifilter-bash\"><span class=\"co4\">$ <\/span><span class=\"kw2\">sudo<\/span> <span class=\"kw2\">apt-get update<\/span> <span class=\"sy0\">&amp;&amp;<\/span> <span class=\"kw2\">sudo<\/span> <span class=\"kw2\">apt-get install<\/span> <span class=\"re5\">-y<\/span> python3-dev libjpeg-dev libatlas-base-dev raspi-gpio libhdf5-dev python3-smbus<\/code><\/span><\/pre>\n<\/li>\n<li>Create a new project directory:\n<pre><span class=\"geshifilter\"><code class=\"bash geshifilter-bash\"><span class=\"co4\">$ <\/span><span class=\"kw2\">mkdir<\/span> rpi-deep-pantilt <span class=\"sy0\">&amp;&amp;<\/span> <span class=\"kw3\">cd<\/span> rpi-deep-pantilt<\/code><\/span><\/pre>\n<\/li>\n<li>Create a new virtual environment:\n<pre><span class=\"geshifilter\"><code class=\"bash geshifilter-bash\"><span class=\"co4\">$ <\/span>python3 <span class=\"re5\">-m<\/span> venv .venv<\/code><\/span><\/pre>\n<\/li>\n<li>Activate the virtual environment:\n<pre><span class=\"geshifilter\"><code class=\"bash geshifilter-bash\"><span class=\"co4\">$ <\/span><span class=\"kw3\">source<\/span> .venv<span class=\"sy0\">\/<\/span>bin<span class=\"sy0\">\/<\/span>activate <span class=\"sy0\">&amp;&amp;<\/span> python3 <span class=\"re5\">-m<\/span> pip <span class=\"kw2\">install<\/span> <span class=\"re5\">--upgrade<\/span> pip<\/code><\/span><\/pre>\n<\/li>\n<li>Install TensorFlow 2.0 from a community-built wheel:\n<pre><span class=\"geshifilter\"><code class=\"bash geshifilter-bash\"><span class=\"co4\">$ <\/span>pip <span class=\"kw2\">install<\/span> https:<span class=\"sy0\">\/\/<\/span>github.com<span class=\"sy0\">\/<\/span>leigh-johnson<span class=\"sy0\">\/<\/span>Tensorflow-bin<span class=\"sy0\">\/<\/span>blob<span class=\"sy0\">\/<\/span>master<span class=\"sy0\">\/<\/span>tensorflow-2.0.0-cp37-cp37m-linux_armv7l.whl?<span class=\"re2\">raw<\/span>=<span class=\"kw2\">true<\/span><\/code><\/span><\/pre>\n<\/li>\n<li>Install the <strong>rpi-deep-pantilt<\/strong> Python package:\n<pre><span class=\"geshifilter\"><code class=\"bash geshifilter-bash\"><span class=\"co4\">$ <\/span>python3 <span class=\"re5\">-m<\/span> pip <span class=\"kw2\">install<\/span> rpi-deep-pantilt<\/code><\/span><\/pre>\n<\/li>\n<\/ol>\n<\/div>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p><a href=\"https:\/\/opensource.com\/article\/20\/1\/object-tracking-camera-raspberry-pi?sc_cid=70160000001273HAAQ\" target=\"_blank\" rel=\"noopener\">Read the full article by Leigh Johnson Feed <\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>[ad_1] Are you just getting started with machine\/deep learning, TensorFlow, or Raspberry Pi? I created rpi-deep-pantilt as an interactive demo of object detection in the wild, and in this article,\u00a0I&#8217;ll show you how to reproduce the video below, which depicts a camera panning and tilting to track my movement across a room. I&#8217;m just a [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":1623,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":"","jetpack_post_was_ever_published":false},"categories":[94,92,98],"tags":[],"class_list":["post-1622","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-artificial-intelligence","category-data-science","category-machine-learning"],"blocksy_meta":[],"jetpack_featured_media_url":"https:\/\/e928cfdc7rs.exactdn.com\/info\/uploads\/sites\/3\/2020\/01\/Create-a-real-time-object-tracking-camera-with-TensorFlow-and-Raspberry.png?strip=all","jetpack_shortlink":"https:\/\/wp.me\/p2TFCd-qa","jetpack_sharing_enabled":true,"jetpack-related-posts":[],"_links":{"self":[{"href":"https:\/\/www.danielparente.net\/en\/wp-json\/wp\/v2\/posts\/1622","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.danielparente.net\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.danielparente.net\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.danielparente.net\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.danielparente.net\/en\/wp-json\/wp\/v2\/comments?post=1622"}],"version-history":[{"count":1,"href":"https:\/\/www.danielparente.net\/en\/wp-json\/wp\/v2\/posts\/1622\/revisions"}],"predecessor-version":[{"id":1701,"href":"https:\/\/www.danielparente.net\/en\/wp-json\/wp\/v2\/posts\/1622\/revisions\/1701"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.danielparente.net\/en\/wp-json\/wp\/v2\/media\/1623"}],"wp:attachment":[{"href":"https:\/\/www.danielparente.net\/en\/wp-json\/wp\/v2\/media?parent=1622"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.danielparente.net\/en\/wp-json\/wp\/v2\/categories?post=1622"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.danielparente.net\/en\/wp-json\/wp\/v2\/tags?post=1622"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}