Ricardo Zamudio
4 years ago
committed by
GitHub
commit
3443836963
@ -0,0 +1,16 @@
|
||||
# Memulai dengan IoT
|
||||
|
||||
Pada bagian ini, Anda akan diperkenalkan dengan Internet of Things, dan mempelajari konsep dasar termasuk membangung proyek IoT 'Hello World' pertama Anda yang terhubung ke *cloud*. Proyek ini merupakan lampu malam yang akan menyala saat tingkat pencahayaan diukur dengan penurunan sensor. This project is a nightlight that lights up as light levels measured by a sensor drop.
|
||||
|
||||
|
||||
|
||||
## Topik
|
||||
|
||||
1. [Pengenalan IoT](lessons/1-introduction-to-iot/README.md)
|
||||
2. [Lebih dalam dengan IoT](lessons/2-deeper-dive/README.md)
|
||||
3. [Berinteraksi dengan dunia menggunakan sensor dan aktuator](lessons/3-sensors-and-actuators/README.md)
|
||||
4. [Menghubungkan perangkat Anda ke Internet](lessons/4-connect-internet/README.md)
|
||||
|
||||
## Kredit
|
||||
|
||||
Semua pelajaran ditulis dengan ♥️ oleh [Jim Bennett](https://GitHub.com/JimBobBennett)
|
||||
@ -0,0 +1,97 @@
|
||||
# Pengenalan IoT
|
||||
|
||||
|
||||
|
||||
> Sketsa dibuat oleh [Nitya Narasimhan](https://github.com/nitya). Klik gambar untuk versi yang lebih besar.
|
||||
|
||||
## Kuis prakuliah
|
||||
|
||||
[Kuis prakuliah](https://brave-island-0b7c7f50f.azurestaticapps.net/quiz/1)
|
||||
|
||||
## Pengantar
|
||||
|
||||
Pelajaran ini mencakup beberapa topok pengantar mengenai Internet of Things, dan membuat Anda dapat mempersiapkan dan mengatur perangkat keras Anda.
|
||||
|
||||
Dalam pelajaran ini kita akan membahas:
|
||||
|
||||
* [Apa itu 'Internet of Things'?](#apa-itu-internet-of-things)
|
||||
* [Perangkat IoT](#perangkat-iot)
|
||||
* [Mengatur Perangkat Anda](#set-up-your-device)
|
||||
* [Penerapan dari IoT](#applications-of-iot)
|
||||
* [Contoh Perangkat IoT yang Mungkin Anda Punya di Sekitar](#examples-of-iot-devices-you-may-have-around-you)
|
||||
|
||||
## Apa itu 'Internet of Things'?
|
||||
|
||||
Istilah 'Internet of Things' diciptakan oleh [Kevin Ashton](https://wikipedia.org/wiki/Kevin_Ashton) pada tahun 1999, yang merujuk pada menghubungkan Internet ke dunia fisik melalui sensor. Sejak saat itu, istilah IoT digunakan untuk menggambarkan perangkat apa pun yang berinteraksi dengan dunia fisik di sekitarnya, baik dengan mengumpulkan data dari sensor, atau menyediakan interaksi dunia nyata melalui aktuator (perangkat yang melakukan sesuatu seperti menyalakan sakelar atau menyalakan LED), dan terhubung ke perangkat lain atau Internet.
|
||||
|
||||
> **Sensor** mengumpulkan informasi dari lingkungan, seperti mengukur kecepatan, suhu, atau lokasi.
|
||||
>
|
||||
> **Aktuator** mengubah sinyal listrik menjadi interaksi pada lingkungan seperti memicu sakelar, menyalakan lampu, membuat suara, atau mengirim *control signal* ke perangkat keras lain, misalnya untuk menyalakan soket listrik.
|
||||
|
||||
IoT sebagai suatu bidang teknologi lebih dari sekadar perangkat. Hal ini mencakup layanan berbasis cloud yang dapat memproses data sensor, atau mengirim permintaan ke aktuator yang terhubung ke perangkat IoT. IoT juga mencakup perangkat yang tidak memiliki atau tidak memerlukan konektivitas Internet, sering disebut sebagai *edge devices* atau perangkat tepi. Perangkat tepi adalah perangkat yang dapat memproses dan merespons data sensor itu sendiri, biasanya menggunakan model AI yang dilatih di cloud.
|
||||
|
||||
IoT merupakan bidang teknologi yang berkembang pesat. Diperkirakan pada akhir tahun 2020, 30 miliar perangkat IoT dikerahkan dan terhubung ke Internet. Jika melihat ke masa depan, diperkirakan pada tahun 2025, perangkat IoT akan mengumpulkan hampir 80 zettabytes data atau 80 triliun gigabyte. Banyak sekali bukan?
|
||||
|
||||
|
||||
|
||||
✅ Lakukan sedikit riset: Berapa banyak data yang dihasilkan oleh perangkat IoT yang benar-benar digunakan, dan berapa banyak yang terbuang? Mengapa begitu banyak data yang diabaikan?
|
||||
|
||||
Data ini adalah kunci kesuksesan IoT. Untuk menjadi pengembang IoT yang sukses, Anda perlu memahami data yang perlu Anda kumpulkan, cara mengumpulkannya, cara membuat keputusan berdasarkan data tersebut, dan cara menggunakan keputusan tersebut untuk berinteraksi dengan lingkungan fisik jika diperlukan.
|
||||
|
||||
## Perangkat IoT
|
||||
|
||||
Huruf **T** di IoT adalah singkatan dari **Things** - perangkat yang berinteraksi dengan lingkungan fisik di sekitarnya baik dengan mengumpulkan data dari sensor atau menyediakan interaksi dunia nyata melalui aktuator.
|
||||
|
||||
Perangkat untuk produksi atau penggunaan komersial, seperti pelacak kebugaran konsumen, atau pengontrol mesin industri, biasanya dibuat khusus. Mereka menggunakan papan sirkuit khusus, bahkan mungkin prosesor khusus, yang dirancang untuk memenuhi kebutuhan tugas tertentu, apakah itu cukup kecil untuk muat di pergelangan tangan, atau cukup kuat untuk bekerja di lingkungan pabrik dengan suhu tinggi, stres tinggi, atau getaran tinggi.
|
||||
|
||||
Sebagai pengembang yang belajar tentang IoT atau membuat prototipe perangkat, Anda harus mulai dengan *developer kit* atau perangkat pengembang. Perangkat tersebut adalah perangkat IoT untuk tujuan umum yang dirancang untuk digunakan pengembang, seringkali dengan fitur yang tidak akan Anda miliki di perangkat produksi, seperti satu set pin eksternal untuk menghubungkan sensor atau aktuator, perangkat keras untuk mendukung debugging, atau sumber daya tambahan yang akan menambah biaya yang tidak perlu saat melakukan produksi manufaktur.
|
||||
|
||||
Perangkat pengembang ini biasanya terbagi dalam dua kategori - mikrokontroler dan komputer papan tunggal. Perangkat tersebut akan diperkenalkan di sini, dan kita akan membahas lebih detail di pelajaran berikutnya.
|
||||
|
||||
> 💁 Ponsel Anda juga dapat dianggap sebagai perangkat IoT tujuan umum, dengan sensor dan aktuator bawaan, dengan berbagai aplikasi yang menggunakan sensor dan aktuator dengan cara yang berbeda dengan layanan cloud yang berbeda. Anda bahkan dapat menemukan beberapa tutorial IoT yang menggunakan aplikasi ponsel sebagai perangkat IoT.
|
||||
|
||||
### Mikrokontroler
|
||||
|
||||
Mikrokontroler atau Pengendali mikro (juga disebut sebagai MCU, kependekan dari microcontroller unit) adalah komputer kecil yang terdiri dari:
|
||||
|
||||
🧠 Satu atau lebih unit pemrosesan pusat (CPU) - 'otak' mikrokontroler yang menjalankan program Anda
|
||||
|
||||
💾 Memori (RAM dan memori program) - tempat program, data, dan variabel Anda disimpan
|
||||
|
||||
🔌 Koneksi input/output (I/O) yang dapat diprogram - untuk berbicara dengan periferal eksternal (perangkat yang terhubung) seperti sensor dan aktuator
|
||||
|
||||
Mikrokontroler biasanya merupakan perangkat komputasi berbiaya rendah, dengan harga rata-rata untuk yang digunakan dalam perangkat keras khusus turun menjadi sekitar US$0,50, dan beberapa perangkat bahkan semurah US$0,03. Perangkat pengembang dapat ditemukan mulai dari harga US$4, dengan biaya meningkat karena Anda menambahkan lebih banyak fitur. [Wio Terminal](https://www.seeedstudio.com/Wio-Terminal-p-4509.html), perangkat pengembang mikrokontroler dari [Seeed studios](https://www.seeedstudio.com) yang memiliki sensor , aktuator, WiFi, dan layar berharga sekitar US$30.
|
||||
|
||||
|
||||
|
||||
> 💁 Saat mencari mikrokontroler di Internet, berhati-hatilah saat mencari istilah **MCU** karena ini akan mengembalikan banyak hasil untuk Marvel Cinematic Universe, bukan mikrokontroler.
|
||||
|
||||
Mikrokontroler dirancang untuk diprogram untuk melakukan sejumlah tugas yang sangat spesifik, daripada menjadi komputer dengan tujuan umum seperti PC atau Mac. Kecuali untuk skenario yang sangat spesifik, Anda tidak dapat menghubungkan monitor, keyboard, dan mouse dan menggunakannya untuk tugas umum.
|
||||
|
||||
Perangkat pengembang mikrokontroler biasanya dilengkapi dengan sensor dan aktuator tambahan. Sebagian besar papan (board) akan memiliki satu atau lebih LED yang dapat Anda program, bersama dengan perangkat lain seperti steker standar untuk menambahkan lebih banyak sensor atau aktuator menggunakan berbagai ekosistem pabrikan atau sensor bawaan (biasanya yang paling populer seperti sensor suhu). Beberapa mikrokontroler memiliki konektivitas nirkabel bawaan seperti Bluetooth atau WiFi atau memiliki mikrokontroler tambahan di papan untuk menambahkan konektivitas ini.
|
||||
|
||||
> 💁 Mikrokontroler biasanya diprogram dalam bahasa C/C++.
|
||||
|
||||
### Komputer papan tunggal
|
||||
|
||||
Komputer papan tunggal adalah perangkat komputasi kecil yang memiliki semua elemen komputer lengkap yang terdapat pada satu papan kecil. Ini adalah perangkat yang memiliki spesifikasi yang mirip dengan desktop atau laptop PC atau Mac, menjalankan sistem operasi lengkap, tetapi berukuran kecil, menggunakan lebih sedikit daya, dan jauh lebih murah.
|
||||
|
||||
|
||||
|
||||
Raspberry Pi adalah salah satu komputer papan tunggal yang paling populer.
|
||||
|
||||
Seperti mikrokontroler, komputer papan tunggal memiliki CPU, memori dan pin input/output, tetapi mereka memiliki fitur tambahan seperti chip grafis untuk memungkinkan Anda menghubungkan monitor, output audio, dan port USB untuk menghubungkan mouse keyboard dan USB standar lainnya. perangkat seperti webcam atau penyimpanan eksternal. Program disimpan di kartu SD atau hard drive bersama dengan sistem operasi, bukan chip memori yang terpasang di papan.
|
||||
|
||||
> 🎓 Anda dapat menganggap komputer papan tunggal sebagai versi PC atau Mac yang lebih kecil dan lebih murah, dengan tambahan pin GPIO (general-purpose input/output) untuk berinteraksi dengan sensor dan aktuator.
|
||||
|
||||
Komputer papan tunggal adalah komputer berfitur lengkap, sehingga dapat diprogram dalam bahasa apa pun. Perangkat IoT biasanya diprogram dengan Python.
|
||||
|
||||
### Pilihan perangkat keras untuk sisa pelajaran
|
||||
|
||||
Semua pelajaran selanjutnya mencakup tugas menggunakan perangkat IoT untuk berinteraksi dengan dunia fisik dan berkomunikasi dengan cloud. Setiap pelajaran mendukung 3 pilihan perangkat - Arduino (menggunakan Terminal Seeed Studios Wio), atau komputer papan tunggal, baik perangkat fisik (Raspberry Pi 4) atau komputer papan tunggal virtual yang berjalan di PC atau Mac Anda.
|
||||
|
||||
Anda dapat membaca tentang perangkat keras yang diperlukan untuk menyelesaikan semua tugas di [panduan perangkat keras](../../../hardware.md).
|
||||
|
||||
> 💁 Anda tidak perlu membeli perangkat keras IoT apa pun untuk menyelesaikan tugas, Anda dapat melakukan semuanya menggunakan komputer papan tunggal virtual.
|
||||
|
||||
Perangkat keras mana yang Anda pilih terserah Anda - itu tergantung pada apa yang Anda miliki di rumah di sekolah Anda, dan bahasa pemrograman apa yang Anda ketahui atau rencanakan untuk dipelajari. Kedua varian perangkat keras akan menggunakan ekosistem sensor yang sama, jadi jika Anda memulai pada salah satu perangkat, Anda dapat dengan mudah melakukannya pada perangkat lain tanpa harus mengganti sebagian besar perangkat pengembang. Komputer papan tunggal virtual akan setara dengan pembelajaran di Raspberry Pi, dengan sebagian besar kode dapat ditransfer ke Pi jika Anda akhirnya mendapatkan perangkat dan sensor.
|
||||
@ -0,0 +1,21 @@
|
||||
<div dir="rtl">
|
||||
|
||||
# بحث مستشعرات و مشغلات
|
||||
|
||||
## تعليمات
|
||||
|
||||
غطى هذا الدرس أجهزة الاستشعار والمحركات. ابحث وأوصف مستشعرًا ومشغلًا واحدًا يمكن استخدامه مع مجموعة أدوات تطوير إنترنت الأشياء ، بما في ذلك:
|
||||
|
||||
* ماذا يفعل
|
||||
* الأجهزة الإلكترونية / الأجهزة المستخدمة بالداخل
|
||||
* هل هو تناظري أم رقمي
|
||||
* ما هي وحدات ونطاق المدخلات أو القياسات
|
||||
|
||||
## الموضوع
|
||||
|
||||
| المعايير | نموذجي | كافية | يحتاج إلى تحسين |
|
||||
| -------- | --------- | -------- | ----------------- |
|
||||
| وصف جهاز استشعار | وصف جهاز استشعار بما في ذلك تفاصيل عن جميع الأقسام الأربعة المذكورة أعلاه. | وصف جهاز استشعار ، ولكنه كان قادرًا فقط على توفير 2-3 من الأقسام أعلاه | وصف جهاز استشعار ، لكنه كان قادرًا فقط على توفير 1 من الأقسام أعلاه |
|
||||
| وصف المشغل | وصف المشغل بما في ذلك التفاصيل لجميع الأقسام الأربعة المذكورة أعلاه. | وصف مشغل ، لكنه كان قادرًا فقط على توفير 2-3 من الأقسام أعلاه | وصف مشغل ، لكنه كان قادرًا فقط على توفير 1 من الأقسام أعلاه |
|
||||
|
||||
</div>
|
||||
@ -0,0 +1,17 @@
|
||||
# সেন্সর এবং অ্যাকচুয়েটর সংক্রান্ত গবেষণা
|
||||
|
||||
## নির্দেশনা
|
||||
|
||||
এই পাঠটিতে সেন্সর এবং অ্যাকচুয়েটর আলোচনা হয়েছে। একটি আইওটি ডেভলাপার কিটে ব্যবহার করা যেতে পারে এমন একটি সেন্সর এবং একটি অ্যাকচুয়েটর বর্ণনা করতে হবে, যেখানে উল্লেখ থাকবে:
|
||||
|
||||
* এটি কী কাজ করে
|
||||
* ভিতরে ব্যবহৃত ইলেকট্রনিক্স/হার্ডওয়্যার
|
||||
* এটি কি অ্যানালগ নাকি ডিজিটাল
|
||||
* ইনপুট বা পরিমাপের একক কী এবং যন্ত্রটির ব্যবহার্য সীমা (range) কতটুকু
|
||||
|
||||
## এসাইনমেন্ট মূল্যায়ন মানদন্ড
|
||||
|
||||
| ক্রাইটেরিয়া | দৃষ্টান্তমূলক ব্যখ্যা (সর্বোত্তম) | পর্যাপ্ত ব্যখ্যা (মাঝারি) | আরো উন্নতির প্রয়োজন (নিম্ন) |
|
||||
| -------- | --------- | -------- | ----------------- |
|
||||
| একটি সেন্সর সংক্রান্ত বর্ণনা | উপরে তালিকাভুক্ত 4 টি বিভাগের বিশদ ব্যখ্যা সহ সেন্সর বর্ণিত হয়েছে | একটি সেন্সর বর্ণিত হয়েছ, তবে উপরের তালিকা থেকে কেবল 2-3টি বিষয় ব্যখ্যা করতে সক্ষম হয়েছে | একটি সেন্সর বর্ণিত হয়েছ, তবে উপরের তালিকা থেকে কেবল 1টি বিষয় ব্যখ্যা করতে সক্ষম হয়েছে |
|
||||
| একটি অ্যাকচুয়েটর সংক্রান্ত বর্ণনা | উপরে তালিকাভুক্ত 4 টি বিভাগের বিশদ ব্যখ্যা সহ অ্যাকচুয়েটর বর্ণিত হয়েছে | একটি অ্যাকচুয়েটর বর্ণিত হয়েছ, তবে উপরের তালিকা থেকে কেবল 2-3টি বিষয় ব্যখ্যা করতে সক্ষম হয়েছে | একটি অ্যাকচুয়েটর বর্ণিত হয়েছ, তবে উপরের তালিকা থেকে কেবল 1টি বিষয় ব্যখ্যা করতে সক্ষম হয়েছে |
|
||||
@ -0,0 +1,128 @@
|
||||
<div dir="rtl">
|
||||
|
||||
# قم ببناء ضوء ليلي - Raspberry Pi
|
||||
|
||||
في هذا الجزء من الدرس ، ستضيف مؤشر LED إلى Raspberry Pi الخاص بك وتستخدمه لإنشاء ضوء ليلي.
|
||||
|
||||
## المعدات
|
||||
|
||||
يحتاج ضوء الليل الآن إلى مشغل.
|
||||
|
||||
المشغل هو ** LED ** ، <a href="https://wikipedia.org/wiki/Light-emitting_diode"> الصمام الثنائي الباعث للضوء</a> الذي ينبعث منه الضوء عندما يتدفق التيار خلاله. هذا مشغل رقمي له حالتان ، تشغيل وإيقاف. يؤدي إرسال القيمة 1 إلى تشغيل مؤشر LED و 0 يؤدي إلى إيقاف تشغيله. LED هو مشغل Grove خارجي ويجب توصيله بقبعة Grove Base على Raspberry Pi.
|
||||
|
||||
منطق ضوء الليل في الكود الزائف هو:
|
||||
|
||||
```output
|
||||
تحقق من مستوى الضوء.
|
||||
إذا كان الضوء أقل من 300
|
||||
قم بتشغيل LED
|
||||
غير ذلك
|
||||
قم بإيقاف تشغيل LED
|
||||
```
|
||||
### قم بتوصيل الصمام
|
||||
|
||||
يأتي Grove LED كوحدة نمطية مع مجموعة مختارة من مصابيح LED ، مما يسمح لك باختيار اللون.
|
||||
|
||||
#### المهمة - قم بتوصيل LED
|
||||
|
||||
قم بتوصيل الصمام.
|
||||
|
||||
|
||||
|
||||
1. اختر مؤشر LED المفضل لديك وأدخل الأرجل في الفتحتين على وحدة LED.
|
||||
|
||||
المصابيح هي صمامات ثنائية باعثة للضوء ، والصمامات الثنائية هي أجهزة إلكترونية يمكنها حمل التيار في اتجاه واحد فقط. هذا يعني أن مؤشر LED يحتاج إلى الاتصال بالطريقة الصحيحة ، وإلا فلن يعمل.
|
||||
|
||||
أحد أرجل مؤشر LED هو الدبوس الموجب ، والآخر هو الدبوس السالب. LED ليس مستديرًا تمامًا وهو مسطح قليلاً من جانب واحد. الجانب المسطح قليلاً هو الدبوس السالب. عندما تقوم بتوصيل مؤشر LED بالوحدة ، تأكد من توصيل دبوس الجانب المستدير بالمقبس المميز بعلامة ** + ** على الجزء الخارجي من الوحدة ، وأن الجانب المسطح متصل بالمقبس الأقرب إلى منتصف الجزء وحدة.
|
||||
|
||||
1. تحتوي وحدة LED على زر دوران يسمح لك بالتحكم في السطوع. اقلب هذا على طول الطريق لتبدأ بتدويره عكس اتجاه عقارب الساعة بقدر ما سيذهب باستخدام مفك براغي صغير من فيليبس.
|
||||
|
||||
1. أدخل أحد طرفي كبل Grove في المقبس الموجود في وحدة LED. سوف تذهب في اتجاه واحد فقط.
|
||||
|
||||
1. مع إيقاف تشغيل Raspberry Pi ، قم بتوصيل الطرف الآخر من كابل Grove بالمقبس الرقمي المميز بعلامة ** D5 ** على قبعة Grove Base المرفقة بـ Pi. هذا المقبس هو الثاني من اليسار ، على صف المقابس بجوار دبابيس GPIO.
|
||||
|
||||
|
||||
|
||||
|
||||
## برمجة ضوء الليل
|
||||
|
||||
يمكن الآن برمجة ضوء الليل باستخدام مستشعر الضوء Grove و Grove LED.
|
||||
|
||||
### المهمة - برمجة ضوء الليل
|
||||
|
||||
برمجة ضوء الليل.
|
||||
|
||||
1. قم بتشغيل Pi وانتظر حتى يتم التمهيد
|
||||
|
||||
1. افتح مشروع Nightlight في VS Code الذي أنشأته في الجزء السابق من هذه المهمة ، سواء كان يعمل مباشرة على Pi أو متصل باستخدام امتداد Remote SSH.
|
||||
|
||||
1. أضف الكود التالي إلى ملف `app.py` للاتصال لاستيراد المكتبة المطلوبة. يجب إضافة هذا إلى الأعلى ، أسفل سطور "الاستيراد" الأخرى.
|
||||
|
||||
|
||||
```python
|
||||
from grove.grove_led import GroveLed
|
||||
```
|
||||
|
||||
تستورد العبارة `from grove.grove_led import GroveLed`` GroveLed` من مكتبات Grove Python. تحتوي هذه المكتبة على رمز للتفاعل مع Grove LED.
|
||||
|
||||
1. أضف الكود التالي بعد إعلان "light_sensor" لإنشاء مثيل للفئة التي تدير مؤشر LED:
|
||||
|
||||
|
||||
```python
|
||||
led = GroveLed(5)
|
||||
```
|
||||
|
||||
يُنشئ السطر `led = GroveLed (5)` مثيلًا لفئة `GroveLed` التي تتصل بالطرف ** D5 ** - دبوس Grove الرقمي الذي يتصل به مؤشر LED.
|
||||
|
||||
> 💁 جميع المقابس لها أرقام دبوس فريدة. الدبابيس 0 و 2 و 4 و 6 هي دبابيس تمثيلية ، والدبابيس 5 و 16 و 18 و 22 و 24 و 26 هي دبابيس رقمية.
|
||||
|
||||
1. أضف فحصًا داخل حلقة "while" وقبل "time.sleep" للتحقق من مستويات الإضاءة وتشغيل مؤشر LED أو إيقاف تشغيله:
|
||||
|
||||
|
||||
</div>
|
||||
|
||||
```python
|
||||
if light < 300:
|
||||
led.on()
|
||||
else:
|
||||
led.off()
|
||||
```
|
||||
|
||||
<div dir="rtl">
|
||||
يتحقق هذا الرمز من قيمة "light". إذا كان هذا أقل من 300 ، فإنه يستدعي طريقة "on" لفئة "GroveLed" التي ترسل قيمة رقمية 1 إلى LED ، وتشغيلها. إذا كانت قيمة الضوء أكبر من أو تساوي 300 ، فإنها تستدعي طريقة "إيقاف التشغيل" ، وإرسال قيمة رقمية بقيمة 0 إلى LED ، وإيقاف تشغيلها.
|
||||
|
||||
> 💁 يجب وضع مسافة بادئة لهذا الرمز إلى نفس مستوى خط الطباعة ('Light level:'، light) `ليكون داخل حلقة while!
|
||||
|
||||
> 💁 عند إرسال القيم الرقمية إلى المشغلات ، تكون القيمة 0 هي 0 فولت ، والقيمة 1 هي أقصى جهد للجهاز. بالنسبة لـ Raspberry Pi مع مستشعرات ومشغلات Grove ، يكون الجهد 1 هو 3.3 فولت.
|
||||
|
||||
1. من VS Code Terminal ، قم بتشغيل ما يلي لتشغيل تطبيق Python:
|
||||
|
||||
```sh
|
||||
python3 app.py
|
||||
```
|
||||
|
||||
Light values will be output to the console.
|
||||
|
||||
```output
|
||||
pi@raspberrypi:~/nightlight $ python3 app.py
|
||||
Light level: 634
|
||||
Light level: 634
|
||||
Light level: 634
|
||||
Light level: 230
|
||||
Light level: 104
|
||||
Light level: 290
|
||||
```
|
||||
1. قم بتغطية وكشف مستشعر الضوء. لاحظ كيف سيضيء مؤشر LED إذا كان مستوى الضوء 300 أو أقل ، وينطفئ عندما يكون مستوى الضوء أكبر من 300.
|
||||
|
||||
> 💁 إذا لم يتم تشغيل مؤشر LED ، فتأكد من توصيله بالطريقة الصحيحة ، وأن زر الدوران مضبوط على الوضع الكامل.
|
||||
|
||||
|
||||
|
||||
> 💁 يمكنك العثور على هذا الرمز في المجلد
|
||||
[code-actuator/pi](../code-actuator/pi)
|
||||
|
||||
😀 كان برنامج الإضاءة الليلية الخاص بك ناجحًا!
|
||||
</div>
|
||||
|
||||
|
||||
|
||||
@ -1,9 +0,0 @@
|
||||
# Dummy File
|
||||
|
||||
This file acts as a placeholder for the `translations` folder. <br>
|
||||
**Please remove this file after adding the first translation**
|
||||
|
||||
For the instructions, follow the directives in the [translations guide](https://github.com/microsoft/IoT-For-Beginners/blob/main/TRANSLATIONS.md) .
|
||||
|
||||
## THANK YOU
|
||||
We truly appreciate your efforts!
|
||||
@ -0,0 +1,43 @@
|
||||
# জুপিটার নোটবুক ব্যবহার করে জিডিডি ডেটা প্রদর্শন করা
|
||||
|
||||
## নির্দেশাবলী
|
||||
|
||||
এই পাঠে আমরা আইওটি সেন্সর ব্যবহার করে জিডিডি ডেটা সংগ্রহ করেছি। ভাল জিডিডি ডেটা পেতে, আমাদেরকে একাধিক দিনের জন্য ডেটা সংগ্রহ করতে হবে। তাপমাত্রার ডেটা ভিজ্যুয়ালাইজ করতে এবং জিডিডি গণনা করা বা ডেটা বিশ্লেষণ করতে [জুপিটার নোটবুকস](https://jupyter.org) এর মতো ট্যুল ব্যবহার করা যেতে পারে।
|
||||
|
||||
কিছুদিন ডেটা সংগ্রহের মাধ্যমে কাজ শুরু করতে হবে। আইওটি ডিভাইসটি চলমান সময়ে আমাদের সার্ভার কোড যে চলমান রয়েছে তা নিশ্চিত করতে হবে, হয় পাওয়ার ম্যানেজমেন্ট সেটিংস ঠিক করে বা কোন [ সিস্টেমকে অ্যাক্টিভ রাখার পাইথন স্ক্রিপ্ট](https://github.com/jaqsparow/keep-system-active) এর মতো কিছু চালিয়ে।
|
||||
|
||||
একবার তাপমাত্রার ডেটা পেয়ে গেলে আমরা এটি দেখতে এবং জিডিডি গণনা করতে এই রেপোতে জুপিটার নোটবুকটি ব্যবহার করতে পার্রি। জুপিটার নোটবুক এ ব্লক হিসেবে *সেল* এ কোড এবং নির্দেশাবলী মিশ্রিত থাকে, এবং সাধারণত পাইথনেই কোড করা হয়। নির্দেশাবলী পড়ে, তারপরে কোডের প্রতিটি ব্লক বাই ব্লক রান করতে হবে। এখানে প্রদত্ত কোডটি এডিট করা যেতে পারে। উদাহরণস্বরূপ, এই নোটবুকটিতে, য়ামরা আমাদের গাছের জন্য জিডিডি গণনা করতে ব্যবহৃত বেস তাপমাত্রা, উদ্ভিদভেদে ভিন্ন ভিন্ন হবে।
|
||||
|
||||
1. `gdd-calculation` নামে একটি ফোল্ডার খুলতে হবে।
|
||||
|
||||
1. [gdd.ipynb](./code-notebook/gdd.ipynb) ডাউনলোড করে তার একটি কপি `gdd-calculation` ফোল্ডারে রাখতে হবে।
|
||||
|
||||
1. MQTT সার্ভার দ্বারা তৈরী `temperature.csv` ফাইলটি কপি করতে হবে।
|
||||
|
||||
1. `gdd-calculation` ফোল্ডারে নতুন পাইথন এনভায়রনমেন্ট তৈরী করতে হবে।
|
||||
|
||||
1. জুপিটার নোটবুকে কিছু পিপ প্যাকেজ ইন্সটল করতে হবে।
|
||||
|
||||
```sh
|
||||
pip install --upgrade pip
|
||||
pip install pandas
|
||||
pip install matplotlib
|
||||
pip install jupyter
|
||||
```
|
||||
|
||||
1. নোটবুকটি জুপিটারে রান করতে হবে
|
||||
|
||||
```sh
|
||||
jupyter notebook gdd.ipynb
|
||||
```
|
||||
|
||||
জুপিটারটি শুরু হয়ে ব্রাউজারে নোটবুকটি খুলবে। তাপমাত্রা পরিমাপ করা এবং জিডিডি গণনা করতে নোটবুকের নির্দেশাবলীর মাধ্যমে কাজ করতে হবে।
|
||||
|
||||
|
||||
|
||||
## এসাইনমেন্ট মূল্যায়ন মানদন্ড
|
||||
|
||||
| ক্রাইটেরিয়া | দৃষ্টান্তমূলক ব্যখ্যা (সর্বোত্তম) | পর্যাপ্ত ব্যখ্যা (মাঝারি) | আরো উন্নতির প্রয়োজন (নিম্ন) |
|
||||
| -------- | --------- | -------- | ----------------- |
|
||||
| ডেটা সংগ্রহ করা | কমপক্ষে ২টি সম্পূর্ণ দিনের ডেটা সংগ্রহ করা | কমপক্ষে ১টি সম্পূর্ণ দিনের ডেটা সংগ্রহ করা | কিছু ডেটা সংগ্রহ করা |
|
||||
| জিডিডি গণনা করা | সফলতার সাথে নোটবুক রান করে জিডিডি গণনা | সফলতার সাথে নোটবুক রান করা | নোটবুক রান করতে ব্যার্থ |
|
||||
@ -1,9 +0,0 @@
|
||||
# Dummy File
|
||||
|
||||
This file acts as a placeholder for the `translations` folder. <br>
|
||||
**Please remove this file after adding the first translation**
|
||||
|
||||
For the instructions, follow the directives in the [translations guide](https://github.com/microsoft/IoT-For-Beginners/blob/main/TRANSLATIONS.md) .
|
||||
|
||||
## THANK YOU
|
||||
We truly appreciate your efforts!
|
||||
@ -0,0 +1,20 @@
|
||||
# কৃষিকাজে IoT
|
||||
|
||||
জনসংখ্যা যেমন বাড়ছে, তেমনি কৃষির চাহিদাও বাড়ছে। কৃষিজমির পরিমাণ অতোটা পরিবর্তন না হলেও, জলবায়ুর পরিবর্তন ঠিকই হচ্ছে - যা কৃষকদের আরও বেশি সমস্যার মুখে ফেলে দিচ্ছে, বিশেষত সেই ২বিলিয়ন [জীবিকা নির্বাহী কৃষক](https://wikipedia.org/wiki/Subsistence_agriculture) যাদের ফসল বেড়ে ওঠার উপর নির্ভর করেই তাদের পরিবারের অন্নসংস্থান হয়। কোন ধরণের ফসল উৎপাদন করা যাবে, কখন কাজ শুরু করা উচিত, ফলনের বৃদ্ধি, শারিরীক শ্রমের পরিমাণ হ্রাস এবং কীটপতঙ্গগুলি সনাক্ত ও তাদেরকে বিনাশ করার বিষয়ে কৃষকদের অনেকাংশে সাহায্য করতে পারে আইওটি ।
|
||||
|
||||
এই ৬টি লেসনে আমরা শিখবো কীভাবে কৃষিকাজ উন্নত ও স্বয়ংক্রিয় করতে ইন্টারনেট অফ থিংস প্রয়োগ করা যায়।
|
||||
|
||||
> 💁 এই লেসনগুলোতে আমরা ক্লাউড রিসোর্স ব্যবহার করবো। যদি এই অধ্যায়ের সমস্ত পাঠ সম্পূর্ণ করা সম্ভব নাও হয়, তবুও [Clean up your project](../clean-up.md) অংশটি অবশ্যই দেখে নিতে হবে।
|
||||
|
||||
## বিষয়াবলী
|
||||
|
||||
1. [আইওটি দ্বারা উদ্ভিদ বৃদ্ধির পূর্বাভাস](lessons/1-predict-plant-growth/README.md)
|
||||
1. [মাটির আর্দ্রতা সনাক্তকরণ](lessons/2-detect-soil-moisture/README.md)
|
||||
1. [স্বয়ংক্রিয়ভাবে গাছে সেচকার্য](lessons/3-automated-plant-watering/README.md)
|
||||
1. [উদ্ভিদকে ক্লাউড থেকে নিয়ন্ত্রণ](lessons/4-migrate-your-plant-to-the-cloud/README.md)
|
||||
1. [ক্লাউড থেকে এপ্লিকেশন নিয়ন্ত্রণ](lessons/5-migrate-application-to-the-cloud/README.md)
|
||||
1. [উদ্ভিদের নিরাপত্তা নিশ্চিতকরণ](lessons/6-keep-your-plant-secure/README.md)
|
||||
|
||||
## ক্রেডিট
|
||||
|
||||
♥️ প্রতিটি লেসনই ভালোবাসার সাথে তৈরী করেছেন [Jim Bennett](https://GitHub.com/JimBobBennett)
|
||||
@ -1,9 +1,13 @@
|
||||
#
|
||||
# Run other services on the edge
|
||||
|
||||
## Instructions
|
||||
|
||||
It's not just image classifiers that can be run on the edge, anything that can be packaged up into a container can be deployed to an IoT Edge device. Serverless code running as Azure Functions, such as the triggers you've created in earlier lessons can be run in containers, and therefor on IoT Edge.
|
||||
|
||||
Pick one of the previous lessons and try to run the Azure Functions app in an IoT Edge container. You can find a guide that shows how to do this using a different Functions app project in the [Tutorial: Deploy Azure Functions as IoT Edge modules on Microsoft docs](https://docs.microsoft.com/azure/iot-edge/tutorial-deploy-function?view=iotedge-2020-11&WT.mc_id=academic-17441-jabenn).
|
||||
|
||||
## Rubric
|
||||
|
||||
| Criteria | Exemplary | Adequate | Needs Improvement |
|
||||
| -------- | --------- | -------- | ----------------- |
|
||||
| | | | |
|
||||
| Deploy an Azure Functions app to IoT Edge | Was able to deploy an Azure Functions app to IoT Edge and use it with an IoT device to run a trigger from IoT data | Was able to deploy a Functions App to IoT Edge, but was unable to get the trigger to fire | Was unable to deploy a Functions App to IoT Edge |
|
||||
|
||||
@ -0,0 +1,28 @@
|
||||
import io
|
||||
import requests
|
||||
import time
|
||||
from picamera import PiCamera
|
||||
|
||||
camera = PiCamera()
|
||||
camera.resolution = (640, 480)
|
||||
camera.rotation = 0
|
||||
|
||||
time.sleep(2)
|
||||
|
||||
image = io.BytesIO()
|
||||
camera.capture(image, 'jpeg')
|
||||
image.seek(0)
|
||||
|
||||
with open('image.jpg', 'wb') as image_file:
|
||||
image_file.write(image.read())
|
||||
|
||||
prediction_url = '<URL>'
|
||||
headers = {
|
||||
'Content-Type' : 'application/octet-stream'
|
||||
}
|
||||
image.seek(0)
|
||||
response = requests.post(prediction_url, headers=headers, data=image)
|
||||
results = response.json()
|
||||
|
||||
for prediction in results['predictions']:
|
||||
print(f'{prediction["tagName"]}:\t{prediction["probability"] * 100:.2f}%')
|
||||
@ -0,0 +1,28 @@
|
||||
from counterfit_connection import CounterFitConnection
|
||||
CounterFitConnection.init('127.0.0.1', 5000)
|
||||
|
||||
import io
|
||||
import requests
|
||||
from counterfit_shims_picamera import PiCamera
|
||||
|
||||
camera = PiCamera()
|
||||
camera.resolution = (640, 480)
|
||||
camera.rotation = 0
|
||||
|
||||
image = io.BytesIO()
|
||||
camera.capture(image, 'jpeg')
|
||||
image.seek(0)
|
||||
|
||||
with open('image.jpg', 'wb') as image_file:
|
||||
image_file.write(image.read())
|
||||
|
||||
prediction_url = '<URL>'
|
||||
headers = {
|
||||
'Content-Type' : 'application/octet-stream'
|
||||
}
|
||||
image.seek(0)
|
||||
response = requests.post(prediction_url, headers=headers, data=image)
|
||||
results = response.json()
|
||||
|
||||
for prediction in results['predictions']:
|
||||
print(f'{prediction["tagName"]}:\t{prediction["probability"] * 100:.2f}%')
|
||||
@ -0,0 +1,5 @@
|
||||
.pio
|
||||
.vscode/.browse.c_cpp.db*
|
||||
.vscode/c_cpp_properties.json
|
||||
.vscode/launch.json
|
||||
.vscode/ipch
|
||||
@ -0,0 +1,7 @@
|
||||
{
|
||||
// See http://go.microsoft.com/fwlink/?LinkId=827846
|
||||
// for the documentation about the extensions.json format
|
||||
"recommendations": [
|
||||
"platformio.platformio-ide"
|
||||
]
|
||||
}
|
||||
@ -0,0 +1,39 @@
|
||||
|
||||
This directory is intended for project header files.
|
||||
|
||||
A header file is a file containing C declarations and macro definitions
|
||||
to be shared between several project source files. You request the use of a
|
||||
header file in your project source file (C, C++, etc) located in `src` folder
|
||||
by including it, with the C preprocessing directive `#include'.
|
||||
|
||||
```src/main.c
|
||||
|
||||
#include "header.h"
|
||||
|
||||
int main (void)
|
||||
{
|
||||
...
|
||||
}
|
||||
```
|
||||
|
||||
Including a header file produces the same results as copying the header file
|
||||
into each source file that needs it. Such copying would be time-consuming
|
||||
and error-prone. With a header file, the related declarations appear
|
||||
in only one place. If they need to be changed, they can be changed in one
|
||||
place, and programs that include the header file will automatically use the
|
||||
new version when next recompiled. The header file eliminates the labor of
|
||||
finding and changing all the copies as well as the risk that a failure to
|
||||
find one copy will result in inconsistencies within a program.
|
||||
|
||||
In C, the usual convention is to give header files names that end with `.h'.
|
||||
It is most portable to use only letters, digits, dashes, and underscores in
|
||||
header file names, and at most one dot.
|
||||
|
||||
Read more about using header files in official GCC documentation:
|
||||
|
||||
* Include Syntax
|
||||
* Include Operation
|
||||
* Once-Only Headers
|
||||
* Computed Includes
|
||||
|
||||
https://gcc.gnu.org/onlinedocs/cpp/Header-Files.html
|
||||
@ -0,0 +1,46 @@
|
||||
|
||||
This directory is intended for project specific (private) libraries.
|
||||
PlatformIO will compile them to static libraries and link into executable file.
|
||||
|
||||
The source code of each library should be placed in a an own separate directory
|
||||
("lib/your_library_name/[here are source files]").
|
||||
|
||||
For example, see a structure of the following two libraries `Foo` and `Bar`:
|
||||
|
||||
|--lib
|
||||
| |
|
||||
| |--Bar
|
||||
| | |--docs
|
||||
| | |--examples
|
||||
| | |--src
|
||||
| | |- Bar.c
|
||||
| | |- Bar.h
|
||||
| | |- library.json (optional, custom build options, etc) https://docs.platformio.org/page/librarymanager/config.html
|
||||
| |
|
||||
| |--Foo
|
||||
| | |- Foo.c
|
||||
| | |- Foo.h
|
||||
| |
|
||||
| |- README --> THIS FILE
|
||||
|
|
||||
|- platformio.ini
|
||||
|--src
|
||||
|- main.c
|
||||
|
||||
and a contents of `src/main.c`:
|
||||
```
|
||||
#include <Foo.h>
|
||||
#include <Bar.h>
|
||||
|
||||
int main (void)
|
||||
{
|
||||
...
|
||||
}
|
||||
|
||||
```
|
||||
|
||||
PlatformIO Library Dependency Finder will find automatically dependent
|
||||
libraries scanning project source files.
|
||||
|
||||
More information about PlatformIO Library Dependency Finder
|
||||
- https://docs.platformio.org/page/librarymanager/ldf.html
|
||||
@ -0,0 +1,26 @@
|
||||
; PlatformIO Project Configuration File
|
||||
;
|
||||
; Build options: build flags, source filter
|
||||
; Upload options: custom upload port, speed and extra flags
|
||||
; Library options: dependencies, extra library storages
|
||||
; Advanced options: extra scripting
|
||||
;
|
||||
; Please visit documentation for the other options and examples
|
||||
; https://docs.platformio.org/page/projectconf.html
|
||||
|
||||
[env:seeed_wio_terminal]
|
||||
platform = atmelsam
|
||||
board = seeed_wio_terminal
|
||||
framework = arduino
|
||||
lib_deps =
|
||||
seeed-studio/Seeed Arduino rpcWiFi @ 1.0.5
|
||||
seeed-studio/Seeed Arduino FS @ 2.0.3
|
||||
seeed-studio/Seeed Arduino SFUD @ 2.0.1
|
||||
seeed-studio/Seeed Arduino rpcUnified @ 2.1.3
|
||||
seeed-studio/Seeed_Arduino_mbedtls @ 3.0.1
|
||||
seeed-studio/Seeed Arduino RTC @ 2.0.0
|
||||
bblanchon/ArduinoJson @ 6.17.3
|
||||
build_flags =
|
||||
-w
|
||||
-DARDUCAM_SHIELD_V2
|
||||
-DOV2640_CAM
|
||||
@ -0,0 +1,160 @@
|
||||
#pragma once
|
||||
|
||||
#include <ArduCAM.h>
|
||||
#include <Wire.h>
|
||||
|
||||
class Camera
|
||||
{
|
||||
public:
|
||||
Camera(int format, int image_size) : _arducam(OV2640, PIN_SPI_SS)
|
||||
{
|
||||
_format = format;
|
||||
_image_size = image_size;
|
||||
}
|
||||
|
||||
bool init()
|
||||
{
|
||||
// Reset the CPLD
|
||||
_arducam.write_reg(0x07, 0x80);
|
||||
delay(100);
|
||||
|
||||
_arducam.write_reg(0x07, 0x00);
|
||||
delay(100);
|
||||
|
||||
// Check if the ArduCAM SPI bus is OK
|
||||
_arducam.write_reg(ARDUCHIP_TEST1, 0x55);
|
||||
if (_arducam.read_reg(ARDUCHIP_TEST1) != 0x55)
|
||||
{
|
||||
return false;
|
||||
}
|
||||
|
||||
// Change MCU mode
|
||||
_arducam.set_mode(MCU2LCD_MODE);
|
||||
|
||||
uint8_t vid, pid;
|
||||
|
||||
// Check if the camera module type is OV2640
|
||||
_arducam.wrSensorReg8_8(0xff, 0x01);
|
||||
_arducam.rdSensorReg8_8(OV2640_CHIPID_HIGH, &vid);
|
||||
_arducam.rdSensorReg8_8(OV2640_CHIPID_LOW, &pid);
|
||||
if ((vid != 0x26) && ((pid != 0x41) || (pid != 0x42)))
|
||||
{
|
||||
return false;
|
||||
}
|
||||
|
||||
_arducam.set_format(_format);
|
||||
_arducam.InitCAM();
|
||||
_arducam.OV2640_set_JPEG_size(_image_size);
|
||||
_arducam.OV2640_set_Light_Mode(Auto);
|
||||
_arducam.OV2640_set_Special_effects(Normal);
|
||||
delay(1000);
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
void startCapture()
|
||||
{
|
||||
_arducam.flush_fifo();
|
||||
_arducam.clear_fifo_flag();
|
||||
_arducam.start_capture();
|
||||
}
|
||||
|
||||
bool captureReady()
|
||||
{
|
||||
return _arducam.get_bit(ARDUCHIP_TRIG, CAP_DONE_MASK);
|
||||
}
|
||||
|
||||
bool readImageToBuffer(byte **buffer, uint32_t &buffer_length)
|
||||
{
|
||||
if (!captureReady()) return false;
|
||||
|
||||
// Get the image file length
|
||||
uint32_t length = _arducam.read_fifo_length();
|
||||
buffer_length = length;
|
||||
|
||||
if (length >= MAX_FIFO_SIZE)
|
||||
{
|
||||
return false;
|
||||
}
|
||||
if (length == 0)
|
||||
{
|
||||
return false;
|
||||
}
|
||||
|
||||
// create the buffer
|
||||
byte *buf = new byte[length];
|
||||
|
||||
uint8_t temp = 0, temp_last = 0;
|
||||
int i = 0;
|
||||
uint32_t buffer_pos = 0;
|
||||
bool is_header = false;
|
||||
|
||||
_arducam.CS_LOW();
|
||||
_arducam.set_fifo_burst();
|
||||
|
||||
while (length--)
|
||||
{
|
||||
temp_last = temp;
|
||||
temp = SPI.transfer(0x00);
|
||||
//Read JPEG data from FIFO
|
||||
if ((temp == 0xD9) && (temp_last == 0xFF)) //If find the end ,break while,
|
||||
{
|
||||
buf[buffer_pos] = temp;
|
||||
|
||||
buffer_pos++;
|
||||
i++;
|
||||
|
||||
_arducam.CS_HIGH();
|
||||
}
|
||||
if (is_header == true)
|
||||
{
|
||||
//Write image data to buffer if not full
|
||||
if (i < 256)
|
||||
{
|
||||
buf[buffer_pos] = temp;
|
||||
buffer_pos++;
|
||||
i++;
|
||||
}
|
||||
else
|
||||
{
|
||||
_arducam.CS_HIGH();
|
||||
|
||||
i = 0;
|
||||
buf[buffer_pos] = temp;
|
||||
|
||||
buffer_pos++;
|
||||
i++;
|
||||
|
||||
_arducam.CS_LOW();
|
||||
_arducam.set_fifo_burst();
|
||||
}
|
||||
}
|
||||
else if ((temp == 0xD8) & (temp_last == 0xFF))
|
||||
{
|
||||
is_header = true;
|
||||
|
||||
buf[buffer_pos] = temp_last;
|
||||
buffer_pos++;
|
||||
i++;
|
||||
|
||||
buf[buffer_pos] = temp;
|
||||
buffer_pos++;
|
||||
i++;
|
||||
}
|
||||
}
|
||||
|
||||
_arducam.clear_fifo_flag();
|
||||
|
||||
_arducam.set_format(_format);
|
||||
_arducam.InitCAM();
|
||||
_arducam.OV2640_set_JPEG_size(_image_size);
|
||||
|
||||
// return the buffer
|
||||
*buffer = buf;
|
||||
}
|
||||
|
||||
private:
|
||||
ArduCAM _arducam;
|
||||
int _format;
|
||||
int _image_size;
|
||||
};
|
||||
@ -0,0 +1,11 @@
|
||||
#pragma once
|
||||
|
||||
#include <string>
|
||||
|
||||
using namespace std;
|
||||
|
||||
// WiFi credentials
|
||||
const char *SSID = "<SSID>";
|
||||
const char *PASSWORD = "<PASSWORD>";
|
||||
|
||||
const char *PREDICTION_URL = "<PREDICTION_URL>";
|
||||
@ -0,0 +1,123 @@
|
||||
#include <Arduino.h>
|
||||
#include <ArduinoJson.h>
|
||||
#include <HTTPClient.h>
|
||||
#include <rpcWiFi.h>
|
||||
#include "SD/Seeed_SD.h"
|
||||
#include <Seeed_FS.h>
|
||||
#include <SPI.h>
|
||||
#include <WiFiClient.h>
|
||||
|
||||
#include "config.h"
|
||||
#include "camera.h"
|
||||
|
||||
Camera camera = Camera(JPEG, OV2640_640x480);
|
||||
|
||||
WiFiClient client;
|
||||
|
||||
void setupCamera()
|
||||
{
|
||||
pinMode(PIN_SPI_SS, OUTPUT);
|
||||
digitalWrite(PIN_SPI_SS, HIGH);
|
||||
|
||||
Wire.begin();
|
||||
SPI.begin();
|
||||
|
||||
if (!camera.init())
|
||||
{
|
||||
Serial.println("Error setting up the camera!");
|
||||
}
|
||||
}
|
||||
|
||||
void connectWiFi()
|
||||
{
|
||||
while (WiFi.status() != WL_CONNECTED)
|
||||
{
|
||||
Serial.println("Connecting to WiFi..");
|
||||
WiFi.begin(SSID, PASSWORD);
|
||||
delay(500);
|
||||
}
|
||||
|
||||
Serial.println("Connected!");
|
||||
}
|
||||
|
||||
void setup()
|
||||
{
|
||||
Serial.begin(9600);
|
||||
|
||||
while (!Serial)
|
||||
; // Wait for Serial to be ready
|
||||
|
||||
delay(1000);
|
||||
|
||||
connectWiFi();
|
||||
|
||||
setupCamera();
|
||||
|
||||
pinMode(WIO_KEY_C, INPUT_PULLUP);
|
||||
}
|
||||
|
||||
void classifyImage(byte *buffer, uint32_t length)
|
||||
{
|
||||
HTTPClient httpClient;
|
||||
httpClient.begin(client, PREDICTION_URL);
|
||||
httpClient.addHeader("Content-Type", "application/octet-stream");
|
||||
|
||||
int httpResponseCode = httpClient.POST(buffer, length);
|
||||
|
||||
if (httpResponseCode == 200)
|
||||
{
|
||||
String result = httpClient.getString();
|
||||
|
||||
DynamicJsonDocument doc(1024);
|
||||
deserializeJson(doc, result.c_str());
|
||||
|
||||
JsonObject obj = doc.as<JsonObject>();
|
||||
JsonArray predictions = obj["predictions"].as<JsonArray>();
|
||||
|
||||
for(JsonVariant prediction : predictions)
|
||||
{
|
||||
String tag = prediction["tagName"].as<String>();
|
||||
float probability = prediction["probability"].as<float>();
|
||||
|
||||
char buff[32];
|
||||
sprintf(buff, "%s:\t%.2f%%", tag.c_str(), probability * 100.0);
|
||||
Serial.println(buff);
|
||||
}
|
||||
}
|
||||
|
||||
httpClient.end();
|
||||
}
|
||||
|
||||
void buttonPressed()
|
||||
{
|
||||
camera.startCapture();
|
||||
|
||||
while (!camera.captureReady())
|
||||
delay(100);
|
||||
|
||||
Serial.println("Image captured");
|
||||
|
||||
byte *buffer;
|
||||
uint32_t length;
|
||||
|
||||
if (camera.readImageToBuffer(&buffer, length))
|
||||
{
|
||||
Serial.print("Image read to buffer with length ");
|
||||
Serial.println(length);
|
||||
|
||||
classifyImage(buffer, length);
|
||||
|
||||
delete (buffer);
|
||||
}
|
||||
}
|
||||
|
||||
void loop()
|
||||
{
|
||||
if (digitalRead(WIO_KEY_C) == LOW)
|
||||
{
|
||||
buttonPressed();
|
||||
delay(2000);
|
||||
}
|
||||
|
||||
delay(200);
|
||||
}
|
||||
@ -0,0 +1,11 @@
|
||||
|
||||
This directory is intended for PlatformIO Unit Testing and project tests.
|
||||
|
||||
Unit Testing is a software testing method by which individual units of
|
||||
source code, sets of one or more MCU program modules together with associated
|
||||
control data, usage procedures, and operating procedures, are tested to
|
||||
determine whether they are fit for use. Unit testing finds problems early
|
||||
in the development cycle.
|
||||
|
||||
More information about PlatformIO Unit Testing:
|
||||
- https://docs.platformio.org/page/plus/unit-testing.html
|
||||
@ -0,0 +1,66 @@
|
||||
{
|
||||
"content": {
|
||||
"modulesContent": {
|
||||
"$edgeAgent": {
|
||||
"properties.desired": {
|
||||
"schemaVersion": "1.1",
|
||||
"runtime": {
|
||||
"type": "docker",
|
||||
"settings": {
|
||||
"minDockerVersion": "v1.25",
|
||||
"loggingOptions": "",
|
||||
"registryCredentials": {
|
||||
"ClassifierRegistry": {
|
||||
"username": "<Container registry name>",
|
||||
"password": "<Container Password>",
|
||||
"address": "<Container registry name>.azurecr.io"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"systemModules": {
|
||||
"edgeAgent": {
|
||||
"type": "docker",
|
||||
"settings": {
|
||||
"image": "mcr.microsoft.com/azureiotedge-agent:1.1",
|
||||
"createOptions": "{}"
|
||||
}
|
||||
},
|
||||
"edgeHub": {
|
||||
"type": "docker",
|
||||
"status": "running",
|
||||
"restartPolicy": "always",
|
||||
"settings": {
|
||||
"image": "mcr.microsoft.com/azureiotedge-hub:1.1",
|
||||
"createOptions": "{\"HostConfig\":{\"PortBindings\":{\"5671/tcp\":[{\"HostPort\":\"5671\"}],\"8883/tcp\":[{\"HostPort\":\"8883\"}],\"443/tcp\":[{\"HostPort\":\"443\"}]}}}"
|
||||
}
|
||||
}
|
||||
},
|
||||
"modules": {
|
||||
"ImageClassifier": {
|
||||
"version": "1.0",
|
||||
"type": "docker",
|
||||
"status": "running",
|
||||
"restartPolicy": "always",
|
||||
"settings": {
|
||||
"image": "<Container registry name>.azurecr.io/classifier:v1",
|
||||
"createOptions": "{\"ExposedPorts\": {\"80/tcp\": {}},\"HostConfig\": {\"PortBindings\": {\"80/tcp\": [{\"HostPort\": \"80\"}]}}}"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"$edgeHub": {
|
||||
"properties.desired": {
|
||||
"schemaVersion": "1.1",
|
||||
"routes": {
|
||||
"upstream": "FROM /messages/* INTO $upstream"
|
||||
},
|
||||
"storeAndForwardConfiguration": {
|
||||
"timeToLiveSecs": 7200
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -0,0 +1,54 @@
|
||||
# Classify an image using an IoT Edge based image classifier - Virtual IoT Hardware and Raspberry Pi
|
||||
|
||||
In this part of the lesson, you will use the Image Classifier running on the IoT Edge device.
|
||||
|
||||
## Use the IoT Edge classifier
|
||||
|
||||
The IoT device can be re-directed to use the IoT Edge image classifier. The URL for the Image Classifier is `http://<IP address or name>/image`, replacing `<IP address or name>` with the IP address or host name of the computer running IoT Edge.
|
||||
|
||||
The Python library for Custom Vision only works with cloud-hosted models, not models hosted on IoT Edge. This means you will need to use the REST API to call the classifier.
|
||||
|
||||
### Task - use the IoT Edge classifier
|
||||
|
||||
1. Open the `fruit-quality-detector` project in VS Code if it is not already open. If you are using a virtual IoT device, then make sure the virtual environment is activated.
|
||||
|
||||
1. Open the `app.py` file, and remove the import statements from `azure.cognitiveservices.vision.customvision.prediction` and `msrest.authentication`.
|
||||
|
||||
1. Add the following import at the top of the file:
|
||||
|
||||
```python
|
||||
import requests
|
||||
```
|
||||
|
||||
1. Delete all the code after the image is saved to a file, from `image_file.write(image.read())` to the end of the file.
|
||||
|
||||
1. Add the following code to the end of the file:
|
||||
|
||||
```python
|
||||
prediction_url = '<URL>'
|
||||
headers = {
|
||||
'Content-Type' : 'application/octet-stream'
|
||||
}
|
||||
image.seek(0)
|
||||
response = requests.post(prediction_url, headers=headers, data=image)
|
||||
results = response.json()
|
||||
|
||||
for prediction in results['predictions']:
|
||||
print(f'{prediction["tagName"]}:\t{prediction["probability"] * 100:.2f}%')
|
||||
```
|
||||
|
||||
Replace `<URL>` with the URL for your classifier.
|
||||
|
||||
This code makes a REST POST request to the classifier, sending the image as the body of the request. The results come back as JSON, and this is decoded to print out the probabilities.
|
||||
|
||||
1. Run your code, with your camera pointing at some fruit, or an appropriate image set, or fruit visible on your webcam if using virtual IoT hardware. You will see the output in the console:
|
||||
|
||||
```output
|
||||
(.venv) ➜ fruit-quality-detector python app.py
|
||||
ripe: 56.84%
|
||||
unripe: 43.16%
|
||||
```
|
||||
|
||||
> 💁 You can find this code in the [code-classify/pi](code-classify/pi) or [code-classify/virtual-iot-device](code-classify/virtual-iot-device) folder.
|
||||
|
||||
😀 Your fruit quality classifier program was a success!
|
||||
@ -0,0 +1,52 @@
|
||||
# Classify an image using an IoT Edge based image classifier - Wio Terminal
|
||||
|
||||
In this part of the lesson, you will use the Image Classifier running on the IoT Edge device.
|
||||
|
||||
## Use the IoT Edge classifier
|
||||
|
||||
The IoT device can be re-directed to use the IoT Edge image classifier. The URL for the Image Classifier is `http://<IP address or name>/image`, replacing `<IP address or name>` with the IP address or host name of the computer running IoT Edge.
|
||||
|
||||
### Task - use the IoT Edge classifier
|
||||
|
||||
1. Open the `fruit-quality-detector` app project if it's not already open.
|
||||
|
||||
1. The image classifier is running as a REST API using HTTP, not HTTPS, so the call needs to use a WiFi client that works with HTTP calls only. This means the certificate is not needed. Delete the `CERTIFICATE` from the `config.h` file.
|
||||
|
||||
1. The prediction URL in the `config.h` file needs to be updated to the new URL. You can also delete the `PREDICTION_KEY` as this is not needed.
|
||||
|
||||
```cpp
|
||||
const char *PREDICTION_URL = "<URL>";
|
||||
```
|
||||
|
||||
Replace `<URL>` with the URL for your classifier.
|
||||
|
||||
1. In `main.cpp`, change the include directive for the WiFi Client Secure to import the standard HTTP version:
|
||||
|
||||
```cpp
|
||||
#include <WiFiClient.h>
|
||||
```
|
||||
|
||||
1. Change the declaration of `WiFiClient` to be the HTTP version:
|
||||
|
||||
```cpp
|
||||
WiFiClient client;
|
||||
```
|
||||
|
||||
1. Select the line that sets the certificate on the WiFi client. Remove the line `client.setCACert(CERTIFICATE);` from the `connectWiFi` function.
|
||||
|
||||
1. In the `classifyImage` function, remove the `httpClient.addHeader("Prediction-Key", PREDICTION_KEY);` line that sets the prediction key in the header.
|
||||
|
||||
1. Upload and run your code. Point the camera at some fruit and press the C button. You will see the output in the serial monitor:
|
||||
|
||||
```output
|
||||
Connecting to WiFi..
|
||||
Connected!
|
||||
Image captured
|
||||
Image read to buffer with length 8200
|
||||
ripe: 56.84%
|
||||
unripe: 43.16%
|
||||
```
|
||||
|
||||
> 💁 You can find this code in the [code-classify/wio-terminal](code-classify/wio-terminal) folder.
|
||||
|
||||
😀 Your fruit quality classifier program was a success!
|
||||
@ -1,9 +1,11 @@
|
||||
#
|
||||
# Use your object detector on the edge
|
||||
|
||||
## Instructions
|
||||
|
||||
In the last project, you deployed your image classifier to the edge. Do the same with your object detector, exporting it as a compact model and running it on the edge, accessing the edge version from your IoT device.
|
||||
|
||||
## Rubric
|
||||
|
||||
| Criteria | Exemplary | Adequate | Needs Improvement |
|
||||
| -------- | --------- | -------- | ----------------- |
|
||||
| | | | |
|
||||
| Deploy your object detector to the edge | Was able to use the correct compact domain, export the object detector and run it on the edge | Was able to use the correct compact domain, and export the object detector, but was unable to run it on the edge | Was unable to use the correct compact domain, export the object detector, and run it on the edge |
|
||||
|
||||
@ -0,0 +1,92 @@
|
||||
import io
|
||||
import time
|
||||
from picamera import PiCamera
|
||||
|
||||
from azure.cognitiveservices.vision.customvision.prediction import CustomVisionPredictionClient
|
||||
from msrest.authentication import ApiKeyCredentials
|
||||
|
||||
from PIL import Image, ImageDraw, ImageColor
|
||||
from shapely.geometry import Polygon
|
||||
|
||||
camera = PiCamera()
|
||||
camera.resolution = (640, 480)
|
||||
camera.rotation = 0
|
||||
|
||||
time.sleep(2)
|
||||
|
||||
image = io.BytesIO()
|
||||
camera.capture(image, 'jpeg')
|
||||
image.seek(0)
|
||||
|
||||
with open('image.jpg', 'wb') as image_file:
|
||||
image_file.write(image.read())
|
||||
|
||||
prediction_url = '<prediction_url>'
|
||||
prediction_key = '<prediction key>'
|
||||
|
||||
parts = prediction_url.split('/')
|
||||
endpoint = 'https://' + parts[2]
|
||||
project_id = parts[6]
|
||||
iteration_name = parts[9]
|
||||
|
||||
prediction_credentials = ApiKeyCredentials(in_headers={"Prediction-key": prediction_key})
|
||||
predictor = CustomVisionPredictionClient(endpoint, prediction_credentials)
|
||||
|
||||
image.seek(0)
|
||||
results = predictor.detect_image(project_id, iteration_name, image)
|
||||
|
||||
threshold = 0.3
|
||||
|
||||
predictions = list(prediction for prediction in results.predictions if prediction.probability > threshold)
|
||||
|
||||
for prediction in predictions:
|
||||
print(f'{prediction.tag_name}:\t{prediction.probability * 100:.2f}%')
|
||||
|
||||
overlap_threshold = 0.002
|
||||
|
||||
def create_polygon(prediction):
|
||||
scale_left = prediction.bounding_box.left
|
||||
scale_top = prediction.bounding_box.top
|
||||
scale_right = prediction.bounding_box.left + prediction.bounding_box.width
|
||||
scale_bottom = prediction.bounding_box.top + prediction.bounding_box.height
|
||||
|
||||
return Polygon([(scale_left, scale_top), (scale_right, scale_top), (scale_right, scale_bottom), (scale_left, scale_bottom)])
|
||||
|
||||
to_delete = []
|
||||
|
||||
for i in range(0, len(predictions)):
|
||||
polygon_1 = create_polygon(predictions[i])
|
||||
|
||||
for j in range(i+1, len(predictions)):
|
||||
polygon_2 = create_polygon(predictions[j])
|
||||
overlap = polygon_1.intersection(polygon_2).area
|
||||
|
||||
smallest_area = min(polygon_1.area, polygon_2.area)
|
||||
|
||||
if overlap > (overlap_threshold * smallest_area):
|
||||
to_delete.append(predictions[i])
|
||||
break
|
||||
|
||||
for d in to_delete:
|
||||
predictions.remove(d)
|
||||
|
||||
print(f'Counted {len(predictions)} stock items')
|
||||
|
||||
|
||||
with Image.open('image.jpg') as im:
|
||||
draw = ImageDraw.Draw(im)
|
||||
|
||||
for prediction in predictions:
|
||||
scale_left = prediction.bounding_box.left
|
||||
scale_top = prediction.bounding_box.top
|
||||
scale_right = prediction.bounding_box.left + prediction.bounding_box.width
|
||||
scale_bottom = prediction.bounding_box.top + prediction.bounding_box.height
|
||||
|
||||
left = scale_left * im.width
|
||||
top = scale_top * im.height
|
||||
right = scale_right * im.width
|
||||
bottom = scale_bottom * im.height
|
||||
|
||||
draw.rectangle([left, top, right, bottom], outline=ImageColor.getrgb('red'), width=2)
|
||||
|
||||
im.save('image.jpg')
|
||||
@ -0,0 +1,92 @@
|
||||
from counterfit_connection import CounterFitConnection
|
||||
CounterFitConnection.init('127.0.0.1', 5000)
|
||||
|
||||
import io
|
||||
from counterfit_shims_picamera import PiCamera
|
||||
|
||||
from azure.cognitiveservices.vision.customvision.prediction import CustomVisionPredictionClient
|
||||
from msrest.authentication import ApiKeyCredentials
|
||||
|
||||
from PIL import Image, ImageDraw, ImageColor
|
||||
from shapely.geometry import Polygon
|
||||
|
||||
camera = PiCamera()
|
||||
camera.resolution = (640, 480)
|
||||
camera.rotation = 0
|
||||
|
||||
image = io.BytesIO()
|
||||
camera.capture(image, 'jpeg')
|
||||
image.seek(0)
|
||||
|
||||
with open('image.jpg', 'wb') as image_file:
|
||||
image_file.write(image.read())
|
||||
|
||||
prediction_url = '<prediction_url>'
|
||||
prediction_key = '<prediction key>'
|
||||
|
||||
parts = prediction_url.split('/')
|
||||
endpoint = 'https://' + parts[2]
|
||||
project_id = parts[6]
|
||||
iteration_name = parts[9]
|
||||
|
||||
prediction_credentials = ApiKeyCredentials(in_headers={"Prediction-key": prediction_key})
|
||||
predictor = CustomVisionPredictionClient(endpoint, prediction_credentials)
|
||||
|
||||
image.seek(0)
|
||||
results = predictor.detect_image(project_id, iteration_name, image)
|
||||
|
||||
threshold = 0.3
|
||||
|
||||
predictions = list(prediction for prediction in results.predictions if prediction.probability > threshold)
|
||||
|
||||
for prediction in predictions:
|
||||
print(f'{prediction.tag_name}:\t{prediction.probability * 100:.2f}%')
|
||||
|
||||
overlap_threshold = 0.002
|
||||
|
||||
def create_polygon(prediction):
|
||||
scale_left = prediction.bounding_box.left
|
||||
scale_top = prediction.bounding_box.top
|
||||
scale_right = prediction.bounding_box.left + prediction.bounding_box.width
|
||||
scale_bottom = prediction.bounding_box.top + prediction.bounding_box.height
|
||||
|
||||
return Polygon([(scale_left, scale_top), (scale_right, scale_top), (scale_right, scale_bottom), (scale_left, scale_bottom)])
|
||||
|
||||
to_delete = []
|
||||
|
||||
for i in range(0, len(predictions)):
|
||||
polygon_1 = create_polygon(predictions[i])
|
||||
|
||||
for j in range(i+1, len(predictions)):
|
||||
polygon_2 = create_polygon(predictions[j])
|
||||
overlap = polygon_1.intersection(polygon_2).area
|
||||
|
||||
smallest_area = min(polygon_1.area, polygon_2.area)
|
||||
|
||||
if overlap > (overlap_threshold * smallest_area):
|
||||
to_delete.append(predictions[i])
|
||||
break
|
||||
|
||||
for d in to_delete:
|
||||
predictions.remove(d)
|
||||
|
||||
print(f'Counted {len(predictions)} stock items')
|
||||
|
||||
|
||||
with Image.open('image.jpg') as im:
|
||||
draw = ImageDraw.Draw(im)
|
||||
|
||||
for prediction in predictions:
|
||||
scale_left = prediction.bounding_box.left
|
||||
scale_top = prediction.bounding_box.top
|
||||
scale_right = prediction.bounding_box.left + prediction.bounding_box.width
|
||||
scale_bottom = prediction.bounding_box.top + prediction.bounding_box.height
|
||||
|
||||
left = scale_left * im.width
|
||||
top = scale_top * im.height
|
||||
right = scale_right * im.width
|
||||
bottom = scale_bottom * im.height
|
||||
|
||||
draw.rectangle([left, top, right, bottom], outline=ImageColor.getrgb('red'), width=2)
|
||||
|
||||
im.save('image.jpg')
|
||||
@ -0,0 +1,5 @@
|
||||
.pio
|
||||
.vscode/.browse.c_cpp.db*
|
||||
.vscode/c_cpp_properties.json
|
||||
.vscode/launch.json
|
||||
.vscode/ipch
|
||||
@ -0,0 +1,7 @@
|
||||
{
|
||||
// See http://go.microsoft.com/fwlink/?LinkId=827846
|
||||
// for the documentation about the extensions.json format
|
||||
"recommendations": [
|
||||
"platformio.platformio-ide"
|
||||
]
|
||||
}
|
||||
@ -0,0 +1,39 @@
|
||||
|
||||
This directory is intended for project header files.
|
||||
|
||||
A header file is a file containing C declarations and macro definitions
|
||||
to be shared between several project source files. You request the use of a
|
||||
header file in your project source file (C, C++, etc) located in `src` folder
|
||||
by including it, with the C preprocessing directive `#include'.
|
||||
|
||||
```src/main.c
|
||||
|
||||
#include "header.h"
|
||||
|
||||
int main (void)
|
||||
{
|
||||
...
|
||||
}
|
||||
```
|
||||
|
||||
Including a header file produces the same results as copying the header file
|
||||
into each source file that needs it. Such copying would be time-consuming
|
||||
and error-prone. With a header file, the related declarations appear
|
||||
in only one place. If they need to be changed, they can be changed in one
|
||||
place, and programs that include the header file will automatically use the
|
||||
new version when next recompiled. The header file eliminates the labor of
|
||||
finding and changing all the copies as well as the risk that a failure to
|
||||
find one copy will result in inconsistencies within a program.
|
||||
|
||||
In C, the usual convention is to give header files names that end with `.h'.
|
||||
It is most portable to use only letters, digits, dashes, and underscores in
|
||||
header file names, and at most one dot.
|
||||
|
||||
Read more about using header files in official GCC documentation:
|
||||
|
||||
* Include Syntax
|
||||
* Include Operation
|
||||
* Once-Only Headers
|
||||
* Computed Includes
|
||||
|
||||
https://gcc.gnu.org/onlinedocs/cpp/Header-Files.html
|
||||
@ -0,0 +1,46 @@
|
||||
|
||||
This directory is intended for project specific (private) libraries.
|
||||
PlatformIO will compile them to static libraries and link into executable file.
|
||||
|
||||
The source code of each library should be placed in a an own separate directory
|
||||
("lib/your_library_name/[here are source files]").
|
||||
|
||||
For example, see a structure of the following two libraries `Foo` and `Bar`:
|
||||
|
||||
|--lib
|
||||
| |
|
||||
| |--Bar
|
||||
| | |--docs
|
||||
| | |--examples
|
||||
| | |--src
|
||||
| | |- Bar.c
|
||||
| | |- Bar.h
|
||||
| | |- library.json (optional, custom build options, etc) https://docs.platformio.org/page/librarymanager/config.html
|
||||
| |
|
||||
| |--Foo
|
||||
| | |- Foo.c
|
||||
| | |- Foo.h
|
||||
| |
|
||||
| |- README --> THIS FILE
|
||||
|
|
||||
|- platformio.ini
|
||||
|--src
|
||||
|- main.c
|
||||
|
||||
and a contents of `src/main.c`:
|
||||
```
|
||||
#include <Foo.h>
|
||||
#include <Bar.h>
|
||||
|
||||
int main (void)
|
||||
{
|
||||
...
|
||||
}
|
||||
|
||||
```
|
||||
|
||||
PlatformIO Library Dependency Finder will find automatically dependent
|
||||
libraries scanning project source files.
|
||||
|
||||
More information about PlatformIO Library Dependency Finder
|
||||
- https://docs.platformio.org/page/librarymanager/ldf.html
|
||||
@ -0,0 +1,26 @@
|
||||
; PlatformIO Project Configuration File
|
||||
;
|
||||
; Build options: build flags, source filter
|
||||
; Upload options: custom upload port, speed and extra flags
|
||||
; Library options: dependencies, extra library storages
|
||||
; Advanced options: extra scripting
|
||||
;
|
||||
; Please visit documentation for the other options and examples
|
||||
; https://docs.platformio.org/page/projectconf.html
|
||||
|
||||
[env:seeed_wio_terminal]
|
||||
platform = atmelsam
|
||||
board = seeed_wio_terminal
|
||||
framework = arduino
|
||||
lib_deps =
|
||||
seeed-studio/Seeed Arduino rpcWiFi @ 1.0.5
|
||||
seeed-studio/Seeed Arduino FS @ 2.0.3
|
||||
seeed-studio/Seeed Arduino SFUD @ 2.0.1
|
||||
seeed-studio/Seeed Arduino rpcUnified @ 2.1.3
|
||||
seeed-studio/Seeed_Arduino_mbedtls @ 3.0.1
|
||||
seeed-studio/Seeed Arduino RTC @ 2.0.0
|
||||
bblanchon/ArduinoJson @ 6.17.3
|
||||
build_flags =
|
||||
-w
|
||||
-DARDUCAM_SHIELD_V2
|
||||
-DOV2640_CAM
|
||||
@ -0,0 +1,160 @@
|
||||
#pragma once
|
||||
|
||||
#include <ArduCAM.h>
|
||||
#include <Wire.h>
|
||||
|
||||
class Camera
|
||||
{
|
||||
public:
|
||||
Camera(int format, int image_size) : _arducam(OV2640, PIN_SPI_SS)
|
||||
{
|
||||
_format = format;
|
||||
_image_size = image_size;
|
||||
}
|
||||
|
||||
bool init()
|
||||
{
|
||||
// Reset the CPLD
|
||||
_arducam.write_reg(0x07, 0x80);
|
||||
delay(100);
|
||||
|
||||
_arducam.write_reg(0x07, 0x00);
|
||||
delay(100);
|
||||
|
||||
// Check if the ArduCAM SPI bus is OK
|
||||
_arducam.write_reg(ARDUCHIP_TEST1, 0x55);
|
||||
if (_arducam.read_reg(ARDUCHIP_TEST1) != 0x55)
|
||||
{
|
||||
return false;
|
||||
}
|
||||
|
||||
// Change MCU mode
|
||||
_arducam.set_mode(MCU2LCD_MODE);
|
||||
|
||||
uint8_t vid, pid;
|
||||
|
||||
// Check if the camera module type is OV2640
|
||||
_arducam.wrSensorReg8_8(0xff, 0x01);
|
||||
_arducam.rdSensorReg8_8(OV2640_CHIPID_HIGH, &vid);
|
||||
_arducam.rdSensorReg8_8(OV2640_CHIPID_LOW, &pid);
|
||||
if ((vid != 0x26) && ((pid != 0x41) || (pid != 0x42)))
|
||||
{
|
||||
return false;
|
||||
}
|
||||
|
||||
_arducam.set_format(_format);
|
||||
_arducam.InitCAM();
|
||||
_arducam.OV2640_set_JPEG_size(_image_size);
|
||||
_arducam.OV2640_set_Light_Mode(Auto);
|
||||
_arducam.OV2640_set_Special_effects(Normal);
|
||||
delay(1000);
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
void startCapture()
|
||||
{
|
||||
_arducam.flush_fifo();
|
||||
_arducam.clear_fifo_flag();
|
||||
_arducam.start_capture();
|
||||
}
|
||||
|
||||
bool captureReady()
|
||||
{
|
||||
return _arducam.get_bit(ARDUCHIP_TRIG, CAP_DONE_MASK);
|
||||
}
|
||||
|
||||
bool readImageToBuffer(byte **buffer, uint32_t &buffer_length)
|
||||
{
|
||||
if (!captureReady()) return false;
|
||||
|
||||
// Get the image file length
|
||||
uint32_t length = _arducam.read_fifo_length();
|
||||
buffer_length = length;
|
||||
|
||||
if (length >= MAX_FIFO_SIZE)
|
||||
{
|
||||
return false;
|
||||
}
|
||||
if (length == 0)
|
||||
{
|
||||
return false;
|
||||
}
|
||||
|
||||
// create the buffer
|
||||
byte *buf = new byte[length];
|
||||
|
||||
uint8_t temp = 0, temp_last = 0;
|
||||
int i = 0;
|
||||
uint32_t buffer_pos = 0;
|
||||
bool is_header = false;
|
||||
|
||||
_arducam.CS_LOW();
|
||||
_arducam.set_fifo_burst();
|
||||
|
||||
while (length--)
|
||||
{
|
||||
temp_last = temp;
|
||||
temp = SPI.transfer(0x00);
|
||||
//Read JPEG data from FIFO
|
||||
if ((temp == 0xD9) && (temp_last == 0xFF)) //If find the end ,break while,
|
||||
{
|
||||
buf[buffer_pos] = temp;
|
||||
|
||||
buffer_pos++;
|
||||
i++;
|
||||
|
||||
_arducam.CS_HIGH();
|
||||
}
|
||||
if (is_header == true)
|
||||
{
|
||||
//Write image data to buffer if not full
|
||||
if (i < 256)
|
||||
{
|
||||
buf[buffer_pos] = temp;
|
||||
buffer_pos++;
|
||||
i++;
|
||||
}
|
||||
else
|
||||
{
|
||||
_arducam.CS_HIGH();
|
||||
|
||||
i = 0;
|
||||
buf[buffer_pos] = temp;
|
||||
|
||||
buffer_pos++;
|
||||
i++;
|
||||
|
||||
_arducam.CS_LOW();
|
||||
_arducam.set_fifo_burst();
|
||||
}
|
||||
}
|
||||
else if ((temp == 0xD8) & (temp_last == 0xFF))
|
||||
{
|
||||
is_header = true;
|
||||
|
||||
buf[buffer_pos] = temp_last;
|
||||
buffer_pos++;
|
||||
i++;
|
||||
|
||||
buf[buffer_pos] = temp;
|
||||
buffer_pos++;
|
||||
i++;
|
||||
}
|
||||
}
|
||||
|
||||
_arducam.clear_fifo_flag();
|
||||
|
||||
_arducam.set_format(_format);
|
||||
_arducam.InitCAM();
|
||||
_arducam.OV2640_set_JPEG_size(_image_size);
|
||||
|
||||
// return the buffer
|
||||
*buffer = buf;
|
||||
}
|
||||
|
||||
private:
|
||||
ArduCAM _arducam;
|
||||
int _format;
|
||||
int _image_size;
|
||||
};
|
||||
@ -0,0 +1,49 @@
|
||||
#pragma once
|
||||
|
||||
#include <string>
|
||||
|
||||
using namespace std;
|
||||
|
||||
// WiFi credentials
|
||||
const char *SSID = "<SSID>";
|
||||
const char *PASSWORD = "<PASSWORD>";
|
||||
|
||||
const char *PREDICTION_URL = "<PREDICTION_URL>";
|
||||
const char *PREDICTION_KEY = "<PREDICTION_KEY>";
|
||||
|
||||
// Microsoft Azure DigiCert Global Root G2 global certificate
|
||||
const char *CERTIFICATE =
|
||||
"-----BEGIN CERTIFICATE-----\r\n"
|
||||
"MIIF8zCCBNugAwIBAgIQAueRcfuAIek/4tmDg0xQwDANBgkqhkiG9w0BAQwFADBh\r\n"
|
||||
"MQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMRkwFwYDVQQLExB3\r\n"
|
||||
"d3cuZGlnaWNlcnQuY29tMSAwHgYDVQQDExdEaWdpQ2VydCBHbG9iYWwgUm9vdCBH\r\n"
|
||||
"MjAeFw0yMDA3MjkxMjMwMDBaFw0yNDA2MjcyMzU5NTlaMFkxCzAJBgNVBAYTAlVT\r\n"
|
||||
"MR4wHAYDVQQKExVNaWNyb3NvZnQgQ29ycG9yYXRpb24xKjAoBgNVBAMTIU1pY3Jv\r\n"
|
||||
"c29mdCBBenVyZSBUTFMgSXNzdWluZyBDQSAwNjCCAiIwDQYJKoZIhvcNAQEBBQAD\r\n"
|
||||
"ggIPADCCAgoCggIBALVGARl56bx3KBUSGuPc4H5uoNFkFH4e7pvTCxRi4j/+z+Xb\r\n"
|
||||
"wjEz+5CipDOqjx9/jWjskL5dk7PaQkzItidsAAnDCW1leZBOIi68Lff1bjTeZgMY\r\n"
|
||||
"iwdRd3Y39b/lcGpiuP2d23W95YHkMMT8IlWosYIX0f4kYb62rphyfnAjYb/4Od99\r\n"
|
||||
"ThnhlAxGtfvSbXcBVIKCYfZgqRvV+5lReUnd1aNjRYVzPOoifgSx2fRyy1+pO1Uz\r\n"
|
||||
"aMMNnIOE71bVYW0A1hr19w7kOb0KkJXoALTDDj1ukUEDqQuBfBxReL5mXiu1O7WG\r\n"
|
||||
"0vltg0VZ/SZzctBsdBlx1BkmWYBW261KZgBivrql5ELTKKd8qgtHcLQA5fl6JB0Q\r\n"
|
||||
"gs5XDaWehN86Gps5JW8ArjGtjcWAIP+X8CQaWfaCnuRm6Bk/03PQWhgdi84qwA0s\r\n"
|
||||
"sRfFJwHUPTNSnE8EiGVk2frt0u8PG1pwSQsFuNJfcYIHEv1vOzP7uEOuDydsmCjh\r\n"
|
||||
"lxuoK2n5/2aVR3BMTu+p4+gl8alXoBycyLmj3J/PUgqD8SL5fTCUegGsdia/Sa60\r\n"
|
||||
"N2oV7vQ17wjMN+LXa2rjj/b4ZlZgXVojDmAjDwIRdDUujQu0RVsJqFLMzSIHpp2C\r\n"
|
||||
"Zp7mIoLrySay2YYBu7SiNwL95X6He2kS8eefBBHjzwW/9FxGqry57i71c2cDAgMB\r\n"
|
||||
"AAGjggGtMIIBqTAdBgNVHQ4EFgQU1cFnOsKjnfR3UltZEjgp5lVou6UwHwYDVR0j\r\n"
|
||||
"BBgwFoAUTiJUIBiV5uNu5g/6+rkS7QYXjzkwDgYDVR0PAQH/BAQDAgGGMB0GA1Ud\r\n"
|
||||
"JQQWMBQGCCsGAQUFBwMBBggrBgEFBQcDAjASBgNVHRMBAf8ECDAGAQH/AgEAMHYG\r\n"
|
||||
"CCsGAQUFBwEBBGowaDAkBggrBgEFBQcwAYYYaHR0cDovL29jc3AuZGlnaWNlcnQu\r\n"
|
||||
"Y29tMEAGCCsGAQUFBzAChjRodHRwOi8vY2FjZXJ0cy5kaWdpY2VydC5jb20vRGln\r\n"
|
||||
"aUNlcnRHbG9iYWxSb290RzIuY3J0MHsGA1UdHwR0MHIwN6A1oDOGMWh0dHA6Ly9j\r\n"
|
||||
"cmwzLmRpZ2ljZXJ0LmNvbS9EaWdpQ2VydEdsb2JhbFJvb3RHMi5jcmwwN6A1oDOG\r\n"
|
||||
"MWh0dHA6Ly9jcmw0LmRpZ2ljZXJ0LmNvbS9EaWdpQ2VydEdsb2JhbFJvb3RHMi5j\r\n"
|
||||
"cmwwHQYDVR0gBBYwFDAIBgZngQwBAgEwCAYGZ4EMAQICMBAGCSsGAQQBgjcVAQQD\r\n"
|
||||
"AgEAMA0GCSqGSIb3DQEBDAUAA4IBAQB2oWc93fB8esci/8esixj++N22meiGDjgF\r\n"
|
||||
"+rA2LUK5IOQOgcUSTGKSqF9lYfAxPjrqPjDCUPHCURv+26ad5P/BYtXtbmtxJWu+\r\n"
|
||||
"cS5BhMDPPeG3oPZwXRHBJFAkY4O4AF7RIAAUW6EzDflUoDHKv83zOiPfYGcpHc9s\r\n"
|
||||
"kxAInCedk7QSgXvMARjjOqdakor21DTmNIUotxo8kHv5hwRlGhBJwps6fEVi1Bt0\r\n"
|
||||
"trpM/3wYxlr473WSPUFZPgP1j519kLpWOJ8z09wxay+Br29irPcBYv0GMXlHqThy\r\n"
|
||||
"8y4m/HyTQeI2IMvMrQnwqPpY+rLIXyviI2vLoI+4xKE4Rn38ZZ8m\r\n"
|
||||
"-----END CERTIFICATE-----\r\n";
|
||||
@ -0,0 +1,223 @@
|
||||
#include <Arduino.h>
|
||||
#include <ArduinoJson.h>
|
||||
#include <HTTPClient.h>
|
||||
#include <rpcWiFi.h>
|
||||
#include "SD/Seeed_SD.h"
|
||||
#include <Seeed_FS.h>
|
||||
#include <SPI.h>
|
||||
#include <vector>
|
||||
#include <WiFiClientSecure.h>
|
||||
|
||||
#include "config.h"
|
||||
#include "camera.h"
|
||||
|
||||
Camera camera = Camera(JPEG, OV2640_640x480);
|
||||
|
||||
WiFiClientSecure client;
|
||||
|
||||
void setupCamera()
|
||||
{
|
||||
pinMode(PIN_SPI_SS, OUTPUT);
|
||||
digitalWrite(PIN_SPI_SS, HIGH);
|
||||
|
||||
Wire.begin();
|
||||
SPI.begin();
|
||||
|
||||
if (!camera.init())
|
||||
{
|
||||
Serial.println("Error setting up the camera!");
|
||||
}
|
||||
}
|
||||
|
||||
void connectWiFi()
|
||||
{
|
||||
while (WiFi.status() != WL_CONNECTED)
|
||||
{
|
||||
Serial.println("Connecting to WiFi..");
|
||||
WiFi.begin(SSID, PASSWORD);
|
||||
delay(500);
|
||||
}
|
||||
|
||||
client.setCACert(CERTIFICATE);
|
||||
Serial.println("Connected!");
|
||||
}
|
||||
|
||||
void setup()
|
||||
{
|
||||
Serial.begin(9600);
|
||||
|
||||
while (!Serial)
|
||||
; // Wait for Serial to be ready
|
||||
|
||||
delay(1000);
|
||||
|
||||
connectWiFi();
|
||||
|
||||
setupCamera();
|
||||
|
||||
pinMode(WIO_KEY_C, INPUT_PULLUP);
|
||||
}
|
||||
|
||||
const float threshold = 0.0f;
|
||||
const float overlap_threshold = 0.20f;
|
||||
|
||||
struct Point {
|
||||
float x, y;
|
||||
};
|
||||
|
||||
struct Rect {
|
||||
Point topLeft, bottomRight;
|
||||
};
|
||||
|
||||
float area(Rect rect)
|
||||
{
|
||||
return abs(rect.bottomRight.x - rect.topLeft.x) * abs(rect.bottomRight.y - rect.topLeft.y);
|
||||
}
|
||||
|
||||
float overlappingArea(Rect rect1, Rect rect2)
|
||||
{
|
||||
float left = max(rect1.topLeft.x, rect2.topLeft.x);
|
||||
float right = min(rect1.bottomRight.x, rect2.bottomRight.x);
|
||||
float top = max(rect1.topLeft.y, rect2.topLeft.y);
|
||||
float bottom = min(rect1.bottomRight.y, rect2.bottomRight.y);
|
||||
|
||||
|
||||
if ( right > left && bottom > top )
|
||||
{
|
||||
return (right-left)*(bottom-top);
|
||||
}
|
||||
|
||||
return 0.0f;
|
||||
}
|
||||
|
||||
Rect rectFromBoundingBox(JsonVariant prediction)
|
||||
{
|
||||
JsonObject bounding_box = prediction["boundingBox"].as<JsonObject>();
|
||||
|
||||
float left = bounding_box["left"].as<float>();
|
||||
float top = bounding_box["top"].as<float>();
|
||||
float width = bounding_box["width"].as<float>();
|
||||
float height = bounding_box["height"].as<float>();
|
||||
|
||||
Point topLeft = {left, top};
|
||||
Point bottomRight = {left + width, top + height};
|
||||
|
||||
return {topLeft, bottomRight};
|
||||
}
|
||||
|
||||
void processPredictions(std::vector<JsonVariant> &predictions)
|
||||
{
|
||||
std::vector<JsonVariant> passed_predictions;
|
||||
|
||||
for (int i = 0; i < predictions.size(); ++i)
|
||||
{
|
||||
Rect prediction_1_rect = rectFromBoundingBox(predictions[i]);
|
||||
float prediction_1_area = area(prediction_1_rect);
|
||||
bool passed = true;
|
||||
|
||||
for (int j = i + 1; j < predictions.size(); ++j)
|
||||
{
|
||||
Rect prediction_2_rect = rectFromBoundingBox(predictions[j]);
|
||||
float prediction_2_area = area(prediction_2_rect);
|
||||
|
||||
float overlap = overlappingArea(prediction_1_rect, prediction_2_rect);
|
||||
float smallest_area = min(prediction_1_area, prediction_2_area);
|
||||
|
||||
if (overlap > (overlap_threshold * smallest_area))
|
||||
{
|
||||
passed = false;
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
if (passed)
|
||||
{
|
||||
passed_predictions.push_back(predictions[i]);
|
||||
}
|
||||
}
|
||||
|
||||
for(JsonVariant prediction : passed_predictions)
|
||||
{
|
||||
String boundingBox = prediction["boundingBox"].as<String>();
|
||||
String tag = prediction["tagName"].as<String>();
|
||||
float probability = prediction["probability"].as<float>();
|
||||
|
||||
char buff[32];
|
||||
sprintf(buff, "%s:\t%.2f%%\t%s", tag.c_str(), probability * 100.0, boundingBox.c_str());
|
||||
Serial.println(buff);
|
||||
}
|
||||
|
||||
Serial.print("Counted ");
|
||||
Serial.print(passed_predictions.size());
|
||||
Serial.println(" stock items.");
|
||||
}
|
||||
|
||||
void detectStock(byte *buffer, uint32_t length)
|
||||
{
|
||||
HTTPClient httpClient;
|
||||
httpClient.begin(client, PREDICTION_URL);
|
||||
httpClient.addHeader("Content-Type", "application/octet-stream");
|
||||
httpClient.addHeader("Prediction-Key", PREDICTION_KEY);
|
||||
|
||||
int httpResponseCode = httpClient.POST(buffer, length);
|
||||
|
||||
if (httpResponseCode == 200)
|
||||
{
|
||||
String result = httpClient.getString();
|
||||
|
||||
DynamicJsonDocument doc(1024);
|
||||
deserializeJson(doc, result.c_str());
|
||||
|
||||
JsonObject obj = doc.as<JsonObject>();
|
||||
JsonArray predictions = obj["predictions"].as<JsonArray>();
|
||||
|
||||
std::vector<JsonVariant> passed_predictions;
|
||||
|
||||
for(JsonVariant prediction : predictions)
|
||||
{
|
||||
float probability = prediction["probability"].as<float>();
|
||||
if (probability > threshold)
|
||||
{
|
||||
passed_predictions.push_back(prediction);
|
||||
}
|
||||
}
|
||||
|
||||
processPredictions(passed_predictions);
|
||||
}
|
||||
|
||||
httpClient.end();
|
||||
}
|
||||
|
||||
void buttonPressed()
|
||||
{
|
||||
camera.startCapture();
|
||||
|
||||
while (!camera.captureReady())
|
||||
delay(100);
|
||||
|
||||
Serial.println("Image captured");
|
||||
|
||||
byte *buffer;
|
||||
uint32_t length;
|
||||
|
||||
if (camera.readImageToBuffer(&buffer, length))
|
||||
{
|
||||
Serial.print("Image read to buffer with length ");
|
||||
Serial.println(length);
|
||||
|
||||
detectStock(buffer, length);
|
||||
|
||||
delete (buffer);
|
||||
}
|
||||
}
|
||||
|
||||
void loop()
|
||||
{
|
||||
if (digitalRead(WIO_KEY_C) == LOW)
|
||||
{
|
||||
buttonPressed();
|
||||
delay(2000);
|
||||
}
|
||||
|
||||
delay(200);
|
||||
}
|
||||
@ -0,0 +1,11 @@
|
||||
|
||||
This directory is intended for PlatformIO Unit Testing and project tests.
|
||||
|
||||
Unit Testing is a software testing method by which individual units of
|
||||
source code, sets of one or more MCU program modules together with associated
|
||||
control data, usage procedures, and operating procedures, are tested to
|
||||
determine whether they are fit for use. Unit testing finds problems early
|
||||
in the development cycle.
|
||||
|
||||
More information about PlatformIO Unit Testing:
|
||||
- https://docs.platformio.org/page/plus/unit-testing.html
|
||||
@ -0,0 +1,40 @@
|
||||
import io
|
||||
import time
|
||||
from picamera import PiCamera
|
||||
|
||||
from azure.cognitiveservices.vision.customvision.prediction import CustomVisionPredictionClient
|
||||
from msrest.authentication import ApiKeyCredentials
|
||||
|
||||
camera = PiCamera()
|
||||
camera.resolution = (640, 480)
|
||||
camera.rotation = 0
|
||||
|
||||
time.sleep(2)
|
||||
|
||||
image = io.BytesIO()
|
||||
camera.capture(image, 'jpeg')
|
||||
image.seek(0)
|
||||
|
||||
with open('image.jpg', 'wb') as image_file:
|
||||
image_file.write(image.read())
|
||||
|
||||
prediction_url = '<prediction_url>'
|
||||
prediction_key = '<prediction key>'
|
||||
|
||||
parts = prediction_url.split('/')
|
||||
endpoint = 'https://' + parts[2]
|
||||
project_id = parts[6]
|
||||
iteration_name = parts[9]
|
||||
|
||||
prediction_credentials = ApiKeyCredentials(in_headers={"Prediction-key": prediction_key})
|
||||
predictor = CustomVisionPredictionClient(endpoint, prediction_credentials)
|
||||
|
||||
image.seek(0)
|
||||
results = predictor.detect_image(project_id, iteration_name, image)
|
||||
|
||||
threshold = 0.3
|
||||
|
||||
predictions = list(prediction for prediction in results.predictions if prediction.probability > threshold)
|
||||
|
||||
for prediction in predictions:
|
||||
print(f'{prediction.tag_name}:\t{prediction.probability * 100:.2f}%')
|
||||
@ -0,0 +1,40 @@
|
||||
from counterfit_connection import CounterFitConnection
|
||||
CounterFitConnection.init('127.0.0.1', 5000)
|
||||
|
||||
import io
|
||||
from counterfit_shims_picamera import PiCamera
|
||||
|
||||
from azure.cognitiveservices.vision.customvision.prediction import CustomVisionPredictionClient
|
||||
from msrest.authentication import ApiKeyCredentials
|
||||
|
||||
camera = PiCamera()
|
||||
camera.resolution = (640, 480)
|
||||
camera.rotation = 0
|
||||
|
||||
image = io.BytesIO()
|
||||
camera.capture(image, 'jpeg')
|
||||
image.seek(0)
|
||||
|
||||
with open('image.jpg', 'wb') as image_file:
|
||||
image_file.write(image.read())
|
||||
|
||||
prediction_url = '<prediction_url>'
|
||||
prediction_key = '<prediction key>'
|
||||
|
||||
parts = prediction_url.split('/')
|
||||
endpoint = 'https://' + parts[2]
|
||||
project_id = parts[6]
|
||||
iteration_name = parts[9]
|
||||
|
||||
prediction_credentials = ApiKeyCredentials(in_headers={"Prediction-key": prediction_key})
|
||||
predictor = CustomVisionPredictionClient(endpoint, prediction_credentials)
|
||||
|
||||
image.seek(0)
|
||||
results = predictor.detect_image(project_id, iteration_name, image)
|
||||
|
||||
threshold = 0.3
|
||||
|
||||
predictions = list(prediction for prediction in results.predictions if prediction.probability > threshold)
|
||||
|
||||
for prediction in predictions:
|
||||
print(f'{prediction.tag_name}:\t{prediction.probability * 100:.2f}%')
|
||||
@ -0,0 +1,5 @@
|
||||
.pio
|
||||
.vscode/.browse.c_cpp.db*
|
||||
.vscode/c_cpp_properties.json
|
||||
.vscode/launch.json
|
||||
.vscode/ipch
|
||||
@ -0,0 +1,7 @@
|
||||
{
|
||||
// See http://go.microsoft.com/fwlink/?LinkId=827846
|
||||
// for the documentation about the extensions.json format
|
||||
"recommendations": [
|
||||
"platformio.platformio-ide"
|
||||
]
|
||||
}
|
||||
@ -0,0 +1,39 @@
|
||||
|
||||
This directory is intended for project header files.
|
||||
|
||||
A header file is a file containing C declarations and macro definitions
|
||||
to be shared between several project source files. You request the use of a
|
||||
header file in your project source file (C, C++, etc) located in `src` folder
|
||||
by including it, with the C preprocessing directive `#include'.
|
||||
|
||||
```src/main.c
|
||||
|
||||
#include "header.h"
|
||||
|
||||
int main (void)
|
||||
{
|
||||
...
|
||||
}
|
||||
```
|
||||
|
||||
Including a header file produces the same results as copying the header file
|
||||
into each source file that needs it. Such copying would be time-consuming
|
||||
and error-prone. With a header file, the related declarations appear
|
||||
in only one place. If they need to be changed, they can be changed in one
|
||||
place, and programs that include the header file will automatically use the
|
||||
new version when next recompiled. The header file eliminates the labor of
|
||||
finding and changing all the copies as well as the risk that a failure to
|
||||
find one copy will result in inconsistencies within a program.
|
||||
|
||||
In C, the usual convention is to give header files names that end with `.h'.
|
||||
It is most portable to use only letters, digits, dashes, and underscores in
|
||||
header file names, and at most one dot.
|
||||
|
||||
Read more about using header files in official GCC documentation:
|
||||
|
||||
* Include Syntax
|
||||
* Include Operation
|
||||
* Once-Only Headers
|
||||
* Computed Includes
|
||||
|
||||
https://gcc.gnu.org/onlinedocs/cpp/Header-Files.html
|
||||
@ -0,0 +1,46 @@
|
||||
|
||||
This directory is intended for project specific (private) libraries.
|
||||
PlatformIO will compile them to static libraries and link into executable file.
|
||||
|
||||
The source code of each library should be placed in a an own separate directory
|
||||
("lib/your_library_name/[here are source files]").
|
||||
|
||||
For example, see a structure of the following two libraries `Foo` and `Bar`:
|
||||
|
||||
|--lib
|
||||
| |
|
||||
| |--Bar
|
||||
| | |--docs
|
||||
| | |--examples
|
||||
| | |--src
|
||||
| | |- Bar.c
|
||||
| | |- Bar.h
|
||||
| | |- library.json (optional, custom build options, etc) https://docs.platformio.org/page/librarymanager/config.html
|
||||
| |
|
||||
| |--Foo
|
||||
| | |- Foo.c
|
||||
| | |- Foo.h
|
||||
| |
|
||||
| |- README --> THIS FILE
|
||||
|
|
||||
|- platformio.ini
|
||||
|--src
|
||||
|- main.c
|
||||
|
||||
and a contents of `src/main.c`:
|
||||
```
|
||||
#include <Foo.h>
|
||||
#include <Bar.h>
|
||||
|
||||
int main (void)
|
||||
{
|
||||
...
|
||||
}
|
||||
|
||||
```
|
||||
|
||||
PlatformIO Library Dependency Finder will find automatically dependent
|
||||
libraries scanning project source files.
|
||||
|
||||
More information about PlatformIO Library Dependency Finder
|
||||
- https://docs.platformio.org/page/librarymanager/ldf.html
|
||||
@ -0,0 +1,26 @@
|
||||
; PlatformIO Project Configuration File
|
||||
;
|
||||
; Build options: build flags, source filter
|
||||
; Upload options: custom upload port, speed and extra flags
|
||||
; Library options: dependencies, extra library storages
|
||||
; Advanced options: extra scripting
|
||||
;
|
||||
; Please visit documentation for the other options and examples
|
||||
; https://docs.platformio.org/page/projectconf.html
|
||||
|
||||
[env:seeed_wio_terminal]
|
||||
platform = atmelsam
|
||||
board = seeed_wio_terminal
|
||||
framework = arduino
|
||||
lib_deps =
|
||||
seeed-studio/Seeed Arduino rpcWiFi @ 1.0.5
|
||||
seeed-studio/Seeed Arduino FS @ 2.0.3
|
||||
seeed-studio/Seeed Arduino SFUD @ 2.0.1
|
||||
seeed-studio/Seeed Arduino rpcUnified @ 2.1.3
|
||||
seeed-studio/Seeed_Arduino_mbedtls @ 3.0.1
|
||||
seeed-studio/Seeed Arduino RTC @ 2.0.0
|
||||
bblanchon/ArduinoJson @ 6.17.3
|
||||
build_flags =
|
||||
-w
|
||||
-DARDUCAM_SHIELD_V2
|
||||
-DOV2640_CAM
|
||||
@ -0,0 +1,160 @@
|
||||
#pragma once
|
||||
|
||||
#include <ArduCAM.h>
|
||||
#include <Wire.h>
|
||||
|
||||
class Camera
|
||||
{
|
||||
public:
|
||||
Camera(int format, int image_size) : _arducam(OV2640, PIN_SPI_SS)
|
||||
{
|
||||
_format = format;
|
||||
_image_size = image_size;
|
||||
}
|
||||
|
||||
bool init()
|
||||
{
|
||||
// Reset the CPLD
|
||||
_arducam.write_reg(0x07, 0x80);
|
||||
delay(100);
|
||||
|
||||
_arducam.write_reg(0x07, 0x00);
|
||||
delay(100);
|
||||
|
||||
// Check if the ArduCAM SPI bus is OK
|
||||
_arducam.write_reg(ARDUCHIP_TEST1, 0x55);
|
||||
if (_arducam.read_reg(ARDUCHIP_TEST1) != 0x55)
|
||||
{
|
||||
return false;
|
||||
}
|
||||
|
||||
// Change MCU mode
|
||||
_arducam.set_mode(MCU2LCD_MODE);
|
||||
|
||||
uint8_t vid, pid;
|
||||
|
||||
// Check if the camera module type is OV2640
|
||||
_arducam.wrSensorReg8_8(0xff, 0x01);
|
||||
_arducam.rdSensorReg8_8(OV2640_CHIPID_HIGH, &vid);
|
||||
_arducam.rdSensorReg8_8(OV2640_CHIPID_LOW, &pid);
|
||||
if ((vid != 0x26) && ((pid != 0x41) || (pid != 0x42)))
|
||||
{
|
||||
return false;
|
||||
}
|
||||
|
||||
_arducam.set_format(_format);
|
||||
_arducam.InitCAM();
|
||||
_arducam.OV2640_set_JPEG_size(_image_size);
|
||||
_arducam.OV2640_set_Light_Mode(Auto);
|
||||
_arducam.OV2640_set_Special_effects(Normal);
|
||||
delay(1000);
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
void startCapture()
|
||||
{
|
||||
_arducam.flush_fifo();
|
||||
_arducam.clear_fifo_flag();
|
||||
_arducam.start_capture();
|
||||
}
|
||||
|
||||
bool captureReady()
|
||||
{
|
||||
return _arducam.get_bit(ARDUCHIP_TRIG, CAP_DONE_MASK);
|
||||
}
|
||||
|
||||
bool readImageToBuffer(byte **buffer, uint32_t &buffer_length)
|
||||
{
|
||||
if (!captureReady()) return false;
|
||||
|
||||
// Get the image file length
|
||||
uint32_t length = _arducam.read_fifo_length();
|
||||
buffer_length = length;
|
||||
|
||||
if (length >= MAX_FIFO_SIZE)
|
||||
{
|
||||
return false;
|
||||
}
|
||||
if (length == 0)
|
||||
{
|
||||
return false;
|
||||
}
|
||||
|
||||
// create the buffer
|
||||
byte *buf = new byte[length];
|
||||
|
||||
uint8_t temp = 0, temp_last = 0;
|
||||
int i = 0;
|
||||
uint32_t buffer_pos = 0;
|
||||
bool is_header = false;
|
||||
|
||||
_arducam.CS_LOW();
|
||||
_arducam.set_fifo_burst();
|
||||
|
||||
while (length--)
|
||||
{
|
||||
temp_last = temp;
|
||||
temp = SPI.transfer(0x00);
|
||||
//Read JPEG data from FIFO
|
||||
if ((temp == 0xD9) && (temp_last == 0xFF)) //If find the end ,break while,
|
||||
{
|
||||
buf[buffer_pos] = temp;
|
||||
|
||||
buffer_pos++;
|
||||
i++;
|
||||
|
||||
_arducam.CS_HIGH();
|
||||
}
|
||||
if (is_header == true)
|
||||
{
|
||||
//Write image data to buffer if not full
|
||||
if (i < 256)
|
||||
{
|
||||
buf[buffer_pos] = temp;
|
||||
buffer_pos++;
|
||||
i++;
|
||||
}
|
||||
else
|
||||
{
|
||||
_arducam.CS_HIGH();
|
||||
|
||||
i = 0;
|
||||
buf[buffer_pos] = temp;
|
||||
|
||||
buffer_pos++;
|
||||
i++;
|
||||
|
||||
_arducam.CS_LOW();
|
||||
_arducam.set_fifo_burst();
|
||||
}
|
||||
}
|
||||
else if ((temp == 0xD8) & (temp_last == 0xFF))
|
||||
{
|
||||
is_header = true;
|
||||
|
||||
buf[buffer_pos] = temp_last;
|
||||
buffer_pos++;
|
||||
i++;
|
||||
|
||||
buf[buffer_pos] = temp;
|
||||
buffer_pos++;
|
||||
i++;
|
||||
}
|
||||
}
|
||||
|
||||
_arducam.clear_fifo_flag();
|
||||
|
||||
_arducam.set_format(_format);
|
||||
_arducam.InitCAM();
|
||||
_arducam.OV2640_set_JPEG_size(_image_size);
|
||||
|
||||
// return the buffer
|
||||
*buffer = buf;
|
||||
}
|
||||
|
||||
private:
|
||||
ArduCAM _arducam;
|
||||
int _format;
|
||||
int _image_size;
|
||||
};
|
||||
@ -0,0 +1,49 @@
|
||||
#pragma once
|
||||
|
||||
#include <string>
|
||||
|
||||
using namespace std;
|
||||
|
||||
// WiFi credentials
|
||||
const char *SSID = "<SSID>";
|
||||
const char *PASSWORD = "<PASSWORD>";
|
||||
|
||||
const char *PREDICTION_URL = "<PREDICTION_URL>";
|
||||
const char *PREDICTION_KEY = "<PREDICTION_KEY>";
|
||||
|
||||
// Microsoft Azure DigiCert Global Root G2 global certificate
|
||||
const char *CERTIFICATE =
|
||||
"-----BEGIN CERTIFICATE-----\r\n"
|
||||
"MIIF8zCCBNugAwIBAgIQAueRcfuAIek/4tmDg0xQwDANBgkqhkiG9w0BAQwFADBh\r\n"
|
||||
"MQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMRkwFwYDVQQLExB3\r\n"
|
||||
"d3cuZGlnaWNlcnQuY29tMSAwHgYDVQQDExdEaWdpQ2VydCBHbG9iYWwgUm9vdCBH\r\n"
|
||||
"MjAeFw0yMDA3MjkxMjMwMDBaFw0yNDA2MjcyMzU5NTlaMFkxCzAJBgNVBAYTAlVT\r\n"
|
||||
"MR4wHAYDVQQKExVNaWNyb3NvZnQgQ29ycG9yYXRpb24xKjAoBgNVBAMTIU1pY3Jv\r\n"
|
||||
"c29mdCBBenVyZSBUTFMgSXNzdWluZyBDQSAwNjCCAiIwDQYJKoZIhvcNAQEBBQAD\r\n"
|
||||
"ggIPADCCAgoCggIBALVGARl56bx3KBUSGuPc4H5uoNFkFH4e7pvTCxRi4j/+z+Xb\r\n"
|
||||
"wjEz+5CipDOqjx9/jWjskL5dk7PaQkzItidsAAnDCW1leZBOIi68Lff1bjTeZgMY\r\n"
|
||||
"iwdRd3Y39b/lcGpiuP2d23W95YHkMMT8IlWosYIX0f4kYb62rphyfnAjYb/4Od99\r\n"
|
||||
"ThnhlAxGtfvSbXcBVIKCYfZgqRvV+5lReUnd1aNjRYVzPOoifgSx2fRyy1+pO1Uz\r\n"
|
||||
"aMMNnIOE71bVYW0A1hr19w7kOb0KkJXoALTDDj1ukUEDqQuBfBxReL5mXiu1O7WG\r\n"
|
||||
"0vltg0VZ/SZzctBsdBlx1BkmWYBW261KZgBivrql5ELTKKd8qgtHcLQA5fl6JB0Q\r\n"
|
||||
"gs5XDaWehN86Gps5JW8ArjGtjcWAIP+X8CQaWfaCnuRm6Bk/03PQWhgdi84qwA0s\r\n"
|
||||
"sRfFJwHUPTNSnE8EiGVk2frt0u8PG1pwSQsFuNJfcYIHEv1vOzP7uEOuDydsmCjh\r\n"
|
||||
"lxuoK2n5/2aVR3BMTu+p4+gl8alXoBycyLmj3J/PUgqD8SL5fTCUegGsdia/Sa60\r\n"
|
||||
"N2oV7vQ17wjMN+LXa2rjj/b4ZlZgXVojDmAjDwIRdDUujQu0RVsJqFLMzSIHpp2C\r\n"
|
||||
"Zp7mIoLrySay2YYBu7SiNwL95X6He2kS8eefBBHjzwW/9FxGqry57i71c2cDAgMB\r\n"
|
||||
"AAGjggGtMIIBqTAdBgNVHQ4EFgQU1cFnOsKjnfR3UltZEjgp5lVou6UwHwYDVR0j\r\n"
|
||||
"BBgwFoAUTiJUIBiV5uNu5g/6+rkS7QYXjzkwDgYDVR0PAQH/BAQDAgGGMB0GA1Ud\r\n"
|
||||
"JQQWMBQGCCsGAQUFBwMBBggrBgEFBQcDAjASBgNVHRMBAf8ECDAGAQH/AgEAMHYG\r\n"
|
||||
"CCsGAQUFBwEBBGowaDAkBggrBgEFBQcwAYYYaHR0cDovL29jc3AuZGlnaWNlcnQu\r\n"
|
||||
"Y29tMEAGCCsGAQUFBzAChjRodHRwOi8vY2FjZXJ0cy5kaWdpY2VydC5jb20vRGln\r\n"
|
||||
"aUNlcnRHbG9iYWxSb290RzIuY3J0MHsGA1UdHwR0MHIwN6A1oDOGMWh0dHA6Ly9j\r\n"
|
||||
"cmwzLmRpZ2ljZXJ0LmNvbS9EaWdpQ2VydEdsb2JhbFJvb3RHMi5jcmwwN6A1oDOG\r\n"
|
||||
"MWh0dHA6Ly9jcmw0LmRpZ2ljZXJ0LmNvbS9EaWdpQ2VydEdsb2JhbFJvb3RHMi5j\r\n"
|
||||
"cmwwHQYDVR0gBBYwFDAIBgZngQwBAgEwCAYGZ4EMAQICMBAGCSsGAQQBgjcVAQQD\r\n"
|
||||
"AgEAMA0GCSqGSIb3DQEBDAUAA4IBAQB2oWc93fB8esci/8esixj++N22meiGDjgF\r\n"
|
||||
"+rA2LUK5IOQOgcUSTGKSqF9lYfAxPjrqPjDCUPHCURv+26ad5P/BYtXtbmtxJWu+\r\n"
|
||||
"cS5BhMDPPeG3oPZwXRHBJFAkY4O4AF7RIAAUW6EzDflUoDHKv83zOiPfYGcpHc9s\r\n"
|
||||
"kxAInCedk7QSgXvMARjjOqdakor21DTmNIUotxo8kHv5hwRlGhBJwps6fEVi1Bt0\r\n"
|
||||
"trpM/3wYxlr473WSPUFZPgP1j519kLpWOJ8z09wxay+Br29irPcBYv0GMXlHqThy\r\n"
|
||||
"8y4m/HyTQeI2IMvMrQnwqPpY+rLIXyviI2vLoI+4xKE4Rn38ZZ8m\r\n"
|
||||
"-----END CERTIFICATE-----\r\n";
|
||||
@ -0,0 +1,145 @@
|
||||
#include <Arduino.h>
|
||||
#include <ArduinoJson.h>
|
||||
#include <HTTPClient.h>
|
||||
#include <list>
|
||||
#include <rpcWiFi.h>
|
||||
#include "SD/Seeed_SD.h"
|
||||
#include <Seeed_FS.h>
|
||||
#include <SPI.h>
|
||||
#include <vector>
|
||||
#include <WiFiClientSecure.h>
|
||||
|
||||
#include "config.h"
|
||||
#include "camera.h"
|
||||
|
||||
Camera camera = Camera(JPEG, OV2640_640x480);
|
||||
|
||||
WiFiClientSecure client;
|
||||
|
||||
void setupCamera()
|
||||
{
|
||||
pinMode(PIN_SPI_SS, OUTPUT);
|
||||
digitalWrite(PIN_SPI_SS, HIGH);
|
||||
|
||||
Wire.begin();
|
||||
SPI.begin();
|
||||
|
||||
if (!camera.init())
|
||||
{
|
||||
Serial.println("Error setting up the camera!");
|
||||
}
|
||||
}
|
||||
|
||||
void connectWiFi()
|
||||
{
|
||||
while (WiFi.status() != WL_CONNECTED)
|
||||
{
|
||||
Serial.println("Connecting to WiFi..");
|
||||
WiFi.begin(SSID, PASSWORD);
|
||||
delay(500);
|
||||
}
|
||||
|
||||
client.setCACert(CERTIFICATE);
|
||||
Serial.println("Connected!");
|
||||
}
|
||||
|
||||
void setup()
|
||||
{
|
||||
Serial.begin(9600);
|
||||
|
||||
while (!Serial)
|
||||
; // Wait for Serial to be ready
|
||||
|
||||
delay(1000);
|
||||
|
||||
connectWiFi();
|
||||
|
||||
setupCamera();
|
||||
|
||||
pinMode(WIO_KEY_C, INPUT_PULLUP);
|
||||
}
|
||||
|
||||
const float threshold = 0.3f;
|
||||
|
||||
void processPredictions(std::vector<JsonVariant> &predictions)
|
||||
{
|
||||
for(JsonVariant prediction : predictions)
|
||||
{
|
||||
String tag = prediction["tagName"].as<String>();
|
||||
float probability = prediction["probability"].as<float>();
|
||||
|
||||
char buff[32];
|
||||
sprintf(buff, "%s:\t%.2f%%", tag.c_str(), probability * 100.0);
|
||||
Serial.println(buff);
|
||||
}
|
||||
}
|
||||
|
||||
void detectStock(byte *buffer, uint32_t length)
|
||||
{
|
||||
HTTPClient httpClient;
|
||||
httpClient.begin(client, PREDICTION_URL);
|
||||
httpClient.addHeader("Content-Type", "application/octet-stream");
|
||||
httpClient.addHeader("Prediction-Key", PREDICTION_KEY);
|
||||
|
||||
int httpResponseCode = httpClient.POST(buffer, length);
|
||||
|
||||
if (httpResponseCode == 200)
|
||||
{
|
||||
String result = httpClient.getString();
|
||||
|
||||
DynamicJsonDocument doc(1024);
|
||||
deserializeJson(doc, result.c_str());
|
||||
|
||||
JsonObject obj = doc.as<JsonObject>();
|
||||
JsonArray predictions = obj["predictions"].as<JsonArray>();
|
||||
|
||||
std::vector<JsonVariant> passed_predictions;
|
||||
|
||||
for(JsonVariant prediction : predictions)
|
||||
{
|
||||
float probability = prediction["probability"].as<float>();
|
||||
if (probability > threshold)
|
||||
{
|
||||
passed_predictions.push_back(prediction);
|
||||
}
|
||||
}
|
||||
|
||||
processPredictions(passed_predictions);
|
||||
}
|
||||
|
||||
httpClient.end();
|
||||
}
|
||||
|
||||
void buttonPressed()
|
||||
{
|
||||
camera.startCapture();
|
||||
|
||||
while (!camera.captureReady())
|
||||
delay(100);
|
||||
|
||||
Serial.println("Image captured");
|
||||
|
||||
byte *buffer;
|
||||
uint32_t length;
|
||||
|
||||
if (camera.readImageToBuffer(&buffer, length))
|
||||
{
|
||||
Serial.print("Image read to buffer with length ");
|
||||
Serial.println(length);
|
||||
|
||||
detectStock(buffer, length);
|
||||
|
||||
delete (buffer);
|
||||
}
|
||||
}
|
||||
|
||||
void loop()
|
||||
{
|
||||
if (digitalRead(WIO_KEY_C) == LOW)
|
||||
{
|
||||
buttonPressed();
|
||||
delay(2000);
|
||||
}
|
||||
|
||||
delay(200);
|
||||
}
|
||||
@ -0,0 +1,11 @@
|
||||
|
||||
This directory is intended for PlatformIO Unit Testing and project tests.
|
||||
|
||||
Unit Testing is a software testing method by which individual units of
|
||||
source code, sets of one or more MCU program modules together with associated
|
||||
control data, usage procedures, and operating procedures, are tested to
|
||||
determine whether they are fit for use. Unit testing finds problems early
|
||||
in the development cycle.
|
||||
|
||||
More information about PlatformIO Unit Testing:
|
||||
- https://docs.platformio.org/page/plus/unit-testing.html
|
||||
@ -0,0 +1,163 @@
|
||||
# Count stock from your IoT device - Virtual IoT Hardware and Raspberry Pi
|
||||
|
||||
A combination of the predictions and their bounding boxes can be used to count stock in an image
|
||||
|
||||
## Show bounding boxes
|
||||
|
||||
As a helpful debugging step you can not only print out the bounding boxes, but you can also draw them on the image that was written to disk when an image was captured.
|
||||
|
||||
### Task - print the bounding boxes
|
||||
|
||||
1. Ensure the `stock-counter` project is open in VS Code, and the virtual environment is activated if you are using a virtual IoT device.
|
||||
|
||||
1. Change the `print` statement in the `for` loop to the following to print the bounding boxes to the console:
|
||||
|
||||
```python
|
||||
print(f'{prediction.tag_name}:\t{prediction.probability * 100:.2f}%\t{prediction.bounding_box}')
|
||||
```
|
||||
|
||||
1. Run the app with the camera pointing at some stock on a shelf. The bounding boxes will be printed to the console, with left, top, width and height values from 0-1.
|
||||
|
||||
```output
|
||||
pi@raspberrypi:~/stock-counter $ python3 app.py
|
||||
tomato paste: 33.42% {'additional_properties': {}, 'left': 0.3455171, 'top': 0.09916268, 'width': 0.14175442, 'height': 0.29405564}
|
||||
tomato paste: 34.41% {'additional_properties': {}, 'left': 0.48283678, 'top': 0.10242918, 'width': 0.11782813, 'height': 0.27467814}
|
||||
tomato paste: 31.25% {'additional_properties': {}, 'left': 0.4923783, 'top': 0.35007596, 'width': 0.13668466, 'height': 0.28304994}
|
||||
tomato paste: 31.05% {'additional_properties': {}, 'left': 0.36416405, 'top': 0.37494493, 'width': 0.14024884, 'height': 0.26880276}
|
||||
```
|
||||
|
||||
### Task - draw bounding boxes on the image
|
||||
|
||||
1. The Pip package [Pillow](https://pypi.org/project/Pillow/) can be used to draw on images. Install this with the following command:
|
||||
|
||||
```sh
|
||||
pip3 install pillow
|
||||
```
|
||||
|
||||
If you are using a virtual IoT device, make sure to run this from inside the activated virtual environment.
|
||||
|
||||
1. Add the following import statement to the top of the `app.py` file:
|
||||
|
||||
```python
|
||||
from PIL import Image, ImageDraw, ImageColor
|
||||
```
|
||||
|
||||
This imports code needed to edit the image.
|
||||
|
||||
1. Add the following code to the end of the `app.py` file:
|
||||
|
||||
```python
|
||||
with Image.open('image.jpg') as im:
|
||||
draw = ImageDraw.Draw(im)
|
||||
|
||||
for prediction in predictions:
|
||||
scale_left = prediction.bounding_box.left
|
||||
scale_top = prediction.bounding_box.top
|
||||
scale_right = prediction.bounding_box.left + prediction.bounding_box.width
|
||||
scale_bottom = prediction.bounding_box.top + prediction.bounding_box.height
|
||||
|
||||
left = scale_left * im.width
|
||||
top = scale_top * im.height
|
||||
right = scale_right * im.width
|
||||
bottom = scale_bottom * im.height
|
||||
|
||||
draw.rectangle([left, top, right, bottom], outline=ImageColor.getrgb('red'), width=2)
|
||||
|
||||
im.save('image.jpg')
|
||||
```
|
||||
|
||||
This code opens the image that was saved earlier for editing. It then loops through the predictions getting the bounding boxes, and calculates the bottom right coordinate using the bounding box values from 0-1. These are then converted to image coordinates by multiplying by the relevant dimension of the image. For example, if the left value was 0.5 on an image that was 600 pixels wide, this would convert it to 300 (0.5 x 600 = 300).
|
||||
|
||||
Each bounding box is drawn on the image using a red line. Finally the edited image is saved, overwriting the original image.
|
||||
|
||||
1. Run the app with the camera pointing at some stock on a shelf. You will see the `image.jpg` file in the VS Code explorer, and you will be able to select it to see the bounding boxes.
|
||||
|
||||
|
||||
|
||||
## Count stock
|
||||
|
||||
In the image shown above, the bounding boxes have a small overlap. If this overlap was much larger, then the bounding boxes may indicate the same object. To count the objects correctly, you need to ignore boxes with a significant overlap.
|
||||
|
||||
### Task - count stock ignoring overlap
|
||||
|
||||
1. The Pip package [Shapely](https://pypi.org/project/Shapely/) can be used to calculate the intersection. If you are using a Raspberry Pi, you will need to install a library dependency first:
|
||||
|
||||
```sh
|
||||
sudo apt install libgeos-dev
|
||||
```
|
||||
|
||||
1. Install the Shapely Pip package:
|
||||
|
||||
```sh
|
||||
pip3 install shapely
|
||||
```
|
||||
|
||||
If you are using a virtual IoT device, make sure to run this from inside the activated virtual environment.
|
||||
|
||||
1. Add the following import statement to the top of the `app.py` file:
|
||||
|
||||
```python
|
||||
from shapely.geometry import Polygon
|
||||
```
|
||||
|
||||
This imports code needed to create polygons to calculate overlap.
|
||||
|
||||
1. Above the code that draws the bounding boxes, add the following code:
|
||||
|
||||
```python
|
||||
overlap_threshold = 0.20
|
||||
```
|
||||
|
||||
This defines the percentage overlap allowed before the bounding boxes are considered to be the same object. 0.20 defines a 20% overlap.
|
||||
|
||||
1. To calculate overlap using Shapely, the bounding boxes need to be converted into Shapely polygons. Add the following function to do this:
|
||||
|
||||
```python
|
||||
def create_polygon(prediction):
|
||||
scale_left = prediction.bounding_box.left
|
||||
scale_top = prediction.bounding_box.top
|
||||
scale_right = prediction.bounding_box.left + prediction.bounding_box.width
|
||||
scale_bottom = prediction.bounding_box.top + prediction.bounding_box.height
|
||||
|
||||
return Polygon([(scale_left, scale_top), (scale_right, scale_top), (scale_right, scale_bottom), (scale_left, scale_bottom)])
|
||||
```
|
||||
|
||||
This creates a polygon using the bounding box of a prediction.
|
||||
|
||||
1. The logic for removing overlapping objects involves comparing all bounding boxes and if any pairs of predictions have bounding boxes that overlap more than the threshold, delete one of the predictions. To compare all the predictions, you compare prediction 1 with 2, 3, 4, etc., then 2 with 3, 4, etc. The following code does this:
|
||||
|
||||
```python
|
||||
to_delete = []
|
||||
|
||||
for i in range(0, len(predictions)):
|
||||
polygon_1 = create_polygon(predictions[i])
|
||||
|
||||
for j in range(i+1, len(predictions)):
|
||||
polygon_2 = create_polygon(predictions[j])
|
||||
overlap = polygon_1.intersection(polygon_2).area
|
||||
|
||||
smallest_area = min(polygon_1.area, polygon_2.area)
|
||||
|
||||
if overlap > (overlap_threshold * smallest_area):
|
||||
to_delete.append(predictions[i])
|
||||
break
|
||||
|
||||
for d in to_delete:
|
||||
predictions.remove(d)
|
||||
|
||||
print(f'Counted {len(predictions)} stock items')
|
||||
```
|
||||
|
||||
The overlap is calculated using the Shapely `Polygon.intersection` method that returns a polygon that has the overlap. The area is then calculated from this polygon. This overlap threshold is not an absolute value, but needs to be a percentage of the bounding box, so the smallest bounding box is found, and the overlap threshold is used to calculate what area the overlap can be to not exceed the percentage overlap threshold of the smallest bounding box. If the overlap exceeds this, the prediction is marked for deletion.
|
||||
|
||||
Once a prediction has been marked for deletion it doesn't need to be checked again, so the inner loop breaks out to check the next prediction. You can't delete items from a list whilst iterating through it, so the bounding boxes that overlap more than the threshold are added to the `to_delete` list, then deleted at the end.
|
||||
|
||||
Finally the stock count is printed to the console. This could then be sent to an IoT service to alert if the stock levels are low. All of this code is before the bounding boxes are drawn, so you will see the stock predictions without overlaps on the generated images.
|
||||
|
||||
> 💁 This is very simplistic way to remove overlaps, just removing the first one in an overlapping pair. For production code, you would want to put more logic in here, such as considering the overlaps between multiple objects, or if one bounding box is contained by another.
|
||||
|
||||
1. Run the app with the camera pointing at some stock on a shelf. The output will indicate the number of bounding boxes without overlaps that exceed the threshold. Try adjusting the `overlap_threshold` value to see predictions being ignored.
|
||||
|
||||
> 💁 You can find this code in the [code-count/pi](code-count/pi) or [code-count/virtual-iot-device](code-count/virtual-iot-device) folder.
|
||||
|
||||
😀 Your stock counter program was a success!
|
||||
@ -0,0 +1,74 @@
|
||||
# Call your object detector from your IoT device - Virtual IoT Hardware and Raspberry Pi
|
||||
|
||||
Once your object detector has been published, it can be used from your IoT device.
|
||||
|
||||
## Copy the image classifier project
|
||||
|
||||
The majority of your stock detector is the same as the image classifier you created in a previous lesson.
|
||||
|
||||
### Task - copy the image classifier project
|
||||
|
||||
1. Create a folder called `stock-counter` either on your computer if you are using a virtual IoT device, or on your Raspberry Pi. If you are using a virtual IoT device make sure you set up a virtual environment.
|
||||
|
||||
1. Set up the camera hardware.
|
||||
|
||||
* If you are using a Raspberry Pi you will need to fit the PiCamera. You might also want to fix the camera in a single position, for example, by hanging the cable over a box or can, or fixing the camera to a box with double-sided tape.
|
||||
* If you are using a virtual IoT device then you will need to install CounterFit and the CounterFit PyCamera shim. If you are going to use still images, then capture some images that your object detector hasn't seen yet, if you are going to use your web cam make sure it is positioned in a way that can see the stock you are detecting.
|
||||
|
||||
1. Replicate the steps from [lesson 2 of the manufacturing project](../../../4-manufacturing/lessons/2-check-fruit-from-device/README.md#task---capture-an-image-using-an-iot-device) to capture images from the camera.
|
||||
|
||||
1. Replicate the steps from [lesson 2 of the manufacturing project](../../../4-manufacturing/lessons/2-check-fruit-from-device/README.md#task---classify-images-from-your-iot-device) to call the image classifier. The majority of this code will be re-used to detect objects.
|
||||
|
||||
## Change the code from a classifier to an image detector
|
||||
|
||||
The code you used to classify images is very similar to the code to detect objects. The main difference is the method called on the Custom Vision SDK, and the results of the call.
|
||||
|
||||
### Task - change the code from a classifier to an image detector
|
||||
|
||||
1. Delete the three lines of code that classifies the image and processes the predictions:
|
||||
|
||||
```python
|
||||
results = predictor.classify_image(project_id, iteration_name, image)
|
||||
|
||||
for prediction in results.predictions:
|
||||
print(f'{prediction.tag_name}:\t{prediction.probability * 100:.2f}%')
|
||||
```
|
||||
|
||||
Remove these three lines.
|
||||
|
||||
1. Add the following code to detect objects in the image:
|
||||
|
||||
```python
|
||||
results = predictor.detect_image(project_id, iteration_name, image)
|
||||
|
||||
threshold = 0.3
|
||||
|
||||
predictions = list(prediction for prediction in results.predictions if prediction.probability > threshold)
|
||||
|
||||
for prediction in predictions:
|
||||
print(f'{prediction.tag_name}:\t{prediction.probability * 100:.2f}%')
|
||||
```
|
||||
|
||||
This code calls the `detect_image` method on the predictor to run the object detector. It then gathers all the predictions with a probability above a threshold, printing them to the console.
|
||||
|
||||
Unlike an image classifier that only returns one result per tag, the object detector will return multiple results, so any with a low probability need to be filtered out.
|
||||
|
||||
1. Run this code and it will capture an image, sending it to the object detector, and print out the detected objects. If you are using a virtual IoT device ensure you have an appropriate image set in CounterFit, or our web cam is selected. If you are using a Raspberry Pi, make sure your camera is pointing to objects on a shelf.
|
||||
|
||||
```output
|
||||
pi@raspberrypi:~/stock-counter $ python3 app.py
|
||||
tomato paste: 34.13%
|
||||
tomato paste: 33.95%
|
||||
tomato paste: 35.05%
|
||||
tomato paste: 32.80%
|
||||
```
|
||||
|
||||
> 💁 You may need to adjust the `threshold` to an appropriate value for your images.
|
||||
|
||||
You will be able to see the image that was taken, and these values in the **Predictions** tab in Custom Vision.
|
||||
|
||||
|
||||
|
||||
> 💁 You can find this code in the [code-detect/pi](code-detect/pi) or [code-detect/virtual-iot-device](code-detect/virtual-iot-device) folder.
|
||||
|
||||
😀 Your stock counter program was a success!
|
||||
@ -0,0 +1,167 @@
|
||||
# Count stock from your IoT device - Wio Terminal
|
||||
|
||||
A combination of the predictions and their bounding boxes can be used to count stock in an image.
|
||||
|
||||
## Count stock
|
||||
|
||||
|
||||
|
||||
In the image shown above, the bounding boxes have a small overlap. If this overlap was much larger, then the bounding boxes may indicate the same object. To count the objects correctly, you need to ignore boxes with a significant overlap.
|
||||
|
||||
### Task - count stock ignoring overlap
|
||||
|
||||
1. Open your `stock-counter` project if it is not already open.
|
||||
|
||||
1. Above the `processPredictions` function, add the following code:
|
||||
|
||||
```cpp
|
||||
const float overlap_threshold = 0.20f;
|
||||
```
|
||||
|
||||
This defines the percentage overlap allowed before the bounding boxes are considered to be the same object. 0.20 defines a 20% overlap.
|
||||
|
||||
1. Below this, and above the `processPredictions` function, add the following code to calculate the overlap between two rectangles:
|
||||
|
||||
```cpp
|
||||
struct Point {
|
||||
float x, y;
|
||||
};
|
||||
|
||||
struct Rect {
|
||||
Point topLeft, bottomRight;
|
||||
};
|
||||
|
||||
float area(Rect rect)
|
||||
{
|
||||
return abs(rect.bottomRight.x - rect.topLeft.x) * abs(rect.bottomRight.y - rect.topLeft.y);
|
||||
}
|
||||
|
||||
float overlappingArea(Rect rect1, Rect rect2)
|
||||
{
|
||||
float left = max(rect1.topLeft.x, rect2.topLeft.x);
|
||||
float right = min(rect1.bottomRight.x, rect2.bottomRight.x);
|
||||
float top = max(rect1.topLeft.y, rect2.topLeft.y);
|
||||
float bottom = min(rect1.bottomRight.y, rect2.bottomRight.y);
|
||||
|
||||
|
||||
if ( right > left && bottom > top )
|
||||
{
|
||||
return (right-left)*(bottom-top);
|
||||
}
|
||||
|
||||
return 0.0f;
|
||||
}
|
||||
```
|
||||
|
||||
This code defines a `Point` struct to store points on the image, and a `Rect` struct to define a rectangle using a top left and bottom right coordinate. It then defines an `area` function that calculates the area of a rectangle from a top left and bottom right coordinate.
|
||||
|
||||
Next it defines a `overlappingArea` function that calculates the overlapping area of 2 rectangles. If they don't overlap, it returns 0.
|
||||
|
||||
1. Below the `overlappingArea` function, declare a function to convert a bounding box to a `Rect`:
|
||||
|
||||
```cpp
|
||||
Rect rectFromBoundingBox(JsonVariant prediction)
|
||||
{
|
||||
JsonObject bounding_box = prediction["boundingBox"].as<JsonObject>();
|
||||
|
||||
float left = bounding_box["left"].as<float>();
|
||||
float top = bounding_box["top"].as<float>();
|
||||
float width = bounding_box["width"].as<float>();
|
||||
float height = bounding_box["height"].as<float>();
|
||||
|
||||
Point topLeft = {left, top};
|
||||
Point bottomRight = {left + width, top + height};
|
||||
|
||||
return {topLeft, bottomRight};
|
||||
}
|
||||
```
|
||||
|
||||
This takes a prediction from the object detector, extracts the bounding box and uses the values on the bounding box to define a rectangle. The right side is calculated from the left plus the width. The bottom is calculated as the top plus the height.
|
||||
|
||||
1. The predictions need to be compared to each other, and if 2 predictions have an overlap of more that the threshold, one of them needs to be deleted. The overlap threshold is a percentage, so needs to be multiplied by the size of the smallest bounding box to check that the overlap exceeds the given percentage of the bounding box, not the given percentage of the whole image. Start by deleting the content of the `processPredictions` function.
|
||||
|
||||
1. Add the following to the empty `processPredictions` function:
|
||||
|
||||
```cpp
|
||||
std::vector<JsonVariant> passed_predictions;
|
||||
|
||||
for (int i = 0; i < predictions.size(); ++i)
|
||||
{
|
||||
Rect prediction_1_rect = rectFromBoundingBox(predictions[i]);
|
||||
float prediction_1_area = area(prediction_1_rect);
|
||||
bool passed = true;
|
||||
|
||||
for (int j = i + 1; j < predictions.size(); ++j)
|
||||
{
|
||||
Rect prediction_2_rect = rectFromBoundingBox(predictions[j]);
|
||||
float prediction_2_area = area(prediction_2_rect);
|
||||
|
||||
float overlap = overlappingArea(prediction_1_rect, prediction_2_rect);
|
||||
float smallest_area = min(prediction_1_area, prediction_2_area);
|
||||
|
||||
if (overlap > (overlap_threshold * smallest_area))
|
||||
{
|
||||
passed = false;
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
if (passed)
|
||||
{
|
||||
passed_predictions.push_back(predictions[i]);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
This code declares a vector to store the predictions that don't overlap. It then loops through all the predictions, creating a `Rect` from the bounding box.
|
||||
|
||||
Next this code loops through the remaining predictions, starting at the one after the current prediction. This stops predictions being compared more than once - once 1 and 2 have been compared, there's no need to compare 2 with 1, only with 3, 4, etc.
|
||||
|
||||
For each pair of predictions the overlapping area is calculated. This is then compared to the area of the smallest bounding box - if the overlap exceeds the threshold percentage of the smallest bounding box, the prediction is marked as not passed. If after comparing all the overlap, the prediction passes the checks it is added to the `passed_predictions` collection.
|
||||
|
||||
> 💁 This is very simplistic way to remove overlaps, just removing the first one in an overlapping pair. For production code, you would want to put more logic in here, such as considering the overlaps between multiple objects, or if one bounding box is contained by another.
|
||||
|
||||
1. After this, add the following code to send details of the passed predictions to the serial monitor:
|
||||
|
||||
```cpp
|
||||
for(JsonVariant prediction : passed_predictions)
|
||||
{
|
||||
String boundingBox = prediction["boundingBox"].as<String>();
|
||||
String tag = prediction["tagName"].as<String>();
|
||||
float probability = prediction["probability"].as<float>();
|
||||
|
||||
char buff[32];
|
||||
sprintf(buff, "%s:\t%.2f%%\t%s", tag.c_str(), probability * 100.0, boundingBox.c_str());
|
||||
Serial.println(buff);
|
||||
}
|
||||
```
|
||||
|
||||
This code loops through the passed predictions and prints their details to the serial monitor.
|
||||
|
||||
1. Below this, add code to print the number of counted items to the serial monitor:
|
||||
|
||||
```cpp
|
||||
Serial.print("Counted ");
|
||||
Serial.print(passed_predictions.size());
|
||||
Serial.println(" stock items.");
|
||||
```
|
||||
|
||||
This could then be sent to an IoT service to alert if the stock levels are low.
|
||||
|
||||
1. Upload and run your code. Point the camera at objects on a shelf and press the C button. Try adjusting the `overlap_threshold` value to see predictions being ignored.
|
||||
|
||||
```output
|
||||
Connecting to WiFi..
|
||||
Connected!
|
||||
Image captured
|
||||
Image read to buffer with length 17416
|
||||
tomato paste: 35.84% {"left":0.395631,"top":0.215897,"width":0.180768,"height":0.359364}
|
||||
tomato paste: 35.87% {"left":0.378554,"top":0.583012,"width":0.14824,"height":0.359382}
|
||||
tomato paste: 34.11% {"left":0.699024,"top":0.592617,"width":0.124411,"height":0.350456}
|
||||
tomato paste: 35.16% {"left":0.513006,"top":0.647853,"width":0.187472,"height":0.325817}
|
||||
Counted 4 stock items.
|
||||
```
|
||||
|
||||
> 💁 You can find this code in the [code-count/wio-terminal](code-count/wio-terminal) folder.
|
||||
|
||||
😀 Your stock counter program was a success!
|
||||
@ -0,0 +1,102 @@
|
||||
# Call your object detector from your IoT device - Wio Terminal
|
||||
|
||||
Once your object detector has been published, it can be used from your IoT device.
|
||||
|
||||
## Copy the image classifier project
|
||||
|
||||
The majority of your stock detector is the same as the image classifier you created in a previous lesson.
|
||||
|
||||
### Task - copy the image classifier project
|
||||
|
||||
1. Connect your ArduCam your Wio Terminal, following the steps from [lesson 2 of the manufacturing project](../../../4-manufacturing/lessons/2-check-fruit-from-device/wio-terminal-camera.md#task---connect-the-camera).
|
||||
|
||||
You might also want to fix the camera in a single position, for example, by hanging the cable over a box or can, or fixing the camera to a box with double-sided tape.
|
||||
|
||||
1. Create a brand new Wio Terminal project using PlatformIO. Call this project `stock-counter`.
|
||||
|
||||
1. Replicate the steps from [lesson 2 of the manufacturing project](../../../4-manufacturing/lessons/2-check-fruit-from-device/README.md#task---capture-an-image-using-an-iot-device) to capture images from the camera.
|
||||
|
||||
1. Replicate the steps from [lesson 2 of the manufacturing project](../../../4-manufacturing/lessons/2-check-fruit-from-device/README.md#task---classify-images-from-your-iot-device) to call the image classifier. The majority of this code will be re-used to detect objects.
|
||||
|
||||
## Change the code from a classifier to an image detector
|
||||
|
||||
The code you used to classify images is very similar to the code to detect objects. The main difference is the URL that is called that you obtained from Custom Vision, and the results of the call.
|
||||
|
||||
### Task - change the code from a classifier to an image detector
|
||||
|
||||
1. Add the following include directive to the top of the `main.cpp` file:
|
||||
|
||||
```cpp
|
||||
#include <vector>
|
||||
```
|
||||
|
||||
1. Rename the `classifyImage` function to `detectStock`, both the name of the function and the call in the `buttonPressed` function.
|
||||
|
||||
1. Above the `detectStock` function, declare a threshold to filter out any detections that have a low probability:
|
||||
|
||||
```cpp
|
||||
const float threshold = 0.3f;
|
||||
```
|
||||
|
||||
Unlike an image classifier that only returns one result per tag, the object detector will return multiple results, so any with a low probability need to be filtered out.
|
||||
|
||||
1. Above the `detectStock` function, declare a function to process the predictions:
|
||||
|
||||
```cpp
|
||||
void processPredictions(std::vector<JsonVariant> &predictions)
|
||||
{
|
||||
for(JsonVariant prediction : predictions)
|
||||
{
|
||||
String tag = prediction["tagName"].as<String>();
|
||||
float probability = prediction["probability"].as<float>();
|
||||
|
||||
char buff[32];
|
||||
sprintf(buff, "%s:\t%.2f%%", tag.c_str(), probability * 100.0);
|
||||
Serial.println(buff);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
This takes a list of predictions and prints them to the serial monitor.
|
||||
|
||||
1. In the `detectStock` function, replace the contents of the `for` loop that loops through the predictions with the following:
|
||||
|
||||
```cpp
|
||||
std::vector<JsonVariant> passed_predictions;
|
||||
|
||||
for(JsonVariant prediction : predictions)
|
||||
{
|
||||
float probability = prediction["probability"].as<float>();
|
||||
if (probability > threshold)
|
||||
{
|
||||
passed_predictions.push_back(prediction);
|
||||
}
|
||||
}
|
||||
|
||||
processPredictions(passed_predictions);
|
||||
```
|
||||
|
||||
This loops through the predictions, comparing the probability to the threshold. All predictions that have a probability higher than the threshold are added to a `list` and passed to the `processPredictions` function.
|
||||
|
||||
1. Upload and run your code. Point the camera at objects on a shelf and press the C button. You will see the output in the serial monitor:
|
||||
|
||||
```output
|
||||
Connecting to WiFi..
|
||||
Connected!
|
||||
Image captured
|
||||
Image read to buffer with length 17416
|
||||
tomato paste: 35.84%
|
||||
tomato paste: 35.87%
|
||||
tomato paste: 34.11%
|
||||
tomato paste: 35.16%
|
||||
```
|
||||
|
||||
> 💁 You may need to adjust the `threshold` to an appropriate value for your images.
|
||||
|
||||
You will be able to see the image that was taken, and these values in the **Predictions** tab in Custom Vision.
|
||||
|
||||
|
||||
|
||||
> 💁 You can find this code in the [code-detect/wio-terminal](code-detect/wio-terminal) folder.
|
||||
|
||||
😀 Your stock counter program was a success!
|
||||
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in new issue