@ -261,6 +261,7 @@ The challenge in the last lesson was to list as many IoT devices as you can that
* Read the [Arduino getting started guide](https://www.arduino.cc/en/Guide/Introduction) to understand more about the Arduino platform.
* Read the [introduction to the Raspberry Pi 4](https://www.raspberrypi.org/products/raspberry-pi-4-model-b/) to learn more about Raspberry Pis.
* Learn more on some of the concepts and acronyms in the [What the FAQ are CPUs, MPUs, MCUs, and GPUs article in the Electrical Engineering Journal](https://www.eejournal.com/article/what-the-faq-are-cpus-mpus-mcus-and-gpus/).
✅ Use these guides, along with the costs shown by following the links in the [hardware guide](../../../hardware.md) to decide on what hardware platform you want to use, or if you would rather use a virtual device.
if response_json['RecognitionStatus'] == 'Success':
return response_json['DisplayText']
@ -84,11 +83,18 @@ The audio can be sent to the speech service using the REST API. To use the speec
This calls the URL and decodes the JSON value that comes in the response. The `RecognitionStatus` value in the response indicates if the call was able to extract speech into text successfully, and if this is `Success` then the text is returned from the function, otherwise an empty string is returned.
1. Finally replace the call to `play_audio` in the `while True` loop with a call to the `convert_speech_to_text` function, as well as printing the text to the console:
1. Above the `while True:` loop, define a function to process the text returned from the speech to text service. This function will just print the text to the console for now.
```python
def process_text(text):
print(text)
```
1. Finally replace the call to `play_audio` in the `while True` loop with a call to the `convert_speech_to_text` function, passing the text to the `process_text` function:
```python
text = convert_speech_to_text(buffer)
print(text)
process_text(text)
```
1. Run the code. Press the button and speak into the microphone. Release the button when you are done, and the audio will be converted to text and printed to the console.
1. The speech recognizer runs on a background thread, listening for audio and converting any speech in it to text. You can get the text using a callback function - a function you define and pass to the recognizer. Every time speech is detected, the callback is called. Add the following code to define a callback that prints the text to the console, and pass this callback to the recognizer:
1. The speech recognizer runs on a background thread, listening for audio and converting any speech in it to text. You can get the text using a callback function - a function you define and pass to the recognizer. Every time speech is detected, the callback is called. Add the following code to define a callback, and pass this callback to the recognizer, as well as defining a function to process the text, writing it to the consoled:
@ -260,7 +260,7 @@ Once published, the LUIS model can be called from code. In previous lessons, you
For a smart timer, we want a response straight away, so we can tell the user that a timer is set, or alert them that the cloud services are unavailable. To do this, our IoT device will call a web endpoint directly, instead of relying on an IoT Hub.
Rather than calling LUIS from the IoT device, you can use serverless code with a different type of trigger - an HTTP trigger. This allows your function app to listen for REST requests, and respond to them.
Rather than calling LUIS from the IoT device, you can use serverless code with a different type of trigger - an HTTP trigger. This allows your function app to listen for REST requests, and respond to them. This function will be a REST endpoint your device can call.
> 💁 Although you can call LUIS directly from your IoT device, it's better to use something like serverless code. This way when of you want to change the LUIS app that you call, for example when you train a better model or train a model in a different language, you only have to update your cloud code, not re-deploy code to potentially thousands or millions of IoT device.
@ -467,6 +467,25 @@ Rather than calling LUIS from the IoT device, you can use serverless code with a
> 💁 You can find this code in the [code/functions](code/functions) folder.
### Task - make your function available to your IoT device
1. For your IoT device to call your REST endpoint, it will need to know the URL. When you accessed it earlier, you used `localhost`, which is a shortcut to access REST endpoints on your local machine. To allow you IoT device to get access, you need to either:
* Publish the Functions app - follow the instructions in earlier lessons to publish your functions app to the cloud. Once published, the URL will be `http://<APP_NAME>.azurewebsites.net/api/text-to-timer`, where `<APP_NAME>` will be the name of your functions app.
* Run the functions app locally, and access using the IP address - you can get the IP address of your computer on your local network, and use that to build the URL.
Find your IP address:
* On Windows 10, follow the [Find your IP address guide](https://support.microsoft.com/windows/find-your-ip-address-f21a9bbc-c582-55cd-35e0-73431160a1b9?WT.mc_id=academic-17441-jabenn)
* On macOS, follow the [How to find you IP address on a Mac guide](https://www.hellotech.com/guide/for/how-to-find-ip-address-on-mac)
* On linux, follow the section on finding your private IP address in the [How to find your IP address in Linux guide](https://opensource.com/article/18/5/how-find-ip-address-linux)
Once you have your IP address, you will able to access the function at `http://<IP_ADDRESS>:7071/api/text-to-timer`, where `<IP_ADDRESS>` will be your IP address, for example `http://192.168.1.10:7071/api/text-to-timer`.
> 💁 This will only work if your IoT device is on the same network as your computer.
1. Test the endpoint by accessing it using your browser.
@ -72,34 +72,11 @@ These large ML models are being trained to combine all three steps into end-to-e
## Set the timer
The timer can be set by sending a command from the serverless code, instructing the IoT device to set the timer. This command will contain the time in seconds till the timer needs to go off.
To set the timer, your IoT device needs to call the REST endpoint you created using serverless code, then use the resulting number of seconds to set a timer.
### Task - set the timer using a command
### Task - call the serverless function to get the timer time
1. In your serverless code, add code to send a direct method request to your IoT device
> ⚠️ You can refer to [the instructions for sending direct method requests in lesson 5 of the farm project if needed](../../../2-farm/lessons/5-migrate-application-to-the-cloud/README.md#send-direct-method-requests-from-serverless-code).
You will need to set up the connection string for the IoT Hub with the service policy (*NOT* the device) in your `local.settings.json` file and add the `azure-iot-hub` pip package to your `requirements.txt` file. The device ID can be extracted from the event.
1. The direct method you send needs to be called `set-timer`, and will need to send the length of the timer as a JSON property called `seconds`. Use the following code to build the `CloudToDeviceMethod` using the `total_seconds` calculated from the data extracted by LUIS:
> 💁 You can find this code in the [code-command/functions](code-command/functions) folder.
### Task - respond to the command on the IoT device
1. On your IoT device, respond to the command.
> ⚠️ You can refer to [the instructions for handling direct method requests from IoT devices in lesson 4 of the farm project if needed](../../../2-farm/lessons/4-migrate-your-plant-to-the-cloud#task---connect-your-iot-device-to-the-cloud).
1. Work through the relevant guide to set a timer for the required time:
Follow the relevant guide to call the REST endpoint from your IoT device and set a timer for the required time:
@ -4,21 +4,59 @@ In this part of the lesson, you will set a timer on your virtual IoT device or R
## Set a timer
The command sent from the serverless function contains the time for the timer in seconds as the payload. This time can be used to set a timer.
The text that comes back from the speech to text call needs to be sent to your serverless code to be processed by LUIS, getting back the number of seconds for the timer. This number of seconds can be used to set a timer.
Timers can be set using the Python `threading.Timer` class. This class takes a delay time and a function, and after the delay time, the function is executed.
### Task - set a timer
### Task - send the text to the serverless function
1. Open the `smart-timer` project in VS Code, and ensure the virtual environment is loaded in the terminal if you are using a virtual IoT device.
1. Above the `process_text` function, declare a function called `get_timer_time` to call the REST endpoint you created:
```python
def get_timer_time(text):
```
1. Add the following code to this function to define the URL to call:
```python
url = '<URL>'
```
Replace `<URL>` with the URL of your rest endpoint that you built in the last lesson, either on your computer or in the cloud.
1. Add the following code to set the text as a parameter on the URL and make the API call:
```python
params = {
'text': text
}
response = requests.post(url, params=params)
```
1. Below this, retrieve the `seconds` from the response payload, returning 0 if the call failed:
```python
if response.status_code != 200:
return 0
payload = response.json()
return payload['seconds']
```
Successful HTTP calls return a status code in the 200 range, and your serverless code returns 200 if the text was processed and recognized as the set timer intent.
### Task - set a timer on a background thread
1. Add the following import statement at the top of the file to import the threading Python library:
```python
import threading
```
1. Above the `handle_method_request` function that handles the method request, add a function to speak a response. Fow now this will just write to the console, but later in this lesson this will speak the text.
1. Above the `process_text` function, add a function to speak a response. Fow now this will just write to the console, but later in this lesson this will speak the text.
```python
def say(text):
@ -43,9 +81,9 @@ Timers can be set using the Python `threading.Timer` class. This class takes a d
1. Below this, add the following `create_timer` function to create a timer:
This function takes the total number of seconds for the timer that will be sent in the command, and converts this to minutes and seconds. It then creates and starts a timer object using the total number of seconds, passing in the `announce_timer` function and a list containing the minutes and seconds. When the timer elapses, it will call the `announce_timer` function, and pass the contents of this list as the parameters - so the first item in the list gets passes as the `minutes` parameter, and the second item as the `seconds` parameter.
@ -64,32 +102,23 @@ Timers can be set using the Python `threading.Timer` class. This class takes a d
Again, this only includes the time unit that has a value. This sentence is then sent to the `say` function.
1. At the start of the `handle_method_request` function, add the following code to check that the `set-timer` direct method was requested:
```python
if request.name == 'set-timer':
```
1. Inside this `if` statement, extract the timer time in seconds from the payload and use this to create a timer:
1. Add the following to the end of the `process_text` function to get the time for the timer from the text, then create the timer:
```python
payload = json.loads(request.payload)
seconds = payload['seconds']
seconds = get_timer_time(text)
if seconds > 0:
create_timer(payload['seconds'])
create_timer(seconds)
```
The timer is only created if the number of seconds is greater than 0
The timer is only created if the number of seconds is greater than 0.
1. Run the app, and ensure the function app is also running. Set some timers, and the output will show the timer being set, and then will show when it elapses:
```output
pi@raspberrypi:~/smart-timer $ python3 app.py
Connecting
Connected
Set a one minute 4 second timer.
1 minute 4 second timer started.
Times up on your 1 minute 4 second timer.
Set a two minute 27 second timer.
2 minute 27 second timer started.
Times up on your 2 minute 27 second timer.
```
> 💁 You can find this code in the [code-timer/pi](code-timer/pi) or [code-timer/virtual-iot-device](code-timer/virtual-iot-device) folder.
@ -22,7 +22,7 @@ The projects cover the journey of food from farm to table. This includes farming
**Hearty thanks to our authors [Jen Fox](https://github.com/jenfoxbot), [Jen Looper](https://github.com/jlooper), [Jim Bennett](https://github.com/jimbobbennett), and our sketchnote artist [Nitya Narasimhan](https://github.com/nitya).**
**Thanks as well to our team of [Microsoft Learn Student Ambassadors](https://studentambassadors.microsoft.com?WT.mc_id=academic-17441-jabenn) who have been reviewing and translating this curriculum - [Aditya Garg](https://github.com/AdityaGarg00), [Aryan Jain](https://www.linkedin.com/in/aryan-jain-47a4a1145/), [Bhavesh Suneja](https://github.com/EliteWarrior315), [Lateefah Bello](https://www.linkedin.com/in/lateefah-bello/), [Manvi Jha](https://github.com/Severus-Matthew), [Mireille Tan](https://www.linkedin.com/in/mireille-tan-a4834819a/), [Mohammad Iftekher (Iftu) Ebne Jalal](https://github.com/Iftu119), [Priyanshu Srivastav](https://www.linkedin.com/in/priyanshu-srivastav-b067241ba), [Thanmai Gowducheruvu](https://github.com/innovation-platform), and [Zina Kamel](https://www.linkedin.com/in/zina-kamel/).**
**Thanks as well to our team of [Microsoft Learn Student Ambassadors](https://studentambassadors.microsoft.com?WT.mc_id=academic-17441-jabenn) who have been reviewing and translating this curriculum - [Aditya Garg](https://github.com/AdityaGarg00), [Arpita Das](https://github.com/Arpiiitaaa), [Aryan Jain](https://www.linkedin.com/in/aryan-jain-47a4a1145/), [Bhavesh Suneja](https://github.com/EliteWarrior315), [Lateefah Bello](https://www.linkedin.com/in/lateefah-bello/), [Manvi Jha](https://github.com/Severus-Matthew), [Mireille Tan](https://www.linkedin.com/in/mireille-tan-a4834819a/), [Mohammad Iftekher (Iftu) Ebne Jalal](https://github.com/Iftu119), [Priyanshu Srivastav](https://www.linkedin.com/in/priyanshu-srivastav-b067241ba), [Thanmai Gowducheruvu](https://github.com/innovation-platform), and [Zina Kamel](https://www.linkedin.com/in/zina-kamel/).**
> **Teachers**, we have [included some suggestions](for-teachers.md) on how to use this curriculum. If you would like to create your own lessons, we have also included a [lesson template](lesson-template/README.md).
Jim Bennett
4 years ago