Adding functions lab to lesson 22

2-farm/lessons/5-migrate-application-to-the-cloud/README.md

@@ -254,12 +254,12 @@ You are now ready to create the event trigger.
 1. From the VS Code terminal run the following command from inside the `soil-moisture-trigger` folder:
 
     ```sh
-    func new --name iot_hub_trigger --template "Azure Event Hub trigger"
+    func new --name iot-hub-trigger --template "Azure Event Hub trigger"
     ```
 
-    This creates a new Function called `iot_hub_trigger`. The trigger will connect to the Event Hub compatible endpoint on the IoT Hub, so you can use an event hub trigger. There is no specific IoT Hub trigger.
+    This creates a new Function called `iot-hub-trigger`. The trigger will connect to the Event Hub compatible endpoint on the IoT Hub, so you can use an event hub trigger. There is no specific IoT Hub trigger.
 
-This will create a folder inside the `soil-moisture-trigger` folder called `iot_hub_trigger` that contains this function. This folder will have the following files inside it:
+This will create a folder inside the `soil-moisture-trigger` folder called `iot-hub-trigger` that contains this function. This folder will have the following files inside it:
 
 * `__init__.py` - this is the Python code file that contains the trigger, using the standard Python file name convention to turn this folder into a Python module.
 
@@ -313,7 +313,7 @@ This will create a folder inside the `soil-moisture-trigger` folder called `iot_
     func start
     ```
 
-    The Functions app will start up, and will discover the `iot_hub_trigger` function. It will then process any events that have already been sent to the IoT Hub in the past day.
+    The Functions app will start up, and will discover the `iot-hub-trigger` function. It will then process any events that have already been sent to the IoT Hub in the past day.
 
     ```output
     (.venv) ➜  soil-moisture-trigger func start
@@ -325,23 +325,23 @@ This will create a folder inside the `soil-moisture-trigger` folder called `iot_
     
     Functions:
     
-            iot_hub_trigger: eventHubTrigger
+            iot-hub-trigger: eventHubTrigger
     
     For detailed output, run func with --verbose flag.
     [2021-05-05T02:44:07.517Z] Worker process started and initialized.
-    [2021-05-05T02:44:09.202Z] Executing 'Functions.iot_hub_trigger' (Reason='(null)', Id=802803a5-eae9-4401-a1f4-176631456ce4)
+    [2021-05-05T02:44:09.202Z] Executing 'Functions.iot-hub-trigger' (Reason='(null)', Id=802803a5-eae9-4401-a1f4-176631456ce4)
     [2021-05-05T02:44:09.205Z] Trigger Details: PartionId: 0, Offset: 1011240-1011632, EnqueueTimeUtc: 2021-05-04T19:04:04.2030000Z-2021-05-04T19:04:04.3900000Z, SequenceNumber: 2546-2547, Count: 2
     [2021-05-05T02:44:09.352Z] Python EventHub trigger processed an event: {"soil_moisture":628}
     [2021-05-05T02:44:09.354Z] Python EventHub trigger processed an event: {"soil_moisture":624}
-    [2021-05-05T02:44:09.395Z] Executed 'Functions.iot_hub_trigger' (Succeeded, Id=802803a5-eae9-4401-a1f4-176631456ce4, Duration=245ms)
+    [2021-05-05T02:44:09.395Z] Executed 'Functions.iot-hub-trigger' (Succeeded, Id=802803a5-eae9-4401-a1f4-176631456ce4, Duration=245ms)
     ```
 
-    Each call to the function will be surrounded by a `Executing 'Functions.iot_hub_trigger'`/`Executed 'Functions.iot_hub_trigger'` block in the output, so you can how many messages were processed in each function call.
+    Each call to the function will be surrounded by a `Executing 'Functions.iot-hub-trigger'`/`Executed 'Functions.iot-hub-trigger'` block in the output, so you can how many messages were processed in each function call.
 
     > If you get the following error:
 
       ```output
-      The listener for function 'Functions.iot_hub_trigger' was unable to start. Microsoft.WindowsAzure.Storage: Connection refused. System.Net.Http: Connection refused. System.Private.CoreLib: Connection refused.
+      The listener for function 'Functions.iot-hub-trigger' was unable to start. Microsoft.WindowsAzure.Storage: Connection refused. System.Net.Http: Connection refused. System.Private.CoreLib: Connection refused.
       ```
 
       Then check Azurite is running and you have set the `AzureWebJobsStorage` in the `local.settings.json` file to `UseDevelopmentStorage=true`.
@@ -561,7 +561,7 @@ Deployment successful.
 Remote build succeeded!
 Syncing triggers...
 Functions in soil-moisture-sensor:
-    iot_hub_trigger - [eventHubTrigger]
+    iot-hub-trigger - [eventHubTrigger]
 ```
 
 Make sure your IoT device is running. Change the moisture levels by adjusting the soil moisture, or moving the sensor in and out of the soil. You will see the relay turn on and off as the soil moisture changes.

2-farm/lessons/5-migrate-application-to-the-cloud/assignment.md

@@ -35,7 +35,7 @@ Some hints:
 
         relay_on: [GET,POST] http://localhost:7071/api/relay_on
 
-        iot_hub_trigger: eventHubTrigger
+        iot-hub-trigger: eventHubTrigger
     ```
 
     Paste the URL into your browser and hit `return`, or `Ctrl+click` (`Cmd+click` on macOS) the link in the terminal window in VS Code to open it in your default browser. This will run the trigger.

3-transport/lessons/2-store-location-data/README.md

@@ -158,7 +158,7 @@ Once data is flowing into your IoT Hub, you can write some serverless code to li
 
 1. Use the Azurite app as a local storage emulator
 
-Run your functions app to ensure it is receiving events from your GPS device. Make sure your IoT device is also running and sending GPS data.
+1. Run your functions app to ensure it is receiving events from your GPS device. Make sure your IoT device is also running and sending GPS data.
 
     ```output
     Python EventHub trigger processed an event: {"gps": {"lat": 47.73481, "lon": -122.25701}}
@@ -258,7 +258,7 @@ The data will be saved as a JSON blob with the following format:
     > pip install --upgrade pip
     > ```
 
-1. In the `__init__.py` file for the `iot_hub_trigger`, add the following import statements:
+1. In the `__init__.py` file for the `iot-hub-trigger`, add the following import statements:
 
     ```python
     import json

6-consumer/README.md

@@ -1,8 +1,8 @@
 # Consumer IoT - build a smart voice assistant
 
-The fod has been grown, driven to a processing plant, sorted for quality, sold in the store and now it's time to cook! One of the core pieces of any kitchen is a timer. Initially these started as simple hour glasses - your food was cooked when all the sand trickled down into the bottom bulb. They then went clockwork, then electric.
+The fod has been grown, driven to a processing plant, sorted for quality, sold in the store and now it's time to cook! One of the core pieces of any kitchen is a timer. Initially these started as hour glasses - your food was cooked when all the sand trickled down into the bottom bulb. They then went clockwork, then electric.
 
-The latest iterations are now part of our smart devices. In kitchens all throughout the world you'll head chefs shouting "Hey Siri - set a 10 minute timer", or "Alexa - cancel my bread timer". No longer do you have to walk back to the kitchen to check on a timer, you can do it from your phone, or a call out across the room.
+The latest iterations are now part of our smart devices. In kitchens in homes all throughout the world you'll hear cooks shouting "Hey Siri - set a 10 minute timer", or "Alexa - cancel my bread timer". No longer do you have to walk back to the kitchen to check on a timer, you can do it from your phone, or a call out across the room.
 
 In these 4 lessons you'll learn how to build a smart timer, using AI to recognize your voice, understand what you are asking for, and reply with information about your timer. You'll also add support for multiple languages.
 
@@ -12,7 +12,7 @@ In these 4 lessons you'll learn how to build a smart timer, using AI to recogniz
 
 1. [Recognize speech with an IoT device](./lessons/1-speech-recognition/README.md)
 1. [Understand language](./lessons/2-language-understanding/README.md)
-1. [Provide spoken feedback](./lessons/3-spoken-feedback/README.md)
+1. [Set a timer and provide spoken feedback](./lessons/3-spoken-feedback/README.md)
 1. [Support multiple languages](./lessons/4-multiple-language-support/README.md)
 
 ## Credits

6-consumer/lessons/1-speech-recognition/README.md

@@ -194,7 +194,7 @@ To use the results of the speech to text conversion, you need to send it to the
     }
     ```
 
-    Where `<converted speech>` is the output from the speech to text call.
+    Where `<converted speech>` is the output from the speech to text call. You only need to send speech that has content, if the call returns an empty string it can be ignored.
 
 1. Verify that messages are being sent by monitoring the Event Hub compatible endpoint using the `az iot hub monitor-events` command.
 

6-consumer/lessons/1-speech-recognition/code-iot-hub/pi/smart-timer/app.py

@@ -10,14 +10,6 @@ from azure.iot.device import IoTHubDeviceClient, Message
 from grove.factory import Factory
 button = Factory.getButton('GPIO-HIGH', 5)
 
-connection_string = '<connection_string>'
-
-device_client = IoTHubDeviceClient.create_from_connection_string(connection_string)
-
-print('Connecting')
-device_client.connect()
-print('Connected')
-
 audio = pyaudio.PyAudio()
 microphone_card_number = 1
 speaker_card_number = 1
@@ -52,6 +44,13 @@ def capture_audio():
 api_key = '<key>'
 location = '<location>'
 language = '<language>'
+connection_string = '<connection_string>'
+
+device_client = IoTHubDeviceClient.create_from_connection_string(connection_string)
+
+print('Connecting')
+device_client.connect()
+print('Connected')
 
 def get_access_token():
     headers = {

6-consumer/lessons/2-language-understanding/README.md

@@ -220,14 +220,173 @@ To use this model from code, you need to publish it. When publishing from LUIS,
     }
     ```
 
-    The JSON above came from querying with `set a timer for 45 minutes and 12 seconds`, The `set timer` was the top intent with a probability of 97%. Two *number* entities were detected, 45 and 12. Two *time-unit* entities were detected, `minute` and `second`.
+    The JSON above came from querying with `set a timer for 45 minutes and 12 seconds`:
+
+    * The `set timer` was the top intent with a probability of 97%.
+    * Two *number* entities were detected, `45` and `12`.
+    * Two *time-unit* entities were detected, `minute` and `second`.
 
 ## Use the language understanding model
 
+Once published, the LUIS model can be called from code. In the last lesson you sent the recognized speech to an IoT Hub, and you can use serverless code to respond to this and understand what was sent.
+
 ### Task - create a serverless functions app
 
+1. Create an Azure Functions app called `smart-timer-trigger`.
+
+1. Add an IoT Hub event trigger to this app called `speech-trigger`.
+
+1. Set the Event Hub compatible endpoint connection string for your IoT Hub in the `local.settings.json` file, and use the key for that entry in the `function.json` file.
+
+1. Use the Azurite app as a local storage emulator.
+
+1. Run your functions app and your IoT device to ensure speech is arriving at the IoT Hub.
+
+    ```output
+    Python EventHub trigger processed an event: {"speech": "Set a 3 minute timer."}
+    ```
+
 ### Task - use the language understanding model
 
+1. The SDK for LUIS is available via a Pip package. Add the following line to the `requirements.txt` file to add the dependency on this package:
+
+    ```sh
+    azure-cognitiveservices-language-luis
+    ```
+
+1. Make sure the VS Code terminal has the virtual environment activated, and run the following command to install the Pip packages:
+
+    ```sh
+    pip install -r requirements.txt
+    ```
+
+1. Add new entries to the `local.settings.json` file for your LUIS API Key, Endpoint URL, and App ID from the **MANAGE** tab of the LUIS portal:
+
+    ```JSON
+    "LUIS_KEY": "<primary key>",
+    "LUIS_ENDPOINT_URL": "<endpoint url>",
+    "LUIS_APP_ID": "<app id>"
+    ```
+
+    Replace `<endpoint url>` with the Endpoint URL from the *Azure Resources* section of the **MANAGE** tab. This will be `https://<location>.api.cognitive.microsoft.com/`.
+
+    Replace `<app id>` with the App ID from the *Settings* section of the **MANAGE** tab.
+
+    Replace `<primary key>` with the Primary Key from the *Azure Resources* section of the **MANAGE** tab.
+
+1. Add the following imports to the `__init__.py` file:
+
+    ```python
+    import json
+    import os
+    from azure.cognitiveservices.language.luis.runtime import LUISRuntimeClient
+    from msrest.authentication import CognitiveServicesCredentials
+    ```
+
+    This imports some system libraries, as well as the libraries to interact with LUIS.
+
+1. In the `main` method, before it loops through all the events, add the following code:
+
+    ```python
+    luis_key = os.environ['LUIS_KEY']
+    endpoint_url = os.environ['LUIS_ENDPOINT_URL']
+    app_id = os.environ['LUIS_APP_ID']
+    
+    credentials = CognitiveServicesCredentials(luis_key)
+    client = LUISRuntimeClient(endpoint=endpoint_url, credentials=credentials)
+    ```
+
+    This loads the values you added to the `local.settings.json` file for your LUIS app, creates a credentials object with your API key, then creates a LUIS client object to interact with your LUIS app.
+
+1. Predictions are requested from LUIS by sending a prediction request - a JSON document containing the text to predict. Create this with the following code inside the `for event in events` loop:
+
+    ```python
+    event_body = json.loads(event.get_body().decode('utf-8'))
+    prediction_request = { 'query' : event_body['speech'] }
+    ```
+
+    This code extracts the speech that was sent to the IoT Hub and uses it to build the prediction request.
+
+1. This request can then be sent to LUIS, using the staging slot that your app was published to:
+
+    ```python
+    prediction_response = client.prediction.get_slot_prediction(app_id, 'Staging', prediction_request)
+    ```
+
+1. The prediction response contains the top intent - the intent with the highest prediction score, along with the entities. If the top intent is `set timer`, then the entities can be read to get the time needed for the timer:
+
+    ```python
+    if prediction_response.prediction.top_intent == 'set timer':
+        numbers = prediction_response.prediction.entities['number']
+        time_units = prediction_response.prediction.entities['time unit']
+        total_time = 0
+    ```
+
+    The `number` entities wil be an array of numbers. For example, if you said *"Set a four minute 17 second timer."*, then the `number` array will contain 2 integers - 4 and 17.
+
+    The `time unit` entities will be an array of arrays of strings, with each time unit as an array of strings inside the array. For example, if you said *"Set a four minute 17 second timer."*, then the `time unit` array will contain 2 arrays with single values each - `['minute']` and `['second']`.
+
+    The JSON version of these entities for *"Set a four minute 17 second timer."* is:
+
+    ```json
+    {
+        "number": [4, 17],
+        "time unit": [
+            ["minute"],
+            ["second"]
+        ]
+    }
+    ```
+
+    This code also defines a count for the total time for the timer in seconds. This will be populated by the values from the entities.
+
+1. The entities aren't linked, but we can make some assumptions about them. They will be in the order spoken, so the position in the array can be used to determine which number matches to which time unit. For example:
+
+    * *"Set a 30 second timer"* - this will have one number, `30`, and one time unit, `second` so the single number will match the single time unit.
+    * *"Set a 2 minute and 30 second timer"* - this will have two numbers, `2` and `30`, and two time units, `minute` and `second` so the first number will be for the first time unit (2 minutes), and the second number for the second time unit (30 seconds).
+
+    The following code gets the count of items in the number entities, and uses that to extract the first item from each array, then the second and so on:
+
+    ```python
+    for i in range(0, len(numbers)):
+        number = numbers[i]
+        time_unit = time_units[i][0]
+    ```
+
+    For *"Set a four minute 17 second timer."*, this will loop twice, giving the following values:
+
+    | loop count | `number` | `time_unit` |
+    | ---------: | -------: | ----------- |
+    | 0          | 4        | minute      |
+    | 1          | 17       | second      |
+
+1. Inside this loop, use the number and time unit to calculate the total time for the timer, adding 60 seconds for each minute, and the number of seconds for any seconds.
+
+    ```python
+    if time_unit == 'minute':
+        total_time += number * 60
+    else:
+        total_time += number
+    ```
+
+1. Finally, outside this loop through the entities, log the total time for the timer:
+
+    ```python
+    logging.info(f'Timer required for {total_time} seconds')
+    ```
+
+1. Run the function app and speak into your IoT device. You will see the total time for the timer in the function app output:
+
+    ```output
+    [2021-06-16T01:38:33.316Z] Executing 'Functions.speech-trigger' (Reason='(null)', Id=39720c37-b9f1-47a9-b213-3650b4d0b034)
+    [2021-06-16T01:38:33.329Z] Trigger Details: PartionId: 0, Offset: 3144-3144, EnqueueTimeUtc: 2021-06-16T01:38:32.7970000Z-2021-06-16T01:38:32.7970000Z, SequenceNumber: 8-8, Count: 1
+    [2021-06-16T01:38:33.605Z] Python EventHub trigger processed an event: {"speech": "Set a four minute 17 second timer."}
+    [2021-06-16T01:38:35.076Z] Timer required for 257 seconds
+    [2021-06-16T01:38:35.128Z] Executed 'Functions.speech-trigger' (Succeeded, Id=39720c37-b9f1-47a9-b213-3650b4d0b034, Duration=1894ms)
+    ```
+
+> 💁 You can find this code in the [code/functions](code/functions) folder.
+
 ---
 
 ## 🚀 Challenge

6-consumer/lessons/2-language-understanding/code/functions/smart-timer-trigger/host.json

@@ -0,0 +1,15 @@
+{
+  "version": "2.0",
+  "logging": {
+    "applicationInsights": {
+      "samplingSettings": {
+        "isEnabled": true,
+        "excludedTypes": "Request"
+      }
+    }
+  },
+  "extensionBundle": {
+    "id": "Microsoft.Azure.Functions.ExtensionBundle",
+    "version": "[2.*, 3.0.0)"
+  }
+}

6-consumer/lessons/2-language-understanding/code/functions/smart-timer-trigger/local.settings.json

@@ -0,0 +1,11 @@
+{
+  "IsEncrypted": false,
+  "Values": {
+    "FUNCTIONS_WORKER_RUNTIME": "python",
+    "AzureWebJobsStorage": "UseDevelopmentStorage=true",
+    "IOT_HUB_CONNECTION_STRING": "<connection string>",
+    "LUIS_KEY": "<primary key>",
+    "LUIS_ENDPOINT_URL": "<endpoint url>",
+    "LUIS_APP_ID": "<app id>"
+  }
+}

6-consumer/lessons/2-language-understanding/code/functions/smart-timer-trigger/requirements.txt

@@ -0,0 +1,4 @@
+# Do not include azure-functions-worker as it may conflict with the Azure Functions platform
+
+azure-functions
+azure-cognitiveservices-language-luis

6-consumer/lessons/2-language-understanding/code/functions/smart-timer-trigger/speech-trigger/__init__.py

@@ -0,0 +1,43 @@
+from typing import List
+import logging
+
+import azure.functions as func
+
+import json
+import os
+from azure.cognitiveservices.language.luis.runtime import LUISRuntimeClient
+from msrest.authentication import CognitiveServicesCredentials
+
+def main(events: List[func.EventHubEvent]):
+    luis_key = os.environ['LUIS_KEY']
+    endpoint_url = os.environ['LUIS_ENDPOINT_URL']
+    app_id = os.environ['LUIS_APP_ID']
+    
+    credentials = CognitiveServicesCredentials(luis_key)
+    client = LUISRuntimeClient(endpoint=endpoint_url, credentials=credentials)
+
+    for event in events:
+        logging.info('Python EventHub trigger processed an event: %s',
+                        event.get_body().decode('utf-8'))
+
+        event_body = json.loads(event.get_body().decode('utf-8'))
+        prediction_request = { 'query' : event_body['speech'] }
+
+        prediction_response = client.prediction.get_slot_prediction(app_id, 'Staging', prediction_request)
+
+        if prediction_response.prediction.top_intent == 'set timer':
+            numbers = prediction_response.prediction.entities['number']
+            time_units = prediction_response.prediction.entities['time unit']
+            total_time = 0
+
+            for i in range(0, len(numbers)):
+                number = numbers[i]
+                time_unit = time_units[i][0]
+
+                if time_unit == 'minute':
+                    total_time += number * 60
+                else:
+                    total_time += number
+
+            logging.info(f'Timer required for {total_time} seconds')
+

6-consumer/lessons/2-language-understanding/code/functions/smart-timer-trigger/speech-trigger/function.json

@@ -0,0 +1,15 @@
+{
+  "scriptFile": "__init__.py",
+  "bindings": [
+    {
+      "type": "eventHubTrigger",
+      "name": "events",
+      "direction": "in",
+      "eventHubName": "samples-workitems",
+      "connection": "IOT_HUB_CONNECTION_STRING",
+      "cardinality": "many",
+      "consumerGroup": "$Default",
+      "dataType": "binary"
+    }
+  ]
+}

6-consumer/lessons/3-spoken-feedback/README.md

@@ -1,4 +1,4 @@
-# Provide spoken feedback
+# Set a timer and provide spoken feedback
 
 Add a sketchnote if possible/appropriate
 

README.md

@@ -83,6 +83,12 @@ We have two choices of IoT hardware to use for the projects depending on persona
 | 16 | [Manufacturing](./4-manufacturing) | Check fruit quality from an IoT device | Learn about using your fruit quality detector from an IoT device | [Check fruit quality from an IoT device](./4-manufacturing/lessons/2-check-fruit-from-device/README.md) |
 | 17 | [Manufacturing](./4-manufacturing) | Run your fruit detector on the edge | Learn about running your fruit detector on an IoT device on the edge | [Run your fruit detector on the edge](./4-manufacturing/lessons/3-run-fruit-detector-edge/README.md) |
 | 18 | [Manufacturing](./4-manufacturing) | Trigger fruit quality detection from a sensor | Learn about triggering fruit quality detection from a sensor | [Trigger fruit quality detection from a sensor](./4-manufacturing/lessons/4-trigger-fruit-detector/README.md) |
+| 19 | [Retail](./5-retail) | | |
+| 20 | [Retail](./5-retail) | | |
+| 21 | [Consumer](./6-consumer) | Recognize speech with an IoT device | Learn how to recognize speech from an IoT device to build a smart timer | [Recognize speech with an IoT device](./6-consumer/lessons/1-speech-recognition/README.md) |
+| 22 | [Consumer](./6-consumer) | Understand language | Learn how to understand sentences spoken to an IoT device | [Understand language](./6-consumer/lessons/2-language-understanding/README.md) |
+| 23 | [Consumer](./6-consumer) | Set a timer and provide spoken feedback | Learn how to set a timer on an IoT device and give spoken feedback on when the timer is set and when it finishes | [Set a timer and provide spoken feedback](./6-consumer/lessons/3-spoken-feedback/README.md) |
+| 24 | [Consumer](./6-consumer) | Support multiple languages | Learn how to support multiple languages, both being spoken to and the responses from your smart timer | [Support multiple languages](./6-consumer/lessons/4-multiple-language-support/README.md) |
 
 ## Offline access