From 24512dc75182285bd0165a4601daece0af9daefc Mon Sep 17 00:00:00 2001 From: Jim Bennett Date: Tue, 15 Jun 2021 19:47:26 -0700 Subject: [PATCH] Almost done with LUIS --- .vscode/settings.json | 2 + .../lessons/1-speech-recognition/README.md | 4 +- .../2-language-understanding/README.md | 39 ++++++++++++++++--- .../2-language-understanding/assignment.md | 9 ++++- 4 files changed, 45 insertions(+), 9 deletions(-) diff --git a/.vscode/settings.json b/.vscode/settings.json index b3d8c5b5..0794c279 100644 --- a/.vscode/settings.json +++ b/.vscode/settings.json @@ -1,10 +1,12 @@ { "cSpell.words": [ "ADCs", + "Alexa", "Geospatial", "Kbps", "Mbps", "Seeed", + "Siri", "Twilio", "UART", "UDID", diff --git a/6-consumer/lessons/1-speech-recognition/README.md b/6-consumer/lessons/1-speech-recognition/README.md index 5eec03e1..3269493b 100644 --- a/6-consumer/lessons/1-speech-recognition/README.md +++ b/6-consumer/lessons/1-speech-recognition/README.md @@ -2,7 +2,9 @@ Add a sketchnote if possible/appropriate -![Embed a video here if available](video-url) +This video gives an overview of the Azure speech service, a topic that will be covered in this lesson: + +[![How to get started using your Cognitive Services Speech resource from the Microsoft Azure YouTube channel](https://img.youtube.com/vi/iW0Fw0l3mrA/0.jpg)](https://www.youtube.com/watch?v=iW0Fw0l3mrA) ## Pre-lecture quiz diff --git a/6-consumer/lessons/2-language-understanding/README.md b/6-consumer/lessons/2-language-understanding/README.md index d043061e..fc3919aa 100644 --- a/6-consumer/lessons/2-language-understanding/README.md +++ b/6-consumer/lessons/2-language-understanding/README.md @@ -25,6 +25,25 @@ In this lesson we'll cover: ## Language understanding +Humans have used language to communicate for hundreds of thousands of years. We communicate with words, sounds, or actions and understand what is said, both the meaning of the words, sounds or actions, but also their context. We understand sincerity and sarcasm, allowing the same words to mean different things depending on the tone of our voice. + +✅ Think about some of the conversations you have had recently. How much of the conversation would be hard for a computer to understand because it needs context? + +Computers, despite all their advances, still have along way to go. When we refer to language understanding with computers, we don't mean anything anywhere near as advanced as human communication, instead we mean taking some words and extracting key details. + +Language understanding, also called natural-language understanding is part of a field of artificial intelligence called natural-language processing (or NLP), and deals with reading comprehension, trying to understand the details of words or sentences. If you use a voice assistant such as Alexa or Siri, you have used language understanding services. These are the behind-the-scenes AI services that convert "Alexa, play the latest album by Taylor Swift" into my daughter dancing around the living room to her favorite tunes. + +As humans, we understand language without really thinking about it. If I asked another human to "play the latest album by Taylor Swift" then they would instinctively know what I meant. For a computer, this is harder. It would have to take the words, converted from speech to text, and work out the following pieces of information: + +* Music needs to be played +* The music is by the artist Taylor Swift +* The specific music is a whole album of multiple tracks in order +* Taylor Swift has many albums, so they need to be sorted by chronological order and the most recently published is the one required + +✅ Think of some other sentences you have spoken when making requests, such as ordering coffee or asking a family member to pass you something. Try to break then down into the pieces of information a computer would need to extract to understand the sentence. + +Language understanding models are AI models that are trained to extract certain details from language, and then are trained for specific tasks using transfer learning, in the same way you trained a Custom Vision model using a small set of images. You can take a model, then train it using the text you want it to understand. + ## Create a language understanding model ![The LUIS logo](../../../images/luis-logo.png) @@ -72,13 +91,18 @@ To use LUIS, you need to create an authoring resource. ## Intents and entities -Language understanding is based around *intents* and *entities*. Intents are what the intent of the words are, for example setting a timer or ordering food. Entities are what the intent is referring to, such as the length of the timer, or the type of food. Each sentence that the model interprets should have at least one intent, and optionally one or more entities. +Language understanding is based around *intents* and *entities*. Intents are what the intent of the words are, for example playing music, setting a timer, or ordering food. Entities are what the intent is referring to, such as the album, the length of the timer, or the type of food. Each sentence that the model interprets should have at least one intent, and optionally one or more entities. -For example: +Some examples: -* "Set a 3 minute timer" - the intent is to *set a timer*, the entity is *3 minutes*. -* "Cancel my timer" - the intent is to *cancel a timer*, and there are no entities. -* "Order 3 large pineapple pizzas and a caesar salad" - the intent is to *order food*, the entities are *3 large pineapple pizzas* and *caesar salad*. +| Sentence | Intent | Entities | +| --------------------------------------------------- | ---------------- | ------------------------------------------ | +| "Play the latest album by Taylor Swift" | *play music* | *the latest album by Taylor Swift* | +| "Set a 3 minute timer" | *set a timer* | *3 minutes* | +| "Cancel my timer" | *cancel a timer* | None | +| "Order 3 large pineapple pizzas and a caesar salad" | *order food* | *3 large pineapple pizzas*, *caesar salad* | + +✅ With the sentences you though about earlier, what would be the intent and any entities in that sentence? To train LUIS, first you set the entities. These can be a fixed list of terms, or learned from the text. For example, you could provide a fixed list of food available from your menu, with variations (or synonyms) of each word, such as *egg plant* and *aubergine* as variations of *aubergine*. LUIS also has pre-built entities that can be used, such as numbers and locations. @@ -391,6 +415,8 @@ Once published, the LUIS model can be called from code. In the last lesson you s ## 🚀 Challenge +There are many ways to request the same thing, such as setting a timer. Think of different ways to do this, and use them as examples in your LUIS app. Test these out, to see how well your model can cope with multiple ways to request a timer. + ## Post-lecture quiz [Post-lecture quiz](https://brave-island-0b7c7f50f.azurestaticapps.net/quiz/34) @@ -398,7 +424,8 @@ Once published, the LUIS model can be called from code. In the last lesson you s ## Review & Self Study * Read more about LUIS and it's capabilities on the [Language Understanding (LUIS) documentation page on Microsoft docs](https://docs.microsoft.com/azure/cognitive-services/luis/?WT.mc_id=academic-17441-jabenn) +* Read more about language understanding on the [Natural-language understanding page on Wikipedia](https://wikipedia.org/wiki/Natural-language_understanding) ## Assignment -[](assignment.md) +[Cancel the timer](assignment.md) diff --git a/6-consumer/lessons/2-language-understanding/assignment.md b/6-consumer/lessons/2-language-understanding/assignment.md index da157d5c..69c4645f 100644 --- a/6-consumer/lessons/2-language-understanding/assignment.md +++ b/6-consumer/lessons/2-language-understanding/assignment.md @@ -1,9 +1,14 @@ -# +# Cancel the timer ## Instructions +So far in this lesson you have trained a model to understand setting a timer. Another useful feature is cancelling a timer - maybe your bread is ready and can be taken out of the oven. + +Add a new intent to your LUIS app to cancel the timer. It won't need any entities, but will need some example sentences. Handle this in your serverless code if it is the top intent, logging that the intent was recognized. + ## Rubric | Criteria | Exemplary | Adequate | Needs Improvement | | -------- | --------- | -------- | ----------------- | -| | | | | +| Add the cancel timer intent to the LUIS app | Was able to add the intent and train the model | Was able to add the intent but not train the model | Was unable to add the intent and train the model | +| Handle the intent in the serverless app | Was able to detect the intent as the top intent and log it | Was able to detect the intent as the top intent | Was unable to detect the intent as the top intent |