+
+
+
+
+
+
+
+
+
Cuisine classifiers 1
+
In this lesson, we’ll explore a variety of classifiers to predict
+a given national cuisine based on a group of ingredients. While
+doing so, we’ll learn more about some of the ways that algorithms can be
+leveraged for classification tasks.
+
+
+
Preparation
+
This lesson builds up on our previous
+lesson where we:
+
+Made a gentle introduction to classifications using a dataset
+about all the brilliant cuisines of Asia and India 😋.
+Explored some dplyr
+verbs to prep and clean our data.
+Made beautiful visualizations using ggplot2.
+Demonstrated how to deal with imbalanced data by preprocessing it
+using recipes.
+Demonstrated how to prep
and bake
our
+recipe to confirm that it will work as supposed to.
+
+
+
Prerequisite
+
For this lesson, we’ll require the following packages to clean, prep
+and visualize our data:
+
+tidyverse
: The tidyverse is a collection of R packages
+designed to makes data science faster, easier and more fun!
+tidymodels
: The tidymodels framework is a collection of packages
+for modeling and machine learning.
+DataExplorer
: The DataExplorer
+package is meant to simplify and automate EDA process and report
+generation.
+themis
: The themis package provides Extra
+Recipes Steps for Dealing with Unbalanced Data.
+nnet
: The nnet
+package provides functions for estimating feed-forward neural
+networks with a single hidden layer, and for multinomial logistic
+regression models.
+
+
You can have them installed as:
+
install.packages(c("tidyverse", "tidymodels", "DataExplorer", "here"))
+
Alternatively, the script below checks whether you have the packages
+required to complete this module and installs them for you in case they
+are missing.
+
suppressWarnings(if (!require("pacman"))install.packages("pacman"))
+
+pacman::p_load(tidyverse, tidymodels, DataExplorer, themis, here)
+
Now, let’s hit the ground running!
+
+
+
+
+
1. Split the data into training and test sets.
+
We’ll start by picking a few steps from our previous lesson.
+
+
Drop the most common ingredients that create confusion between
+distinct cuisines, using dplyr::select()
.
+
Everyone loves rice, garlic and ginger!
+
# Load the original cuisines data
+df <- read_csv(file = "https://raw.githubusercontent.com/microsoft/ML-For-Beginners/main/4-Classification/data/cuisines.csv")
+
## New names:
+## Rows: 2448 Columns: 385
+## ── Column specification
+## ──────────────────────────────────────────────────────── Delimiter: "," chr
+## (1): cuisine dbl (384): ...1, almond, angelica, anise, anise_seed, apple,
+## apple_brandy, a...
+## ℹ Use `spec()` to retrieve the full column specification for this data. ℹ
+## Specify the column types or set `show_col_types = FALSE` to quiet this message.
+## • `` -> `...1`
+
# Drop id column, rice, garlic and ginger from our original data set
+df_select <- df %>%
+ select(-c(1, rice, garlic, ginger)) %>%
+ # Encode cuisine column as categorical
+ mutate(cuisine = factor(cuisine))
+
+# Display new data set
+df_select %>%
+ slice_head(n = 5)
+
+
+
+
# Display distribution of cuisines
+df_select %>%
+ count(cuisine) %>%
+ arrange(desc(n))
+
+
+
+
Perfect! Now, time to split the data such that 70% of the data goes
+to training and 30% goes to testing. We’ll also apply a
+stratification
technique when splitting the data to
+maintain the proportion of each cuisine
in the training and
+validation datasets.
+
rsample, a package in
+Tidymodels, provides infrastructure for efficient data splitting and
+resampling:
+
# Load the core Tidymodels packages into R session
+library(tidymodels)
+
+# Create split specification
+set.seed(2056)
+cuisines_split <- initial_split(data = df_select,
+ strata = cuisine,
+ prop = 0.7)
+
+# Extract the data in each split
+cuisines_train <- training(cuisines_split)
+cuisines_test <- testing(cuisines_split)
+
+# Print the number of cases in each split
+cat("Training cases: ", nrow(cuisines_train), "\n",
+ "Test cases: ", nrow(cuisines_test), sep = "")
+
## Training cases: 1712
+## Test cases: 736
+
# Display the first few rows of the training set
+cuisines_train %>%
+ slice_head(n = 5)
+
+
+
+
# Display distribution of cuisines in the training set
+cuisines_train %>%
+ count(cuisine) %>%
+ arrange(desc(n))
+
+
+
+
+
+
+
2. Deal with imbalanced data
+
As you might have noticed in the original data set as well as in our
+training set, there is quite an unequal distribution in the number of
+cuisines. Korean cuisines are almost 3 times Thai cuisines.
+Imbalanced data often has negative effects on the model performance.
+Many models perform best when the number of observations is equal and,
+thus, tend to struggle with unbalanced data.
+
There are majorly two ways of dealing with imbalanced data sets:
+
+adding observations to the minority class:
+Over-sampling
e.g using a SMOTE algorithm which
+synthetically generates new examples of the minority class using nearest
+neighbors of these cases.
+removing observations from majority class:
+Under-sampling
+
+
In our previous lesson, we demonstrated how to deal with imbalanced
+data sets using a recipe
. A recipe can be thought of as a
+blueprint that describes what steps should be applied to a data set in
+order to get it ready for data analysis. In our case, we want to have an
+equal distribution in the number of our cuisines for our
+training set
. Let’s get right into it.
+
# Load themis package for dealing with imbalanced data
+library(themis)
+
+# Create a recipe for preprocessing training data
+cuisines_recipe <- recipe(cuisine ~ ., data = cuisines_train) %>%
+ step_smote(cuisine)
+
+# Print recipe
+cuisines_recipe
+
##
+
## ── Recipe ──────────────────────────────────────────────────────────────────────
+
##
+
## ── Inputs
+
## Number of variables by role
+
## outcome: 1
+## predictor: 380
+
##
+
## ── Operations
+
## • SMOTE based on: cuisine
+
You can of course go ahead and confirm (using prep+bake) that the
+recipe will work as you expect it - all the cuisine labels having
+559
observations.
+
Since we’ll be using this recipe as a preprocessor for modeling, a
+workflow()
will do all the prep and bake for us, so we
+won’t have to manually estimate the recipe.
+
Now we are ready to train a model 👩💻👨💻!
+
+
+
3. Choosing your classifier
+
+
+
Artwork by @allison_horst
+
+
Now we have to decide which algorithm to use for the job 🤔.
+
In Tidymodels, the parsnip package
+provides consistent interface for working with models across different
+engines (packages). Please see the parsnip documentation to explore model types &
+engines and their corresponding model
+arguments. The variety is quite bewildering at first sight. For
+instance, the following methods all include classification
+techniques:
+
+C5.0 Rule-Based Classification Models
+Flexible Discriminant Models
+Linear Discriminant Models
+Regularized Discriminant Models
+Logistic Regression Models
+Multinomial Regression Models
+Naive Bayes Models
+Support Vector Machines
+Nearest Neighbors
+Decision Trees
+Ensemble methods
+Neural Networks
+
+
The list goes on!
+
+
What classifier to go with?
+
So, which classifier should you choose? Often, running through
+several and looking for a good result is a way to test.
+
+AutoML solves this problem neatly by running these comparisons in the
+cloud, allowing you to choose the best algorithm for your data. Try it
+here
+
+
Also the choice of classifier depends on our problem. For instance,
+when the outcome can be categorized into
+more than two classes
, like in our case, you must use a
+multiclass classification algorithm
as opposed to
+binary classification.
+
+
+
A better approach
+
A better way than wildly guessing, however, is to follow the ideas on
+this downloadable ML
+Cheat sheet. Here, we discover that, for our multiclass problem, we
+have some choices:
+
+
+
A section of Microsoft’s Algorithm Cheat Sheet,
+detailing multiclass classification options
+
+
+
+
Reasoning
+
Let’s see if we can reason our way through different approaches given
+the constraints we have:
+
+Deep Neural networks are too heavy. Given our
+clean, but minimal dataset, and the fact that we are running training
+locally via notebooks, deep neural networks are too heavyweight for this
+task.
+No two-class classifier. We do not use a
+two-class classifier, so that rules out one-vs-all.
+Decision tree or logistic regression could work.
+A decision tree might work, or multinomial regression/multiclass
+logistic regression for multiclass data.
+Multiclass Boosted Decision Trees solve a different
+problem. The multiclass boosted decision tree is most suitable
+for nonparametric tasks, e.g. tasks designed to build rankings, so it is
+not useful for us.
+
+
Also, normally before embarking on more complex machine learning
+models e.g ensemble methods, it’s a good idea to build the simplest
+possible model to get an idea of what is going on. So for this lesson,
+we’ll start with a multinomial logistic regression
+model.
+
+Logistic regression is a technique used when the outcome variable is
+categorical (or nominal). For Binary logistic regression the number of
+outcome variables is two, whereas the number of outcome variables for
+multinomial logistic regression is more than two. See Advanced
+Regression Methods for further reading.
+
+
+
+
+
4. Train and evaluate a Multinomial logistic regression model.
+
In Tidymodels, parsnip::multinom_reg()
, defines a model
+that uses linear predictors to predict multiclass data using the
+multinomial distribution. See ?multinom_reg()
for the
+different ways/engines you can use to fit this model.
+
For this example, we’ll fit a Multinomial regression model via the
+default nnet
+engine.
+
+I picked a value for penalty
sort of randomly. There are
+better ways to choose this value that is, by using
+resampling
and tuning
the model which we’ll
+discuss later.
+See Tidymodels:
+Get Started in case you want to learn more on how to tune model
+hyperparameters.
+
+
# Create a multinomial regression model specification
+mr_spec <- multinom_reg(penalty = 1) %>%
+ set_engine("nnet", MaxNWts = 2086) %>%
+ set_mode("classification")
+
+# Print model specification
+mr_spec
+
## Multinomial Regression Model Specification (classification)
+##
+## Main Arguments:
+## penalty = 1
+##
+## Engine-Specific Arguments:
+## MaxNWts = 2086
+##
+## Computational engine: nnet
+
Great job 🥳! Now that we have a recipe and a model specification, we
+need to find a way of bundling them together into an object that will
+first preprocess the data then fit the model on the preprocessed data
+and also allow for potential post-processing activities. In Tidymodels,
+this convenient object is called a workflow
and
+conveniently holds your modeling components! This is what we’d call
+pipelines in Python.
+
So let’s bundle everything up into a workflow!📦
+
# Bundle recipe and model specification
+mr_wf <- workflow() %>%
+ add_recipe(cuisines_recipe) %>%
+ add_model(mr_spec)
+
+# Print out workflow
+mr_wf
+
## ══ Workflow ════════════════════════════════════════════════════════════════════
+## Preprocessor: Recipe
+## Model: multinom_reg()
+##
+## ── Preprocessor ────────────────────────────────────────────────────────────────
+## 1 Recipe Step
+##
+## • step_smote()
+##
+## ── Model ───────────────────────────────────────────────────────────────────────
+## Multinomial Regression Model Specification (classification)
+##
+## Main Arguments:
+## penalty = 1
+##
+## Engine-Specific Arguments:
+## MaxNWts = 2086
+##
+## Computational engine: nnet
+
Workflows 👌👌! A workflow()
can be fit
+in much the same way a model can. So, time to train a model!
+
# Train a multinomial regression model
+mr_fit <- fit(object = mr_wf, data = cuisines_train)
+
+mr_fit
+
## ══ Workflow [trained] ══════════════════════════════════════════════════════════
+## Preprocessor: Recipe
+## Model: multinom_reg()
+##
+## ── Preprocessor ────────────────────────────────────────────────────────────────
+## 1 Recipe Step
+##
+## • step_smote()
+##
+## ── Model ───────────────────────────────────────────────────────────────────────
+## Call:
+## nnet::multinom(formula = ..y ~ ., data = data, decay = ~1, MaxNWts = ~2086,
+## trace = FALSE)
+##
+## Coefficients:
+## (Intercept) almond angelica anise anise_seed apple
+## indian 0.19723325 0.2409661 0 -5.004955e-05 -0.1657635 -0.05769734
+## japanese 0.13961959 -0.6262400 0 -1.169155e-04 -0.4893596 -0.08585717
+## korean 0.22377347 -0.1833485 0 -5.560395e-05 -0.2489401 -0.15657804
+## thai -0.04336577 -0.6106258 0 4.903828e-04 -0.5782866 0.63451105
+## apple_brandy apricot armagnac artemisia artichoke asparagus
+## indian 0 0.37042636 0 -0.09122797 0 -0.27181970
+## japanese 0 0.28895643 0 -0.12651100 0 0.14054037
+## korean 0 -0.07981259 0 0.55756709 0 -0.66979948
+## thai 0 -0.33160904 0 -0.10725182 0 -0.02602152
+## avocado bacon baked_potato balm banana barley
+## indian -0.46624197 0.16008055 0 0 -0.2838796 0.2230625
+## japanese 0.90341344 0.02932727 0 0 -0.4142787 2.0953906
+## korean -0.06925382 -0.35804134 0 0 -0.2686963 -0.7233404
+## thai -0.21473955 -0.75594439 0 0 0.6784880 -0.4363320
+## bartlett_pear basil bay bean beech
+## indian 0 -0.7128756 0.1011587 -0.8777275 -0.0004380795
+## japanese 0 0.1288697 0.9425626 -0.2380748 0.3373437611
+## korean 0 -0.2445193 -0.4744318 -0.8957870 -0.0048784496
+## thai 0 1.5365848 0.1333256 0.2196970 -0.0113078024
+## beef beef_broth beef_liver beer beet
+## indian -0.7985278 0.2430186 -0.035598065 -0.002173738 0.01005813
+## japanese 0.2241875 -0.3653020 -0.139551027 0.128905553 0.04923911
+## korean 0.5366515 -0.6153237 0.213455197 -0.010828645 0.27325423
+## thai 0.1570012 -0.9364154 -0.008032213 -0.035063746 -0.28279823
+## bell_pepper bergamot berry bitter_orange black_bean
+## indian 0.49074330 0 0.58947607 0.191256164 -0.1945233
+## japanese 0.09074167 0 -0.25917977 -0.118915977 -0.3442400
+## korean -0.57876763 0 -0.07874180 -0.007729435 -0.5220672
+## thai 0.92554006 0 -0.07210196 -0.002983296 -0.4614426
+## black_currant black_mustard_seed_oil black_pepper black_raspberry
+## indian 0 0.38935801 -0.4453495 0
+## japanese 0 -0.05452887 -0.5440869 0
+## korean 0 -0.03929970 0.8025454 0
+## thai 0 -0.21498372 -0.9854806 0
+## black_sesame_seed black_tea blackberry blackberry_brandy
+## indian -0.2759246 0.3079977 0.191256164 0
+## japanese -0.6101687 -0.1671913 -0.118915977 0
+## korean 1.5197674 -0.3036261 -0.007729435 0
+## thai -0.1755656 -0.1487033 -0.002983296 0
+## blue_cheese blueberry bone_oil bourbon_whiskey brandy
+## indian 0 0.216164294 -0.2276744 0 0.22427587
+## japanese 0 -0.119186087 0.3913019 0 -0.15595599
+## korean 0 -0.007821986 0.2854487 0 -0.02562342
+## thai 0 -0.004947048 -0.0253658 0 -0.05715244
+##
+## ...
+## and 308 more lines.
+
The output shows the coefficients that the model learned during
+training.
+
+
Evaluate the Trained Model
+
It’s time to see how the model performed 📏 by evaluating it on a
+test set! Let’s begin by making predictions on the test set.
+
# Make predictions on the test set
+results <- cuisines_test %>% select(cuisine) %>%
+ bind_cols(mr_fit %>% predict(new_data = cuisines_test))
+
+# Print out results
+results %>%
+ slice_head(n = 5)
+
+
+
+
Great job! In Tidymodels, evaluating model performance can be done
+using yardstick - a
+package used to measure the effectiveness of models using performance
+metrics. As we did in our logistic regression lesson, let’s begin by
+computing a confusion matrix.
+
# Confusion matrix for categorical data
+conf_mat(data = results, truth = cuisine, estimate = .pred_class)
+
## Truth
+## Prediction chinese indian japanese korean thai
+## chinese 83 1 8 15 10
+## indian 4 163 1 2 6
+## japanese 21 5 73 25 1
+## korean 15 0 11 191 0
+## thai 10 11 3 7 70
+
When dealing with multiple classes, it’s generally more intuitive to
+visualize this as a heat map, like this:
+
update_geom_defaults(geom = "tile", new = list(color = "black", alpha = 0.7))
+# Visualize confusion matrix
+results %>%
+ conf_mat(cuisine, .pred_class) %>%
+ autoplot(type = "heatmap")
+
+
The darker squares in the confusion matrix plot indicate high numbers
+of cases, and you can hopefully see a diagonal line of darker squares
+indicating cases where the predicted and actual label are the same.
+
Let’s now calculate summary statistics for the confusion matrix.
+
# Summary stats for confusion matrix
+conf_mat(data = results, truth = cuisine, estimate = .pred_class) %>% summary()
+
+
+
+
If we narrow down to some metrics such as accuracy, sensitivity, ppv,
+we are not badly off for a start 🥳!
+
+
+
+
4. Digging Deeper
+
Let’s ask one subtle question: What criteria is used to settle for a
+given type of cuisine as the predicted outcome?
+
Well, Statistical machine learning algorithms, like logistic
+regression, are based on probability
; so what actually gets
+predicted by a classifier is a probability distribution over a set of
+possible outcomes. The class with the highest probability is then chosen
+as the most likely outcome for the given observations.
+
Let’s see this in action by making both hard class predictions and
+probabilities.
+
# Make hard class prediction and probabilities
+results_prob <- cuisines_test %>%
+ select(cuisine) %>%
+ bind_cols(mr_fit %>% predict(new_data = cuisines_test)) %>%
+ bind_cols(mr_fit %>% predict(new_data = cuisines_test, type = "prob"))
+
+# Print out results
+results_prob %>%
+ slice_head(n = 5)
+
+
+
+
Much better!
+
✅ Can you explain why the model is pretty sure that the first
+observation is Thai?
+
+
+
🚀Challenge
+
In this lesson, you used your cleaned data to build a machine
+learning model that can predict a national cuisine based on a series of
+ingredients. Take some time to read through the many options
+Tidymodels provides to classify data and other
+ways to fit multinomial regression.
+
+
THANK YOU TO:
+
Allison Horst
+for creating the amazing illustrations that make R more welcoming and
+engaging. Find more illustrations at her gallery.
+
Cassie Breviu and Jen Looper for creating the
+original Python version of this module ♥️
+
Happy Learning,
+
Eric, Gold Microsoft Learn
+Student Ambassador.
+
+
+
+
LS0tCnRpdGxlOiAnQnVpbGQgYSBjbGFzc2lmaWNhdGlvbiBtb2RlbDogRGVsaWNpb3VzIEFzaWFuIGFuZCBJbmRpYW4gQ3Vpc2luZXMnCm91dHB1dDoKICBodG1sX2RvY3VtZW50OgogICAgZGZfcHJpbnQ6IHBhZ2VkCiAgICB0aGVtZTogZmxhdGx5CiAgICBoaWdobGlnaHQ6IGJyZWV6ZWRhcmsKICAgIHRvYzogeWVzCiAgICB0b2NfZmxvYXQ6IHllcwogICAgY29kZV9kb3dubG9hZDogeWVzCi0tLQoKIyMgQ3Vpc2luZSBjbGFzc2lmaWVycyAxCgpJbiB0aGlzIGxlc3Nvbiwgd2UnbGwgZXhwbG9yZSBhIHZhcmlldHkgb2YgY2xhc3NpZmllcnMgdG8gKnByZWRpY3QgYSBnaXZlbiBuYXRpb25hbCBjdWlzaW5lIGJhc2VkIG9uIGEgZ3JvdXAgb2YgaW5ncmVkaWVudHMuKiBXaGlsZSBkb2luZyBzbywgd2UnbGwgbGVhcm4gbW9yZSBhYm91dCBzb21lIG9mIHRoZSB3YXlzIHRoYXQgYWxnb3JpdGhtcyBjYW4gYmUgbGV2ZXJhZ2VkIGZvciBjbGFzc2lmaWNhdGlvbiB0YXNrcy4KCiMjIyBbKipQcmUtbGVjdHVyZSBxdWl6KipdKGh0dHBzOi8vZ3JheS1zYW5kLTA3YTEwZjQwMy4xLmF6dXJlc3RhdGljYXBwcy5uZXQvcXVpei8yMS8pCgojIyMgKipQcmVwYXJhdGlvbioqCgpUaGlzIGxlc3NvbiBidWlsZHMgdXAgb24gb3VyIFtwcmV2aW91cyBsZXNzb25dKGh0dHBzOi8vZ2l0aHViLmNvbS9taWNyb3NvZnQvTUwtRm9yLUJlZ2lubmVycy9ibG9iL21haW4vNC1DbGFzc2lmaWNhdGlvbi8xLUludHJvZHVjdGlvbi9zb2x1dGlvbi9SL2xlc3Nvbl8xMC5tZCkgd2hlcmUgd2U6CgotICAgTWFkZSBhIGdlbnRsZSBpbnRyb2R1Y3Rpb24gdG8gY2xhc3NpZmljYXRpb25zIHVzaW5nIGEgZGF0YXNldCBhYm91dCBhbGwgdGhlIGJyaWxsaWFudCBjdWlzaW5lcyBvZiBBc2lhIGFuZCBJbmRpYSDwn5iLLgoKLSAgIEV4cGxvcmVkIHNvbWUgW2RwbHlyIHZlcmJzXShodHRwczovL2RwbHlyLnRpZHl2ZXJzZS5vcmcvKSB0byBwcmVwIGFuZCBjbGVhbiBvdXIgZGF0YS4KCi0gICBNYWRlIGJlYXV0aWZ1bCB2aXN1YWxpemF0aW9ucyB1c2luZyBnZ3Bsb3QyLgoKLSAgIERlbW9uc3RyYXRlZCBob3cgdG8gZGVhbCB3aXRoIGltYmFsYW5jZWQgZGF0YSBieSBwcmVwcm9jZXNzaW5nIGl0IHVzaW5nIFtyZWNpcGVzXShodHRwczovL3JlY2lwZXMudGlkeW1vZGVscy5vcmcvYXJ0aWNsZXMvU2ltcGxlX0V4YW1wbGUuaHRtbCkuCgotICAgRGVtb25zdHJhdGVkIGhvdyB0byBgcHJlcGAgYW5kIGBiYWtlYCBvdXIgcmVjaXBlIHRvIGNvbmZpcm0gdGhhdCBpdCB3aWxsIHdvcmsgYXMgc3VwcG9zZWQgdG8uCgojIyMjICoqUHJlcmVxdWlzaXRlKioKCkZvciB0aGlzIGxlc3Nvbiwgd2UnbGwgcmVxdWlyZSB0aGUgZm9sbG93aW5nIHBhY2thZ2VzIHRvIGNsZWFuLCBwcmVwIGFuZCB2aXN1YWxpemUgb3VyIGRhdGE6CgotICAgYHRpZHl2ZXJzZWA6IFRoZSBbdGlkeXZlcnNlXShodHRwczovL3d3dy50aWR5dmVyc2Uub3JnLykgaXMgYSBbY29sbGVjdGlvbiBvZiBSIHBhY2thZ2VzXShodHRwczovL3d3dy50aWR5dmVyc2Uub3JnL3BhY2thZ2VzKSBkZXNpZ25lZCB0byBtYWtlcyBkYXRhIHNjaWVuY2UgZmFzdGVyLCBlYXNpZXIgYW5kIG1vcmUgZnVuIQoKLSAgIGB0aWR5bW9kZWxzYDogVGhlIFt0aWR5bW9kZWxzXShodHRwczovL3d3dy50aWR5bW9kZWxzLm9yZy8pIGZyYW1ld29yayBpcyBhIFtjb2xsZWN0aW9uIG9mIHBhY2thZ2VzXShodHRwczovL3d3dy50aWR5bW9kZWxzLm9yZy9wYWNrYWdlcy8pIGZvciBtb2RlbGluZyBhbmQgbWFjaGluZSBsZWFybmluZy4KCi0gICBgRGF0YUV4cGxvcmVyYDogVGhlIFtEYXRhRXhwbG9yZXIgcGFja2FnZV0oaHR0cHM6Ly9jcmFuLnItcHJvamVjdC5vcmcvd2ViL3BhY2thZ2VzL0RhdGFFeHBsb3Jlci92aWduZXR0ZXMvZGF0YWV4cGxvcmVyLWludHJvLmh0bWwpIGlzIG1lYW50IHRvIHNpbXBsaWZ5IGFuZCBhdXRvbWF0ZSBFREEgcHJvY2VzcyBhbmQgcmVwb3J0IGdlbmVyYXRpb24uCgotICAgYHRoZW1pc2A6IFRoZSBbdGhlbWlzIHBhY2thZ2VdKGh0dHBzOi8vdGhlbWlzLnRpZHltb2RlbHMub3JnLykgcHJvdmlkZXMgRXh0cmEgUmVjaXBlcyBTdGVwcyBmb3IgRGVhbGluZyB3aXRoIFVuYmFsYW5jZWQgRGF0YS4KCi0gICBgbm5ldGA6IFRoZSBbbm5ldCBwYWNrYWdlXShodHRwczovL2NyYW4uci1wcm9qZWN0Lm9yZy93ZWIvcGFja2FnZXMvbm5ldC9ubmV0LnBkZikgcHJvdmlkZXMgZnVuY3Rpb25zIGZvciBlc3RpbWF0aW5nIGZlZWQtZm9yd2FyZCBuZXVyYWwgbmV0d29ya3Mgd2l0aCBhIHNpbmdsZSBoaWRkZW4gbGF5ZXIsIGFuZCBmb3IgbXVsdGlub21pYWwgbG9naXN0aWMgcmVncmVzc2lvbiBtb2RlbHMuCgpZb3UgY2FuIGhhdmUgdGhlbSBpbnN0YWxsZWQgYXM6CgpgaW5zdGFsbC5wYWNrYWdlcyhjKCJ0aWR5dmVyc2UiLCAidGlkeW1vZGVscyIsICJEYXRhRXhwbG9yZXIiLCAiaGVyZSIpKWAKCkFsdGVybmF0aXZlbHksIHRoZSBzY3JpcHQgYmVsb3cgY2hlY2tzIHdoZXRoZXIgeW91IGhhdmUgdGhlIHBhY2thZ2VzIHJlcXVpcmVkIHRvIGNvbXBsZXRlIHRoaXMgbW9kdWxlIGFuZCBpbnN0YWxscyB0aGVtIGZvciB5b3UgaW4gY2FzZSB0aGV5IGFyZSBtaXNzaW5nLgoKYGBge3IsIG1lc3NhZ2U9Riwgd2FybmluZz1GfQpzdXBwcmVzc1dhcm5pbmdzKGlmICghcmVxdWlyZSgicGFjbWFuIikpaW5zdGFsbC5wYWNrYWdlcygicGFjbWFuIikpCgpwYWNtYW46OnBfbG9hZCh0aWR5dmVyc2UsIHRpZHltb2RlbHMsIERhdGFFeHBsb3JlciwgdGhlbWlzLCBoZXJlKQpgYGAKCk5vdywgbGV0J3MgaGl0IHRoZSBncm91bmQgcnVubmluZyEKCiMjIDEuIFNwbGl0IHRoZSBkYXRhIGludG8gdHJhaW5pbmcgYW5kIHRlc3Qgc2V0cy4KCldlJ2xsIHN0YXJ0IGJ5IHBpY2tpbmcgYSBmZXcgc3RlcHMgZnJvbSBvdXIgcHJldmlvdXMgbGVzc29uLgoKIyMjIERyb3AgdGhlIG1vc3QgY29tbW9uIGluZ3JlZGllbnRzIHRoYXQgY3JlYXRlIGNvbmZ1c2lvbiBiZXR3ZWVuIGRpc3RpbmN0IGN1aXNpbmVzLCB1c2luZyBgZHBseXI6OnNlbGVjdCgpYC4KCkV2ZXJ5b25lIGxvdmVzIHJpY2UsIGdhcmxpYyBhbmQgZ2luZ2VyIQoKYGBge3IgcmVjYXBfZHJvcH0KIyBMb2FkIHRoZSBvcmlnaW5hbCBjdWlzaW5lcyBkYXRhCmRmIDwtIHJlYWRfY3N2KGZpbGUgPSAiaHR0cHM6Ly9yYXcuZ2l0aHVidXNlcmNvbnRlbnQuY29tL21pY3Jvc29mdC9NTC1Gb3ItQmVnaW5uZXJzL21haW4vNC1DbGFzc2lmaWNhdGlvbi9kYXRhL2N1aXNpbmVzLmNzdiIpCgojIERyb3AgaWQgY29sdW1uLCByaWNlLCBnYXJsaWMgYW5kIGdpbmdlciBmcm9tIG91ciBvcmlnaW5hbCBkYXRhIHNldApkZl9zZWxlY3QgPC0gZGYgJT4lIAogIHNlbGVjdCgtYygxLCByaWNlLCBnYXJsaWMsIGdpbmdlcikpICU+JQogICMgRW5jb2RlIGN1aXNpbmUgY29sdW1uIGFzIGNhdGVnb3JpY2FsCiAgbXV0YXRlKGN1aXNpbmUgPSBmYWN0b3IoY3Vpc2luZSkpCgojIERpc3BsYXkgbmV3IGRhdGEgc2V0CmRmX3NlbGVjdCAlPiUgCiAgc2xpY2VfaGVhZChuID0gNSkKCiMgRGlzcGxheSBkaXN0cmlidXRpb24gb2YgY3Vpc2luZXMKZGZfc2VsZWN0ICU+JSAKICBjb3VudChjdWlzaW5lKSAlPiUgCiAgYXJyYW5nZShkZXNjKG4pKQpgYGAKClBlcmZlY3QhIE5vdywgdGltZSB0byBzcGxpdCB0aGUgZGF0YSBzdWNoIHRoYXQgNzAlIG9mIHRoZSBkYXRhIGdvZXMgdG8gdHJhaW5pbmcgYW5kIDMwJSBnb2VzIHRvIHRlc3RpbmcuIFdlJ2xsIGFsc28gYXBwbHkgYSBgc3RyYXRpZmljYXRpb25gIHRlY2huaXF1ZSB3aGVuIHNwbGl0dGluZyB0aGUgZGF0YSB0byBgbWFpbnRhaW4gdGhlIHByb3BvcnRpb24gb2YgZWFjaCBjdWlzaW5lYCBpbiB0aGUgdHJhaW5pbmcgYW5kIHZhbGlkYXRpb24gZGF0YXNldHMuCgpbcnNhbXBsZV0oaHR0cHM6Ly9yc2FtcGxlLnRpZHltb2RlbHMub3JnLyksIGEgcGFja2FnZSBpbiBUaWR5bW9kZWxzLCBwcm92aWRlcyBpbmZyYXN0cnVjdHVyZSBmb3IgZWZmaWNpZW50IGRhdGEgc3BsaXR0aW5nIGFuZCByZXNhbXBsaW5nOgoKYGBge3IgZGF0YV9zcGxpdH0KIyBMb2FkIHRoZSBjb3JlIFRpZHltb2RlbHMgcGFja2FnZXMgaW50byBSIHNlc3Npb24KbGlicmFyeSh0aWR5bW9kZWxzKQoKIyBDcmVhdGUgc3BsaXQgc3BlY2lmaWNhdGlvbgpzZXQuc2VlZCgyMDU2KQpjdWlzaW5lc19zcGxpdCA8LSBpbml0aWFsX3NwbGl0KGRhdGEgPSBkZl9zZWxlY3QsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgc3RyYXRhID0gY3Vpc2luZSwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBwcm9wID0gMC43KQoKIyBFeHRyYWN0IHRoZSBkYXRhIGluIGVhY2ggc3BsaXQKY3Vpc2luZXNfdHJhaW4gPC0gdHJhaW5pbmcoY3Vpc2luZXNfc3BsaXQpCmN1aXNpbmVzX3Rlc3QgPC0gdGVzdGluZyhjdWlzaW5lc19zcGxpdCkKCiMgUHJpbnQgdGhlIG51bWJlciBvZiBjYXNlcyBpbiBlYWNoIHNwbGl0CmNhdCgiVHJhaW5pbmcgY2FzZXM6ICIsIG5yb3coY3Vpc2luZXNfdHJhaW4pLCAiXG4iLAogICAgIlRlc3QgY2FzZXM6ICIsIG5yb3coY3Vpc2luZXNfdGVzdCksIHNlcCA9ICIiKQoKIyBEaXNwbGF5IHRoZSBmaXJzdCBmZXcgcm93cyBvZiB0aGUgdHJhaW5pbmcgc2V0CmN1aXNpbmVzX3RyYWluICU+JSAKICBzbGljZV9oZWFkKG4gPSA1KQoKCiMgRGlzcGxheSBkaXN0cmlidXRpb24gb2YgY3Vpc2luZXMgaW4gdGhlIHRyYWluaW5nIHNldApjdWlzaW5lc190cmFpbiAlPiUgCiAgY291bnQoY3Vpc2luZSkgJT4lIAogIGFycmFuZ2UoZGVzYyhuKSkKCgpgYGAKCiMjIDIuIERlYWwgd2l0aCBpbWJhbGFuY2VkIGRhdGEKCkFzIHlvdSBtaWdodCBoYXZlIG5vdGljZWQgaW4gdGhlIG9yaWdpbmFsIGRhdGEgc2V0IGFzIHdlbGwgYXMgaW4gb3VyIHRyYWluaW5nIHNldCwgdGhlcmUgaXMgcXVpdGUgYW4gdW5lcXVhbCBkaXN0cmlidXRpb24gaW4gdGhlIG51bWJlciBvZiBjdWlzaW5lcy4gS29yZWFuIGN1aXNpbmVzIGFyZSAqYWxtb3N0KiAzIHRpbWVzIFRoYWkgY3Vpc2luZXMuIEltYmFsYW5jZWQgZGF0YSBvZnRlbiBoYXMgbmVnYXRpdmUgZWZmZWN0cyBvbiB0aGUgbW9kZWwgcGVyZm9ybWFuY2UuIE1hbnkgbW9kZWxzIHBlcmZvcm0gYmVzdCB3aGVuIHRoZSBudW1iZXIgb2Ygb2JzZXJ2YXRpb25zIGlzIGVxdWFsIGFuZCwgdGh1cywgdGVuZCB0byBzdHJ1Z2dsZSB3aXRoIHVuYmFsYW5jZWQgZGF0YS4KClRoZXJlIGFyZSBtYWpvcmx5IHR3byB3YXlzIG9mIGRlYWxpbmcgd2l0aCBpbWJhbGFuY2VkIGRhdGEgc2V0czoKCi0gICBhZGRpbmcgb2JzZXJ2YXRpb25zIHRvIHRoZSBtaW5vcml0eSBjbGFzczogYE92ZXItc2FtcGxpbmdgIGUuZyB1c2luZyBhIFNNT1RFIGFsZ29yaXRobSB3aGljaCBzeW50aGV0aWNhbGx5IGdlbmVyYXRlcyBuZXcgZXhhbXBsZXMgb2YgdGhlIG1pbm9yaXR5IGNsYXNzIHVzaW5nIG5lYXJlc3QgbmVpZ2hib3JzIG9mIHRoZXNlIGNhc2VzLgoKLSAgIHJlbW92aW5nIG9ic2VydmF0aW9ucyBmcm9tIG1ham9yaXR5IGNsYXNzOiBgVW5kZXItc2FtcGxpbmdgCgpJbiBvdXIgcHJldmlvdXMgbGVzc29uLCB3ZSBkZW1vbnN0cmF0ZWQgaG93IHRvIGRlYWwgd2l0aCBpbWJhbGFuY2VkIGRhdGEgc2V0cyB1c2luZyBhIGByZWNpcGVgLiBBIHJlY2lwZSBjYW4gYmUgdGhvdWdodCBvZiBhcyBhIGJsdWVwcmludCB0aGF0IGRlc2NyaWJlcyB3aGF0IHN0ZXBzIHNob3VsZCBiZSBhcHBsaWVkIHRvIGEgZGF0YSBzZXQgaW4gb3JkZXIgdG8gZ2V0IGl0IHJlYWR5IGZvciBkYXRhIGFuYWx5c2lzLiBJbiBvdXIgY2FzZSwgd2Ugd2FudCB0byBoYXZlIGFuIGVxdWFsIGRpc3RyaWJ1dGlvbiBpbiB0aGUgbnVtYmVyIG9mIG91ciBjdWlzaW5lcyBmb3Igb3VyIGB0cmFpbmluZyBzZXRgLiBMZXQncyBnZXQgcmlnaHQgaW50byBpdC4KCmBgYHtyIHJlY2FwX2JhbGFuY2V9CiMgTG9hZCB0aGVtaXMgcGFja2FnZSBmb3IgZGVhbGluZyB3aXRoIGltYmFsYW5jZWQgZGF0YQpsaWJyYXJ5KHRoZW1pcykKCiMgQ3JlYXRlIGEgcmVjaXBlIGZvciBwcmVwcm9jZXNzaW5nIHRyYWluaW5nIGRhdGEKY3Vpc2luZXNfcmVjaXBlIDwtIHJlY2lwZShjdWlzaW5lIH4gLiwgZGF0YSA9IGN1aXNpbmVzX3RyYWluKSAlPiUgCiAgc3RlcF9zbW90ZShjdWlzaW5lKQoKIyBQcmludCByZWNpcGUKY3Vpc2luZXNfcmVjaXBlCgpgYGAKCllvdSBjYW4gb2YgY291cnNlIGdvIGFoZWFkIGFuZCBjb25maXJtICh1c2luZyBwcmVwK2Jha2UpIHRoYXQgdGhlIHJlY2lwZSB3aWxsIHdvcmsgYXMgeW91IGV4cGVjdCBpdCAtIGFsbCB0aGUgY3Vpc2luZSBsYWJlbHMgaGF2aW5nIGA1NTlgIG9ic2VydmF0aW9ucy4KClNpbmNlIHdlJ2xsIGJlIHVzaW5nIHRoaXMgcmVjaXBlIGFzIGEgcHJlcHJvY2Vzc29yIGZvciBtb2RlbGluZywgYSBgd29ya2Zsb3coKWAgd2lsbCBkbyBhbGwgdGhlIHByZXAgYW5kIGJha2UgZm9yIHVzLCBzbyB3ZSB3b24ndCBoYXZlIHRvIG1hbnVhbGx5IGVzdGltYXRlIHRoZSByZWNpcGUuCgpOb3cgd2UgYXJlIHJlYWR5IHRvIHRyYWluIGEgbW9kZWwg8J+RqeKAjfCfkrvwn5Go4oCN8J+SuyEKCiMjIDMuIENob29zaW5nIHlvdXIgY2xhc3NpZmllcgoKIVtBcnR3b3JrIGJ5IFxAYWxsaXNvbl9ob3JzdF0oLi4vLi4vaW1hZ2VzL3BhcnNuaXAuanBnKXt3aWR0aD0iNjAwIn0KCk5vdyB3ZSBoYXZlIHRvIGRlY2lkZSB3aGljaCBhbGdvcml0aG0gdG8gdXNlIGZvciB0aGUgam9iIPCfpJQuCgpJbiBUaWR5bW9kZWxzLCB0aGUgW2BwYXJzbmlwIHBhY2thZ2VgXShodHRwczovL3BhcnNuaXAudGlkeW1vZGVscy5vcmcvaW5kZXguaHRtbCkgcHJvdmlkZXMgY29uc2lzdGVudCBpbnRlcmZhY2UgZm9yIHdvcmtpbmcgd2l0aCBtb2RlbHMgYWNyb3NzIGRpZmZlcmVudCBlbmdpbmVzIChwYWNrYWdlcykuIFBsZWFzZSBzZWUgdGhlIHBhcnNuaXAgZG9jdW1lbnRhdGlvbiB0byBleHBsb3JlIFttb2RlbCB0eXBlcyAmIGVuZ2luZXNdKGh0dHBzOi8vd3d3LnRpZHltb2RlbHMub3JnL2ZpbmQvcGFyc25pcC8jbW9kZWxzKSBhbmQgdGhlaXIgY29ycmVzcG9uZGluZyBbbW9kZWwgYXJndW1lbnRzXShodHRwczovL3d3dy50aWR5bW9kZWxzLm9yZy9maW5kL3BhcnNuaXAvI21vZGVsLWFyZ3MpLiBUaGUgdmFyaWV0eSBpcyBxdWl0ZSBiZXdpbGRlcmluZyBhdCBmaXJzdCBzaWdodC4gRm9yIGluc3RhbmNlLCB0aGUgZm9sbG93aW5nIG1ldGhvZHMgYWxsIGluY2x1ZGUgY2xhc3NpZmljYXRpb24gdGVjaG5pcXVlczoKCi0gICBDNS4wIFJ1bGUtQmFzZWQgQ2xhc3NpZmljYXRpb24gTW9kZWxzCgotICAgRmxleGlibGUgRGlzY3JpbWluYW50IE1vZGVscwoKLSAgIExpbmVhciBEaXNjcmltaW5hbnQgTW9kZWxzCgotICAgUmVndWxhcml6ZWQgRGlzY3JpbWluYW50IE1vZGVscwoKLSAgIExvZ2lzdGljIFJlZ3Jlc3Npb24gTW9kZWxzCgotICAgTXVsdGlub21pYWwgUmVncmVzc2lvbiBNb2RlbHMKCi0gICBOYWl2ZSBCYXllcyBNb2RlbHMKCi0gICBTdXBwb3J0IFZlY3RvciBNYWNoaW5lcwoKLSAgIE5lYXJlc3QgTmVpZ2hib3JzCgotICAgRGVjaXNpb24gVHJlZXMKCi0gICBFbnNlbWJsZSBtZXRob2RzCgotICAgTmV1cmFsIE5ldHdvcmtzCgpUaGUgbGlzdCBnb2VzIG9uIQoKIyMjICoqV2hhdCBjbGFzc2lmaWVyIHRvIGdvIHdpdGg/KioKClNvLCB3aGljaCBjbGFzc2lmaWVyIHNob3VsZCB5b3UgY2hvb3NlPyBPZnRlbiwgcnVubmluZyB0aHJvdWdoIHNldmVyYWwgYW5kIGxvb2tpbmcgZm9yIGEgZ29vZCByZXN1bHQgaXMgYSB3YXkgdG8gdGVzdC4KCj4gQXV0b01MIHNvbHZlcyB0aGlzIHByb2JsZW0gbmVhdGx5IGJ5IHJ1bm5pbmcgdGhlc2UgY29tcGFyaXNvbnMgaW4gdGhlIGNsb3VkLCBhbGxvd2luZyB5b3UgdG8gY2hvb3NlIHRoZSBiZXN0IGFsZ29yaXRobSBmb3IgeW91ciBkYXRhLiBUcnkgaXQgW2hlcmVdKGh0dHBzOi8vZG9jcy5taWNyb3NvZnQuY29tL2xlYXJuL21vZHVsZXMvYXV0b21hdGUtbW9kZWwtc2VsZWN0aW9uLXdpdGgtYXp1cmUtYXV0b21sLz9XVC5tY19pZD1hY2FkZW1pYy03Nzk1Mi1sZWVzdG90dCkKCkFsc28gdGhlIGNob2ljZSBvZiBjbGFzc2lmaWVyIGRlcGVuZHMgb24gb3VyIHByb2JsZW0uIEZvciBpbnN0YW5jZSwgd2hlbiB0aGUgb3V0Y29tZSBjYW4gYmUgY2F0ZWdvcml6ZWQgaW50byBgbW9yZSB0aGFuIHR3byBjbGFzc2VzYCwgbGlrZSBpbiBvdXIgY2FzZSwgeW91IG11c3QgdXNlIGEgYG11bHRpY2xhc3MgY2xhc3NpZmljYXRpb24gYWxnb3JpdGhtYCBhcyBvcHBvc2VkIHRvIGBiaW5hcnkgY2xhc3NpZmljYXRpb24uYAoKIyMjICoqQSBiZXR0ZXIgYXBwcm9hY2gqKgoKQSBiZXR0ZXIgd2F5IHRoYW4gd2lsZGx5IGd1ZXNzaW5nLCBob3dldmVyLCBpcyB0byBmb2xsb3cgdGhlIGlkZWFzIG9uIHRoaXMgZG93bmxvYWRhYmxlIFtNTCBDaGVhdCBzaGVldF0oaHR0cHM6Ly9kb2NzLm1pY3Jvc29mdC5jb20vYXp1cmUvbWFjaGluZS1sZWFybmluZy9hbGdvcml0aG0tY2hlYXQtc2hlZXQ/V1QubWNfaWQ9YWNhZGVtaWMtNzc5NTItbGVlc3RvdHQpLiBIZXJlLCB3ZSBkaXNjb3ZlciB0aGF0LCBmb3Igb3VyIG11bHRpY2xhc3MgcHJvYmxlbSwgd2UgaGF2ZSBzb21lIGNob2ljZXM6CgohW0Egc2VjdGlvbiBvZiBNaWNyb3NvZnQncyBBbGdvcml0aG0gQ2hlYXQgU2hlZXQsIGRldGFpbGluZyBtdWx0aWNsYXNzIGNsYXNzaWZpY2F0aW9uIG9wdGlvbnNdKC4uLy4uL2ltYWdlcy9jaGVhdHNoZWV0LnBuZyl7d2lkdGg9IjUwMCJ9CgojIyMgKipSZWFzb25pbmcqKgoKTGV0J3Mgc2VlIGlmIHdlIGNhbiByZWFzb24gb3VyIHdheSB0aHJvdWdoIGRpZmZlcmVudCBhcHByb2FjaGVzIGdpdmVuIHRoZSBjb25zdHJhaW50cyB3ZSBoYXZlOgoKLSAgICoqRGVlcCBOZXVyYWwgbmV0d29ya3MgYXJlIHRvbyBoZWF2eSoqLiBHaXZlbiBvdXIgY2xlYW4sIGJ1dCBtaW5pbWFsIGRhdGFzZXQsIGFuZCB0aGUgZmFjdCB0aGF0IHdlIGFyZSBydW5uaW5nIHRyYWluaW5nIGxvY2FsbHkgdmlhIG5vdGVib29rcywgZGVlcCBuZXVyYWwgbmV0d29ya3MgYXJlIHRvbyBoZWF2eXdlaWdodCBmb3IgdGhpcyB0YXNrLgoKLSAgICoqTm8gdHdvLWNsYXNzIGNsYXNzaWZpZXIqKi4gV2UgZG8gbm90IHVzZSBhIHR3by1jbGFzcyBjbGFzc2lmaWVyLCBzbyB0aGF0IHJ1bGVzIG91dCBvbmUtdnMtYWxsLgoKLSAgICoqRGVjaXNpb24gdHJlZSBvciBsb2dpc3RpYyByZWdyZXNzaW9uIGNvdWxkIHdvcmsqKi4gQSBkZWNpc2lvbiB0cmVlIG1pZ2h0IHdvcmssIG9yIG11bHRpbm9taWFsIHJlZ3Jlc3Npb24vbXVsdGljbGFzcyBsb2dpc3RpYyByZWdyZXNzaW9uIGZvciBtdWx0aWNsYXNzIGRhdGEuCgotICAgKipNdWx0aWNsYXNzIEJvb3N0ZWQgRGVjaXNpb24gVHJlZXMgc29sdmUgYSBkaWZmZXJlbnQgcHJvYmxlbSoqLiBUaGUgbXVsdGljbGFzcyBib29zdGVkIGRlY2lzaW9uIHRyZWUgaXMgbW9zdCBzdWl0YWJsZSBmb3Igbm9ucGFyYW1ldHJpYyB0YXNrcywgZS5nLiB0YXNrcyBkZXNpZ25lZCB0byBidWlsZCByYW5raW5ncywgc28gaXQgaXMgbm90IHVzZWZ1bCBmb3IgdXMuCgpBbHNvLCBub3JtYWxseSBiZWZvcmUgZW1iYXJraW5nIG9uIG1vcmUgY29tcGxleCBtYWNoaW5lIGxlYXJuaW5nIG1vZGVscyBlLmcgZW5zZW1ibGUgbWV0aG9kcywgaXQncyBhIGdvb2QgaWRlYSB0byBidWlsZCB0aGUgc2ltcGxlc3QgcG9zc2libGUgbW9kZWwgdG8gZ2V0IGFuIGlkZWEgb2Ygd2hhdCBpcyBnb2luZyBvbi4gU28gZm9yIHRoaXMgbGVzc29uLCB3ZSdsbCBzdGFydCB3aXRoIGEgYG11bHRpbm9taWFsIGxvZ2lzdGljIHJlZ3Jlc3Npb25gIG1vZGVsLgoKPiBMb2dpc3RpYyByZWdyZXNzaW9uIGlzIGEgdGVjaG5pcXVlIHVzZWQgd2hlbiB0aGUgb3V0Y29tZSB2YXJpYWJsZSBpcyBjYXRlZ29yaWNhbCAob3Igbm9taW5hbCkuIEZvciBCaW5hcnkgbG9naXN0aWMgcmVncmVzc2lvbiB0aGUgbnVtYmVyIG9mIG91dGNvbWUgdmFyaWFibGVzIGlzIHR3bywgd2hlcmVhcyB0aGUgbnVtYmVyIG9mIG91dGNvbWUgdmFyaWFibGVzIGZvciBtdWx0aW5vbWlhbCBsb2dpc3RpYyByZWdyZXNzaW9uIGlzIG1vcmUgdGhhbiB0d28uIFNlZSBbQWR2YW5jZWQgUmVncmVzc2lvbiBNZXRob2RzXShodHRwczovL2Jvb2tkb3duLm9yZy9jaHVhL2JlcjY0Ml9hZHZhbmNlZF9yZWdyZXNzaW9uL211bHRpbm9taWFsLWxvZ2lzdGljLXJlZ3Jlc3Npb24uaHRtbCkgZm9yIGZ1cnRoZXIgcmVhZGluZy4KCiMjIDQuIFRyYWluIGFuZCBldmFsdWF0ZSBhIE11bHRpbm9taWFsIGxvZ2lzdGljIHJlZ3Jlc3Npb24gbW9kZWwuCgpJbiBUaWR5bW9kZWxzLCBgcGFyc25pcDo6bXVsdGlub21fcmVnKClgLCBkZWZpbmVzIGEgbW9kZWwgdGhhdCB1c2VzIGxpbmVhciBwcmVkaWN0b3JzIHRvIHByZWRpY3QgbXVsdGljbGFzcyBkYXRhIHVzaW5nIHRoZSBtdWx0aW5vbWlhbCBkaXN0cmlidXRpb24uIFNlZSBgP211bHRpbm9tX3JlZygpYCBmb3IgdGhlIGRpZmZlcmVudCB3YXlzL2VuZ2luZXMgeW91IGNhbiB1c2UgdG8gZml0IHRoaXMgbW9kZWwuCgpGb3IgdGhpcyBleGFtcGxlLCB3ZSdsbCBmaXQgYSBNdWx0aW5vbWlhbCByZWdyZXNzaW9uIG1vZGVsIHZpYSB0aGUgZGVmYXVsdCBbbm5ldF0oaHR0cHM6Ly9jcmFuLnItcHJvamVjdC5vcmcvd2ViL3BhY2thZ2VzL25uZXQvbm5ldC5wZGYpIGVuZ2luZS4KCj4gSSBwaWNrZWQgYSB2YWx1ZSBmb3IgYHBlbmFsdHlgIHNvcnQgb2YgcmFuZG9tbHkuIFRoZXJlIGFyZSBiZXR0ZXIgd2F5cyB0byBjaG9vc2UgdGhpcyB2YWx1ZSB0aGF0IGlzLCBieSB1c2luZyBgcmVzYW1wbGluZ2AgYW5kIGB0dW5pbmdgIHRoZSBtb2RlbCB3aGljaCB3ZSdsbCBkaXNjdXNzIGxhdGVyLgo+Cj4gU2VlIFtUaWR5bW9kZWxzOiBHZXQgU3RhcnRlZF0oaHR0cHM6Ly93d3cudGlkeW1vZGVscy5vcmcvc3RhcnQvdHVuaW5nLykgaW4gY2FzZSB5b3Ugd2FudCB0byBsZWFybiBtb3JlIG9uIGhvdyB0byB0dW5lIG1vZGVsIGh5cGVycGFyYW1ldGVycy4KCmBgYHtyIG11bHRpbm9ybV9yZWd9CiMgQ3JlYXRlIGEgbXVsdGlub21pYWwgcmVncmVzc2lvbiBtb2RlbCBzcGVjaWZpY2F0aW9uCm1yX3NwZWMgPC0gbXVsdGlub21fcmVnKHBlbmFsdHkgPSAxKSAlPiUgCiAgc2V0X2VuZ2luZSgibm5ldCIsIE1heE5XdHMgPSAyMDg2KSAlPiUgCiAgc2V0X21vZGUoImNsYXNzaWZpY2F0aW9uIikKCiMgUHJpbnQgbW9kZWwgc3BlY2lmaWNhdGlvbgptcl9zcGVjCgpgYGAKCkdyZWF0IGpvYiDwn6WzISBOb3cgdGhhdCB3ZSBoYXZlIGEgcmVjaXBlIGFuZCBhIG1vZGVsIHNwZWNpZmljYXRpb24sIHdlIG5lZWQgdG8gZmluZCBhIHdheSBvZiBidW5kbGluZyB0aGVtIHRvZ2V0aGVyIGludG8gYW4gb2JqZWN0IHRoYXQgd2lsbCBmaXJzdCBwcmVwcm9jZXNzIHRoZSBkYXRhIHRoZW4gZml0IHRoZSBtb2RlbCBvbiB0aGUgcHJlcHJvY2Vzc2VkIGRhdGEgYW5kIGFsc28gYWxsb3cgZm9yIHBvdGVudGlhbCBwb3N0LXByb2Nlc3NpbmcgYWN0aXZpdGllcy4gSW4gVGlkeW1vZGVscywgdGhpcyBjb252ZW5pZW50IG9iamVjdCBpcyBjYWxsZWQgYSBbYHdvcmtmbG93YF0oaHR0cHM6Ly93b3JrZmxvd3MudGlkeW1vZGVscy5vcmcvKSBhbmQgY29udmVuaWVudGx5IGhvbGRzIHlvdXIgbW9kZWxpbmcgY29tcG9uZW50cyEgVGhpcyBpcyB3aGF0IHdlJ2QgY2FsbCAqcGlwZWxpbmVzKiBpbiAqUHl0aG9uKi4KClNvIGxldCdzIGJ1bmRsZSBldmVyeXRoaW5nIHVwIGludG8gYSB3b3JrZmxvdyHwn5OmCgpgYGB7ciB3b3JrZmxvd30KIyBCdW5kbGUgcmVjaXBlIGFuZCBtb2RlbCBzcGVjaWZpY2F0aW9uCm1yX3dmIDwtIHdvcmtmbG93KCkgJT4lIAogIGFkZF9yZWNpcGUoY3Vpc2luZXNfcmVjaXBlKSAlPiUgCiAgYWRkX21vZGVsKG1yX3NwZWMpCgojIFByaW50IG91dCB3b3JrZmxvdwptcl93ZgoKYGBgCgpXb3JrZmxvd3Mg8J+RjPCfkYwhIEEgKipgd29ya2Zsb3coKWAqKiBjYW4gYmUgZml0IGluIG11Y2ggdGhlIHNhbWUgd2F5IGEgbW9kZWwgY2FuLiBTbywgdGltZSB0byB0cmFpbiBhIG1vZGVsIQoKYGBge3IgdHJhaW59CiMgVHJhaW4gYSBtdWx0aW5vbWlhbCByZWdyZXNzaW9uIG1vZGVsCm1yX2ZpdCA8LSBmaXQob2JqZWN0ID0gbXJfd2YsIGRhdGEgPSBjdWlzaW5lc190cmFpbikKCm1yX2ZpdApgYGAKClRoZSBvdXRwdXQgc2hvd3MgdGhlIGNvZWZmaWNpZW50cyB0aGF0IHRoZSBtb2RlbCBsZWFybmVkIGR1cmluZyB0cmFpbmluZy4KCiMjIyBFdmFsdWF0ZSB0aGUgVHJhaW5lZCBNb2RlbAoKSXQncyB0aW1lIHRvIHNlZSBob3cgdGhlIG1vZGVsIHBlcmZvcm1lZCDwn5OPIGJ5IGV2YWx1YXRpbmcgaXQgb24gYSB0ZXN0IHNldCEgTGV0J3MgYmVnaW4gYnkgbWFraW5nIHByZWRpY3Rpb25zIG9uIHRoZSB0ZXN0IHNldC4KCmBgYHtyIHRlc3R9CiMgTWFrZSBwcmVkaWN0aW9ucyBvbiB0aGUgdGVzdCBzZXQKcmVzdWx0cyA8LSBjdWlzaW5lc190ZXN0ICU+JSBzZWxlY3QoY3Vpc2luZSkgJT4lIAogIGJpbmRfY29scyhtcl9maXQgJT4lIHByZWRpY3QobmV3X2RhdGEgPSBjdWlzaW5lc190ZXN0KSkKCiMgUHJpbnQgb3V0IHJlc3VsdHMKcmVzdWx0cyAlPiUgCiAgc2xpY2VfaGVhZChuID0gNSkKCmBgYAoKR3JlYXQgam9iISBJbiBUaWR5bW9kZWxzLCBldmFsdWF0aW5nIG1vZGVsIHBlcmZvcm1hbmNlIGNhbiBiZSBkb25lIHVzaW5nIFt5YXJkc3RpY2tdKGh0dHBzOi8veWFyZHN0aWNrLnRpZHltb2RlbHMub3JnLykgLSBhIHBhY2thZ2UgdXNlZCB0byBtZWFzdXJlIHRoZSBlZmZlY3RpdmVuZXNzIG9mIG1vZGVscyB1c2luZyBwZXJmb3JtYW5jZSBtZXRyaWNzLiBBcyB3ZSBkaWQgaW4gb3VyIGxvZ2lzdGljIHJlZ3Jlc3Npb24gbGVzc29uLCBsZXQncyBiZWdpbiBieSBjb21wdXRpbmcgYSBjb25mdXNpb24gbWF0cml4LgoKYGBge3IgY29uZl9tYXR9CiMgQ29uZnVzaW9uIG1hdHJpeCBmb3IgY2F0ZWdvcmljYWwgZGF0YQpjb25mX21hdChkYXRhID0gcmVzdWx0cywgdHJ1dGggPSBjdWlzaW5lLCBlc3RpbWF0ZSA9IC5wcmVkX2NsYXNzKQoKCmBgYAoKV2hlbiBkZWFsaW5nIHdpdGggbXVsdGlwbGUgY2xhc3NlcywgaXQncyBnZW5lcmFsbHkgbW9yZSBpbnR1aXRpdmUgdG8gdmlzdWFsaXplIHRoaXMgYXMgYSBoZWF0IG1hcCwgbGlrZSB0aGlzOgoKYGBge3IgY29uZl92aXp9CnVwZGF0ZV9nZW9tX2RlZmF1bHRzKGdlb20gPSAidGlsZSIsIG5ldyA9IGxpc3QoY29sb3IgPSAiYmxhY2siLCBhbHBoYSA9IDAuNykpCiMgVmlzdWFsaXplIGNvbmZ1c2lvbiBtYXRyaXgKcmVzdWx0cyAlPiUgCiAgY29uZl9tYXQoY3Vpc2luZSwgLnByZWRfY2xhc3MpICU+JSAKICBhdXRvcGxvdCh0eXBlID0gImhlYXRtYXAiKQpgYGAKClRoZSBkYXJrZXIgc3F1YXJlcyBpbiB0aGUgY29uZnVzaW9uIG1hdHJpeCBwbG90IGluZGljYXRlIGhpZ2ggbnVtYmVycyBvZiBjYXNlcywgYW5kIHlvdSBjYW4gaG9wZWZ1bGx5IHNlZSBhIGRpYWdvbmFsIGxpbmUgb2YgZGFya2VyIHNxdWFyZXMgaW5kaWNhdGluZyBjYXNlcyB3aGVyZSB0aGUgcHJlZGljdGVkIGFuZCBhY3R1YWwgbGFiZWwgYXJlIHRoZSBzYW1lLgoKTGV0J3Mgbm93IGNhbGN1bGF0ZSBzdW1tYXJ5IHN0YXRpc3RpY3MgZm9yIHRoZSBjb25mdXNpb24gbWF0cml4LgoKYGBge3IgY29uZl9zdGF0c30KIyBTdW1tYXJ5IHN0YXRzIGZvciBjb25mdXNpb24gbWF0cml4CmNvbmZfbWF0KGRhdGEgPSByZXN1bHRzLCB0cnV0aCA9IGN1aXNpbmUsIGVzdGltYXRlID0gLnByZWRfY2xhc3MpICU+JSBzdW1tYXJ5KCkKYGBgCgpJZiB3ZSBuYXJyb3cgZG93biB0byBzb21lIG1ldHJpY3Mgc3VjaCBhcyBhY2N1cmFjeSwgc2Vuc2l0aXZpdHksIHBwdiwgd2UgYXJlIG5vdCBiYWRseSBvZmYgZm9yIGEgc3RhcnQg8J+lsyEKCiMjIDQuIERpZ2dpbmcgRGVlcGVyCgpMZXQncyBhc2sgb25lIHN1YnRsZSBxdWVzdGlvbjogV2hhdCBjcml0ZXJpYSBpcyB1c2VkIHRvIHNldHRsZSBmb3IgYSBnaXZlbiB0eXBlIG9mIGN1aXNpbmUgYXMgdGhlIHByZWRpY3RlZCBvdXRjb21lPwoKV2VsbCwgU3RhdGlzdGljYWwgbWFjaGluZSBsZWFybmluZyBhbGdvcml0aG1zLCBsaWtlIGxvZ2lzdGljIHJlZ3Jlc3Npb24sIGFyZSBiYXNlZCBvbiBgcHJvYmFiaWxpdHlgOyBzbyB3aGF0IGFjdHVhbGx5IGdldHMgcHJlZGljdGVkIGJ5IGEgY2xhc3NpZmllciBpcyBhIHByb2JhYmlsaXR5IGRpc3RyaWJ1dGlvbiBvdmVyIGEgc2V0IG9mIHBvc3NpYmxlIG91dGNvbWVzLiBUaGUgY2xhc3Mgd2l0aCB0aGUgaGlnaGVzdCBwcm9iYWJpbGl0eSBpcyB0aGVuIGNob3NlbiBhcyB0aGUgbW9zdCBsaWtlbHkgb3V0Y29tZSBmb3IgdGhlIGdpdmVuIG9ic2VydmF0aW9ucy4KCkxldCdzIHNlZSB0aGlzIGluIGFjdGlvbiBieSBtYWtpbmcgYm90aCBoYXJkIGNsYXNzIHByZWRpY3Rpb25zIGFuZCBwcm9iYWJpbGl0aWVzLgoKYGBge3IgcHJlZF9wcm9ifQojIE1ha2UgaGFyZCBjbGFzcyBwcmVkaWN0aW9uIGFuZCBwcm9iYWJpbGl0aWVzCnJlc3VsdHNfcHJvYiA8LSBjdWlzaW5lc190ZXN0ICU+JQogIHNlbGVjdChjdWlzaW5lKSAlPiUgCiAgYmluZF9jb2xzKG1yX2ZpdCAlPiUgcHJlZGljdChuZXdfZGF0YSA9IGN1aXNpbmVzX3Rlc3QpKSAlPiUgCiAgYmluZF9jb2xzKG1yX2ZpdCAlPiUgcHJlZGljdChuZXdfZGF0YSA9IGN1aXNpbmVzX3Rlc3QsIHR5cGUgPSAicHJvYiIpKQoKIyBQcmludCBvdXQgcmVzdWx0cwpyZXN1bHRzX3Byb2IgJT4lIAogIHNsaWNlX2hlYWQobiA9IDUpCiAgCgpgYGAKCk11Y2ggYmV0dGVyIQoK4pyFIENhbiB5b3UgZXhwbGFpbiB3aHkgdGhlIG1vZGVsIGlzIHByZXR0eSBzdXJlIHRoYXQgdGhlIGZpcnN0IG9ic2VydmF0aW9uIGlzIFRoYWk/CgojIyAqKvCfmoBDaGFsbGVuZ2UqKgoKSW4gdGhpcyBsZXNzb24sIHlvdSB1c2VkIHlvdXIgY2xlYW5lZCBkYXRhIHRvIGJ1aWxkIGEgbWFjaGluZSBsZWFybmluZyBtb2RlbCB0aGF0IGNhbiBwcmVkaWN0IGEgbmF0aW9uYWwgY3Vpc2luZSBiYXNlZCBvbiBhIHNlcmllcyBvZiBpbmdyZWRpZW50cy4gVGFrZSBzb21lIHRpbWUgdG8gcmVhZCB0aHJvdWdoIHRoZSBbbWFueSBvcHRpb25zXShodHRwczovL3d3dy50aWR5bW9kZWxzLm9yZy9maW5kL3BhcnNuaXAvI21vZGVscykgVGlkeW1vZGVscyBwcm92aWRlcyB0byBjbGFzc2lmeSBkYXRhIGFuZCBbb3RoZXIgd2F5c10oaHR0cHM6Ly9wYXJzbmlwLnRpZHltb2RlbHMub3JnL2FydGljbGVzL2FydGljbGVzL0V4YW1wbGVzLmh0bWwjbXVsdGlub21fcmVnLW1vZGVscykgdG8gZml0IG11bHRpbm9taWFsIHJlZ3Jlc3Npb24uCgojIyMjIFRIQU5LIFlPVSBUTzoKCltgQWxsaXNvbiBIb3JzdGBdKGh0dHBzOi8vdHdpdHRlci5jb20vYWxsaXNvbl9ob3JzdC8pIGZvciBjcmVhdGluZyB0aGUgYW1hemluZyBpbGx1c3RyYXRpb25zIHRoYXQgbWFrZSBSIG1vcmUgd2VsY29taW5nIGFuZCBlbmdhZ2luZy4gRmluZCBtb3JlIGlsbHVzdHJhdGlvbnMgYXQgaGVyIFtnYWxsZXJ5XShodHRwczovL3d3dy5nb29nbGUuY29tL3VybD9xPWh0dHBzOi8vZ2l0aHViLmNvbS9hbGxpc29uaG9yc3Qvc3RhdHMtaWxsdXN0cmF0aW9ucyZzYT1EJnNvdXJjZT1lZGl0b3JzJnVzdD0xNjI2MzgwNzcyNTMwMDAwJnVzZz1BT3ZWYXczemNmeUNpekZRWnBrU0x6eGlpUUVNKS4KCltDYXNzaWUgQnJldml1XShodHRwczovL3d3dy50d2l0dGVyLmNvbS9jYXNzaWV2aWV3KSBhbmQgW0plbiBMb29wZXJdKGh0dHBzOi8vd3d3LnR3aXR0ZXIuY29tL2plbmxvb3BlcikgZm9yIGNyZWF0aW5nIHRoZSBvcmlnaW5hbCBQeXRob24gdmVyc2lvbiBvZiB0aGlzIG1vZHVsZSDimaXvuI8KCkhhcHB5IExlYXJuaW5nLAoKW0VyaWNdKGh0dHBzOi8vdHdpdHRlci5jb20vZXJpY250YXkpLCBHb2xkIE1pY3Jvc29mdCBMZWFybiBTdHVkZW50IEFtYmFzc2Fkb3IuCg==
+
+
+