+
+
+
+
+
+
+
+
+
Cuisine classifiers 2
+
In this second classification lesson, we will explore
+more ways to classify categorical data. We will also learn
+about the ramifications for choosing one classifier over the other.
+
+
+
Prerequisite
+
We assume that you have completed the previous lessons since we will
+be carrying forward some concepts we learned before.
+
For this lesson, we’ll require the following packages:
+
+
You can have them installed as:
+
install.packages(c("tidyverse", "tidymodels", "kernlab", "themis", "ranger", "xgboost", "kknn"))
+
Alternatively, the script below checks whether you have the packages
+required to complete this module and installs them for you in case they
+are missing.
+
suppressWarnings(if (!require("pacman"))install.packages("pacman"))
+
+pacman::p_load(tidyverse, tidymodels, themis, kernlab, ranger, xgboost, kknn)
+
##
+## The downloaded binary packages are in
+## /var/folders/c9/r3f6t3kj3wv9jrh50g63hp1r0000gn/T//RtmpE2TSCy/downloaded_packages
+##
+## The downloaded binary packages are in
+## /var/folders/c9/r3f6t3kj3wv9jrh50g63hp1r0000gn/T//RtmpE2TSCy/downloaded_packages
+##
+## The downloaded binary packages are in
+## /var/folders/c9/r3f6t3kj3wv9jrh50g63hp1r0000gn/T//RtmpE2TSCy/downloaded_packages
+##
+## The downloaded binary packages are in
+## /var/folders/c9/r3f6t3kj3wv9jrh50g63hp1r0000gn/T//RtmpE2TSCy/downloaded_packages
+
Now, let’s hit the ground running!
+
+
+
+
1. A classification map
+
In our previous
+lesson, we tried to address the question: how do we choose between
+multiple models? To a great extent, it depends on the characteristics of
+the data and the type of problem we want to solve (for instance
+classification or regression?)
+
Previously, we learned about the various options you have when
+classifying data using Microsoft’s cheat sheet. Python’s Machine
+Learning framework, Scikit-learn, offers a similar but more granular
+cheat sheet that can further help narrow down your estimators (another
+term for classifiers):
+
+
+
+Tip: visit
+this map online and click along the path to read documentation.
+The Tidymodels
+reference site also provides an excellent documentation about
+different types of model.
+
+
+
The plan 🗺️
+
This map is very helpful once you have a clear grasp of your data, as
+you can ‘walk’ along its paths to a decision:
+
+We have >50 samples
We want to predict a category
We have labeled data
We have fewer than 100K samples
✨ We can choose a Linear SVC
If that doesn’t work, since we have numeric data
+
+
This is a very helpful trail to follow. Now, let’s get right into it
+using the tidymodels modelling
+framework: a consistent and flexible collection of R packages developed
+to encourage good statistical practice 😊.
+
+
+
+
2. Split the data and deal with imbalanced data set.
+
From our previous lessons, we learnt that there were a set of common
+ingredients across our cuisines. Also, there was quite an unequal
+distribution in the number of cuisines.
+
We’ll deal with these by
+
+Dropping the most common ingredients that create confusion
+between distinct cuisines, using dplyr::select().
Use a recipe that preprocesses the data to get it
+ready for modelling by applying an over-sampling
+algorithm.
+
We already looked at the above in the previous lesson so this should
+be a breeze 🥳!
+
# Load the core Tidyverse and Tidymodels packages
+library(tidyverse)
+library(tidymodels)
+
+# Load the original cuisines data
+df <- read_csv(file = "https://raw.githubusercontent.com/microsoft/ML-For-Beginners/main/4-Classification/data/cuisines.csv")
+
## New names:
+## Rows: 2448 Columns: 385
+## ── Column specification
+## ──────────────────────────────────────────────────────── Delimiter: "," chr
+## (1): cuisine dbl (384): ...1, almond, angelica, anise, anise_seed, apple,
+## apple_brandy, a...
+## ℹ Use `spec()` to retrieve the full column specification for this data. ℹ
+## Specify the column types or set `show_col_types = FALSE` to quiet this message.
+## • `` -> `...1`
+
# Drop id column, rice, garlic and ginger from our original data set
+df_select <- df %>%
+ select(-c(1, rice, garlic, ginger)) %>%
+ # Encode cuisine column as categorical
+ mutate(cuisine = factor(cuisine))
+
+
+# Create data split specification
+set.seed(2056)
+cuisines_split <- initial_split(data = df_select,
+ strata = cuisine,
+ prop = 0.7)
+
+# Extract the data in each split
+cuisines_train <- training(cuisines_split)
+cuisines_test <- testing(cuisines_split)
+
+# Display distribution of cuisines in the training set
+cuisines_train %>%
+ count(cuisine) %>%
+ arrange(desc(n))
+
+
+
+
+
Deal with imbalanced data
+
Imbalanced data often has negative effects on the model performance.
+Many models perform best when the number of observations is equal and,
+thus, tend to struggle with unbalanced data.
+
There are majorly two ways of dealing with imbalanced data sets:
+
+adding observations to the minority class:
+Over-sampling e.g using a SMOTE algorithm which
+synthetically generates new examples of the minority class using nearest
+neighbors of these cases.
removing observations from majority class:
+Under-sampling
+
In our previous lesson, we demonstrated how to deal with imbalanced
+data sets using a recipe. A recipe can be thought of as a
+blueprint that describes what steps should be applied to a data set in
+order to get it ready for data analysis. In our case, we want to have an
+equal distribution in the number of our cuisines for our
+training set. Let’s get right into it.
+
# Load themis package for dealing with imbalanced data
+library(themis)
+
+# Create a recipe for preprocessing training data
+cuisines_recipe <- recipe(cuisine ~ ., data = cuisines_train) %>%
+ step_smote(cuisine)
+
+# Print recipe
+cuisines_recipe
+
##
+
## ── Recipe ──────────────────────────────────────────────────────────────────────
+
##
+
## ── Inputs
+
## Number of variables by role
+
## outcome: 1
+## predictor: 380
+
##
+
## ── Operations
+
## • SMOTE based on: cuisine
+
Now we are ready to train models 👩💻👨💻!
+
+
+
+
3. Beyond multinomial regression models
+
In our previous lesson, we looked at multinomial regression models.
+Let’s explore some more flexible models for classification.
+
+
Support Vector Machines.
+
In the context of classification,
+Support Vector Machines is a machine learning technique
+that tries to find a hyperplane that “best” separates the
+classes. Let’s look at a simple example:
+
+
+
+
+
H1 does not separate the classes. H2 does, but
+only with a small margin. H3 separates them with the maximal
+margin.
+
+
Linear Support Vector Classifier
+
Support-Vector clustering (SVC) is a child of the Support-Vector
+machines family of ML techniques. In SVC, the hyperplane is chosen to
+correctly separate most of the training observations, but
+may misclassify a few observations. By allowing some points
+to be on the wrong side, the SVM becomes more robust to outliers hence
+better generalization to new data. The parameter that regulates this
+violation is referred to as cost which has a default value
+of 1 (see help("svm_poly")).
+
Let’s create a linear SVC by setting degree = 1 in a
+polynomial SVM model.
+
# Make a linear SVC specification
+svc_linear_spec <- svm_poly(degree = 1) %>%
+ set_engine("kernlab") %>%
+ set_mode("classification")
+
+# Bundle specification and recipe into a worklow
+svc_linear_wf <- workflow() %>%
+ add_recipe(cuisines_recipe) %>%
+ add_model(svc_linear_spec)
+
+# Print out workflow
+svc_linear_wf
+
## ══ Workflow ════════════════════════════════════════════════════════════════════
+## Preprocessor: Recipe
+## Model: svm_poly()
+##
+## ── Preprocessor ────────────────────────────────────────────────────────────────
+## 1 Recipe Step
+##
+## • step_smote()
+##
+## ── Model ───────────────────────────────────────────────────────────────────────
+## Polynomial Support Vector Machine Model Specification (classification)
+##
+## Main Arguments:
+## degree = 1
+##
+## Computational engine: kernlab
+
Now that we have captured the preprocessing steps and model
+specification into a workflow, we can go ahead and train the
+linear SVC and evaluate results while at it. For performance metrics,
+let’s create a metric set that will evaluate: accuracy,
+sensitivity, Positive Predicted Value and
+F Measure
+
+augment() will add column(s) for predictions to the
+given data.
+
+
# Train a linear SVC model
+svc_linear_fit <- svc_linear_wf %>%
+ fit(data = cuisines_train)
+
## Warning in .local(x, ...): Variable(s) `' constant. Cannot scale data.
+
# Create a metric set
+eval_metrics <- metric_set(ppv, sens, accuracy, f_meas)
+
+
+# Make predictions and Evaluate model performance
+svc_linear_fit %>%
+ augment(new_data = cuisines_test) %>%
+ eval_metrics(truth = cuisine, estimate = .pred_class)
+
+
+
+
+
+
+
+
+
Support Vector Machine
+
The support vector machine (SVM) is an extension of the support
+vector classifier in order to accommodate a non-linear boundary between
+the classes. In essence, SVMs use the kernel trick to enlarge
+the feature space to adapt to nonlinear relationships between classes.
+One popular and extremely flexible kernel function used by SVMs is the
+Radial basis function. Let’s see how it will perform on our
+data.
+
set.seed(2056)
+
+# Make an RBF SVM specification
+svm_rbf_spec <- svm_rbf() %>%
+ set_engine("kernlab") %>%
+ set_mode("classification")
+
+# Bundle specification and recipe into a worklow
+svm_rbf_wf <- workflow() %>%
+ add_recipe(cuisines_recipe) %>%
+ add_model(svm_rbf_spec)
+
+
+# Train an RBF model
+svm_rbf_fit <- svm_rbf_wf %>%
+ fit(data = cuisines_train)
+
## Warning in .local(x, ...): Variable(s) `' constant. Cannot scale data.
+
# Make predictions and Evaluate model performance
+svm_rbf_fit %>%
+ augment(new_data = cuisines_test) %>%
+ eval_metrics(truth = cuisine, estimate = .pred_class)
+
+
+
+
Much better 🤩!
+
+✅ Please see:
+
+for further reading.
+
+
+
+
+
Nearest Neighbor classifiers
+
K-nearest neighbor (KNN) is an algorithm in which each
+observation is predicted based on its similarity to other
+observations.
+
Let’s fit one to our data.
+
# Make a KNN specification
+knn_spec <- nearest_neighbor() %>%
+ set_engine("kknn") %>%
+ set_mode("classification")
+
+# Bundle recipe and model specification into a workflow
+knn_wf <- workflow() %>%
+ add_recipe(cuisines_recipe) %>%
+ add_model(knn_spec)
+
+# Train a boosted tree model
+knn_wf_fit <- knn_wf %>%
+ fit(data = cuisines_train)
+
+
+# Make predictions and Evaluate model performance
+knn_wf_fit %>%
+ augment(new_data = cuisines_test) %>%
+ eval_metrics(truth = cuisine, estimate = .pred_class)
+
+
+
+
It appears that this model is not performing that well. Probably
+changing the model’s arguments (see
+help("nearest_neighbor") will improve model performance. Be
+sure to try it out.
+
+✅ Please see:
+
+to learn more about K-Nearest Neighbors classifiers.
+
+
+
+
Ensemble classifiers
+
Ensemble algorithms work by combining multiple base estimators to
+produce an optimal model either by:
+
bagging: applying an averaging function to a
+collection of base models
+
boosting: building a sequence of models that build on
+one another to improve predictive performance.
+
Let’s start by trying out a Random Forest model, which builds a large
+collection of decision trees then applies an averaging function to for a
+better overall model.
+
# Make a random forest specification
+rf_spec <- rand_forest() %>%
+ set_engine("ranger") %>%
+ set_mode("classification")
+
+# Bundle recipe and model specification into a workflow
+rf_wf <- workflow() %>%
+ add_recipe(cuisines_recipe) %>%
+ add_model(rf_spec)
+
+# Train a random forest model
+rf_wf_fit <- rf_wf %>%
+ fit(data = cuisines_train)
+
+
+# Make predictions and Evaluate model performance
+rf_wf_fit %>%
+ augment(new_data = cuisines_test) %>%
+ eval_metrics(truth = cuisine, estimate = .pred_class)
+
+
+
+
Good job 👏!
+
Let’s also experiment with a Boosted Tree model.
+
Boosted Tree defines an ensemble method that creates a series of
+sequential decision trees where each tree depends on the results of
+previous trees in an attempt to incrementally reduce the error. It
+focuses on the weights of incorrectly classified items and adjusts the
+fit for the next classifier to correct.
+
There are different ways to fit this model (see
+help("boost_tree")). In this example, we’ll fit Boosted
+trees via xgboost engine.
+
# Make a boosted tree specification
+boost_spec <- boost_tree(trees = 200) %>%
+ set_engine("xgboost") %>%
+ set_mode("classification")
+
+# Bundle recipe and model specification into a workflow
+boost_wf <- workflow() %>%
+ add_recipe(cuisines_recipe) %>%
+ add_model(boost_spec)
+
+# Train a boosted tree model
+boost_wf_fit <- boost_wf %>%
+ fit(data = cuisines_train)
+
+
+# Make predictions and Evaluate model performance
+boost_wf_fit %>%
+ augment(new_data = cuisines_test) %>%
+ eval_metrics(truth = cuisine, estimate = .pred_class)
+
+
+
+
+✅ Please see:
+
+to learn more about Ensemble classifiers.
+
+
+
+
+
+
🚀Challenge
+
Each of these techniques has a large number of parameters that you
+can tweak for instance cost in SVMs, neighbors
+in KNN, mtry (Randomly Selected Predictors) in Random
+Forest.
+
Research each one’s default parameters and think about what tweaking
+these parameters would mean for the model’s quality.
+
To find out more about a particular model and its parameters, use:
+help("model") e.g help("rand_forest")
+
+In practice, we usually estimate the best values
+for these by training many models on a simulated data set
+and measuring how well all these models perform. This process is called
+tuning.
+
+
+
+
Review & Self Study
+
There’s a lot of jargon in these lessons, so take a minute to review
+this
+list of useful terminology!
+
+
THANK YOU TO:
+
Allison Horst
+for creating the amazing illustrations that make R more welcoming and
+engaging. Find more illustrations at her gallery.
+
Cassie Breviu and Jen Looper for creating the
+original Python version of this module ♥️
+
Happy Learning,
+
Eric, Gold Microsoft Learn
+Student Ambassador.
+
+
+
Artwork by @allison_horst
+
+
+
+
+
+
LS0tCnRpdGxlOiAnQnVpbGQgYSBjbGFzc2lmaWNhdGlvbiBtb2RlbDogRGVsaWNpb3VzIEFzaWFuIGFuZCBJbmRpYW4gQ3Vpc2luZXMnCm91dHB1dDoKICBodG1sX2RvY3VtZW50OgogICAgZGZfcHJpbnQ6IHBhZ2VkCiAgICB0aGVtZTogZmxhdGx5CiAgICBoaWdobGlnaHQ6IGJyZWV6ZWRhcmsKICAgIHRvYzogeWVzCiAgICB0b2NfZmxvYXQ6IHllcwogICAgY29kZV9kb3dubG9hZDogeWVzCi0tLQoKIyMgQ3Vpc2luZSBjbGFzc2lmaWVycyAyCgpJbiB0aGlzIHNlY29uZCBjbGFzc2lmaWNhdGlvbiBsZXNzb24sIHdlIHdpbGwgZXhwbG9yZSBgbW9yZSB3YXlzYCB0byBjbGFzc2lmeSBjYXRlZ29yaWNhbCBkYXRhLiBXZSB3aWxsIGFsc28gbGVhcm4gYWJvdXQgdGhlIHJhbWlmaWNhdGlvbnMgZm9yIGNob29zaW5nIG9uZSBjbGFzc2lmaWVyIG92ZXIgdGhlIG90aGVyLgoKIyMjIFsqKlByZS1sZWN0dXJlIHF1aXoqKl0oaHR0cHM6Ly9ncmF5LXNhbmQtMDdhMTBmNDAzLjEuYXp1cmVzdGF0aWNhcHBzLm5ldC9xdWl6LzIzLykKCiMjIyAqKlByZXJlcXVpc2l0ZSoqCgpXZSBhc3N1bWUgdGhhdCB5b3UgaGF2ZSBjb21wbGV0ZWQgdGhlIHByZXZpb3VzIGxlc3NvbnMgc2luY2Ugd2Ugd2lsbCBiZSBjYXJyeWluZyBmb3J3YXJkIHNvbWUgY29uY2VwdHMgd2UgbGVhcm5lZCBiZWZvcmUuCgpGb3IgdGhpcyBsZXNzb24sIHdlJ2xsIHJlcXVpcmUgdGhlIGZvbGxvd2luZyBwYWNrYWdlczoKCi0gICBgdGlkeXZlcnNlYDogVGhlIFt0aWR5dmVyc2VdKGh0dHBzOi8vd3d3LnRpZHl2ZXJzZS5vcmcvKSBpcyBhIFtjb2xsZWN0aW9uIG9mIFIgcGFja2FnZXNdKGh0dHBzOi8vd3d3LnRpZHl2ZXJzZS5vcmcvcGFja2FnZXMpIGRlc2lnbmVkIHRvIG1ha2VzIGRhdGEgc2NpZW5jZSBmYXN0ZXIsIGVhc2llciBhbmQgbW9yZSBmdW4hCgotICAgYHRpZHltb2RlbHNgOiBUaGUgW3RpZHltb2RlbHNdKGh0dHBzOi8vd3d3LnRpZHltb2RlbHMub3JnLykgZnJhbWV3b3JrIGlzIGEgW2NvbGxlY3Rpb24gb2YgcGFja2FnZXNdKGh0dHBzOi8vd3d3LnRpZHltb2RlbHMub3JnL3BhY2thZ2VzLykgZm9yIG1vZGVsaW5nIGFuZCBtYWNoaW5lIGxlYXJuaW5nLgoKLSAgIGB0aGVtaXNgOiBUaGUgW3RoZW1pcyBwYWNrYWdlXShodHRwczovL3RoZW1pcy50aWR5bW9kZWxzLm9yZy8pIHByb3ZpZGVzIEV4dHJhIFJlY2lwZXMgU3RlcHMgZm9yIERlYWxpbmcgd2l0aCBVbmJhbGFuY2VkIERhdGEuCgpZb3UgY2FuIGhhdmUgdGhlbSBpbnN0YWxsZWQgYXM6CgpgaW5zdGFsbC5wYWNrYWdlcyhjKCJ0aWR5dmVyc2UiLCAidGlkeW1vZGVscyIsICJrZXJubGFiIiwgInRoZW1pcyIsICJyYW5nZXIiLCAieGdib29zdCIsICJra25uIikpYAoKQWx0ZXJuYXRpdmVseSwgdGhlIHNjcmlwdCBiZWxvdyBjaGVja3Mgd2hldGhlciB5b3UgaGF2ZSB0aGUgcGFja2FnZXMgcmVxdWlyZWQgdG8gY29tcGxldGUgdGhpcyBtb2R1bGUgYW5kIGluc3RhbGxzIHRoZW0gZm9yIHlvdSBpbiBjYXNlIHRoZXkgYXJlIG1pc3NpbmcuCgpgYGB7ciwgbWVzc2FnZT1GLCB3YXJuaW5nPUZ9CnN1cHByZXNzV2FybmluZ3MoaWYgKCFyZXF1aXJlKCJwYWNtYW4iKSlpbnN0YWxsLnBhY2thZ2VzKCJwYWNtYW4iKSkKCnBhY21hbjo6cF9sb2FkKHRpZHl2ZXJzZSwgdGlkeW1vZGVscywgdGhlbWlzLCBrZXJubGFiLCByYW5nZXIsIHhnYm9vc3QsIGtrbm4pCmBgYAoKTm93LCBsZXQncyBoaXQgdGhlIGdyb3VuZCBydW5uaW5nIQoKIyMgKioxLiBBIGNsYXNzaWZpY2F0aW9uIG1hcCoqCgpJbiBvdXIgW3ByZXZpb3VzIGxlc3Nvbl0oaHR0cHM6Ly9naXRodWIuY29tL21pY3Jvc29mdC9NTC1Gb3ItQmVnaW5uZXJzL3RyZWUvbWFpbi80LUNsYXNzaWZpY2F0aW9uLzItQ2xhc3NpZmllcnMtMSksIHdlIHRyaWVkIHRvIGFkZHJlc3MgdGhlIHF1ZXN0aW9uOiBob3cgZG8gd2UgY2hvb3NlIGJldHdlZW4gbXVsdGlwbGUgbW9kZWxzPyBUbyBhIGdyZWF0IGV4dGVudCwgaXQgZGVwZW5kcyBvbiB0aGUgY2hhcmFjdGVyaXN0aWNzIG9mIHRoZSBkYXRhIGFuZCB0aGUgdHlwZSBvZiBwcm9ibGVtIHdlIHdhbnQgdG8gc29sdmUgKGZvciBpbnN0YW5jZSBjbGFzc2lmaWNhdGlvbiBvciByZWdyZXNzaW9uPykKClByZXZpb3VzbHksIHdlIGxlYXJuZWQgYWJvdXQgdGhlIHZhcmlvdXMgb3B0aW9ucyB5b3UgaGF2ZSB3aGVuIGNsYXNzaWZ5aW5nIGRhdGEgdXNpbmcgTWljcm9zb2Z0J3MgY2hlYXQgc2hlZXQuIFB5dGhvbidzIE1hY2hpbmUgTGVhcm5pbmcgZnJhbWV3b3JrLCBTY2lraXQtbGVhcm4sIG9mZmVycyBhIHNpbWlsYXIgYnV0IG1vcmUgZ3JhbnVsYXIgY2hlYXQgc2hlZXQgdGhhdCBjYW4gZnVydGhlciBoZWxwIG5hcnJvdyBkb3duIHlvdXIgZXN0aW1hdG9ycyAoYW5vdGhlciB0ZXJtIGZvciBjbGFzc2lmaWVycyk6CgohW10oLi4vLi4vaW1hZ2VzL21hcC5wbmcpe3dpZHRoPSI2NTAifVwKCj4gVGlwOiBbdmlzaXQgdGhpcyBtYXAgb25saW5lXShodHRwczovL3NjaWtpdC1sZWFybi5vcmcvc3RhYmxlL3R1dG9yaWFsL21hY2hpbmVfbGVhcm5pbmdfbWFwLykgYW5kIGNsaWNrIGFsb25nIHRoZSBwYXRoIHRvIHJlYWQgZG9jdW1lbnRhdGlvbi4KPgo+IFRoZSBbVGlkeW1vZGVscyByZWZlcmVuY2Ugc2l0ZV0oaHR0cHM6Ly93d3cudGlkeW1vZGVscy5vcmcvZmluZC9wYXJzbmlwLyNtb2RlbHMpIGFsc28gcHJvdmlkZXMgYW4gZXhjZWxsZW50IGRvY3VtZW50YXRpb24gYWJvdXQgZGlmZmVyZW50IHR5cGVzIG9mIG1vZGVsLgoKIyMjICoqVGhlIHBsYW4qKiDwn5e677iPCgpUaGlzIG1hcCBpcyB2ZXJ5IGhlbHBmdWwgb25jZSB5b3UgaGF2ZSBhIGNsZWFyIGdyYXNwIG9mIHlvdXIgZGF0YSwgYXMgeW91IGNhbiAnd2FsaycgYWxvbmcgaXRzIHBhdGhzIHRvIGEgZGVjaXNpb246CgotICAgV2UgaGF2ZSBcPjUwIHNhbXBsZXMKCi0gICBXZSB3YW50IHRvIHByZWRpY3QgYSBjYXRlZ29yeQoKLSAgIFdlIGhhdmUgbGFiZWxlZCBkYXRhCgotICAgV2UgaGF2ZSBmZXdlciB0aGFuIDEwMEsgc2FtcGxlcwoKLSAgIOKcqCBXZSBjYW4gY2hvb3NlIGEgTGluZWFyIFNWQwoKLSAgIElmIHRoYXQgZG9lc24ndCB3b3JrLCBzaW5jZSB3ZSBoYXZlIG51bWVyaWMgZGF0YQoKICAgIC0gICBXZSBjYW4gdHJ5IGEg4pyoIEtOZWlnaGJvcnMgQ2xhc3NpZmllcgoKICAgICAgICAtICAgSWYgdGhhdCBkb2Vzbid0IHdvcmssIHRyeSDinKggU1ZDIGFuZCDinKggRW5zZW1ibGUgQ2xhc3NpZmllcnMKClRoaXMgaXMgYSB2ZXJ5IGhlbHBmdWwgdHJhaWwgdG8gZm9sbG93LiBOb3csIGxldCdzIGdldCByaWdodCBpbnRvIGl0IHVzaW5nIHRoZSBbdGlkeW1vZGVsc10oaHR0cHM6Ly93d3cudGlkeW1vZGVscy5vcmcvKSBtb2RlbGxpbmcgZnJhbWV3b3JrOiBhIGNvbnNpc3RlbnQgYW5kIGZsZXhpYmxlIGNvbGxlY3Rpb24gb2YgUiBwYWNrYWdlcyBkZXZlbG9wZWQgdG8gZW5jb3VyYWdlIGdvb2Qgc3RhdGlzdGljYWwgcHJhY3RpY2Ug8J+Yii4KCiMjIDIuIFNwbGl0IHRoZSBkYXRhIGFuZCBkZWFsIHdpdGggaW1iYWxhbmNlZCBkYXRhIHNldC4KCkZyb20gb3VyIHByZXZpb3VzIGxlc3NvbnMsIHdlIGxlYXJudCB0aGF0IHRoZXJlIHdlcmUgYSBzZXQgb2YgY29tbW9uIGluZ3JlZGllbnRzIGFjcm9zcyBvdXIgY3Vpc2luZXMuIEFsc28sIHRoZXJlIHdhcyBxdWl0ZSBhbiB1bmVxdWFsIGRpc3RyaWJ1dGlvbiBpbiB0aGUgbnVtYmVyIG9mIGN1aXNpbmVzLgoKV2UnbGwgZGVhbCB3aXRoIHRoZXNlIGJ5CgotICAgRHJvcHBpbmcgdGhlIG1vc3QgY29tbW9uIGluZ3JlZGllbnRzIHRoYXQgY3JlYXRlIGNvbmZ1c2lvbiBiZXR3ZWVuIGRpc3RpbmN0IGN1aXNpbmVzLCB1c2luZyBgZHBseXI6OnNlbGVjdCgpYC4KCi0gICBVc2UgYSBgcmVjaXBlYCB0aGF0IHByZXByb2Nlc3NlcyB0aGUgZGF0YSB0byBnZXQgaXQgcmVhZHkgZm9yIG1vZGVsbGluZyBieSBhcHBseWluZyBhbiBgb3Zlci1zYW1wbGluZ2AgYWxnb3JpdGhtLgoKV2UgYWxyZWFkeSBsb29rZWQgYXQgdGhlIGFib3ZlIGluIHRoZSBwcmV2aW91cyBsZXNzb24gc28gdGhpcyBzaG91bGQgYmUgYSBicmVlemUg8J+lsyEKCmBgYHtyIGNsZWFuX2ltYmFsYW5jZX0KIyBMb2FkIHRoZSBjb3JlIFRpZHl2ZXJzZSBhbmQgVGlkeW1vZGVscyBwYWNrYWdlcwpsaWJyYXJ5KHRpZHl2ZXJzZSkKbGlicmFyeSh0aWR5bW9kZWxzKQoKIyBMb2FkIHRoZSBvcmlnaW5hbCBjdWlzaW5lcyBkYXRhCmRmIDwtIHJlYWRfY3N2KGZpbGUgPSAiaHR0cHM6Ly9yYXcuZ2l0aHVidXNlcmNvbnRlbnQuY29tL21pY3Jvc29mdC9NTC1Gb3ItQmVnaW5uZXJzL21haW4vNC1DbGFzc2lmaWNhdGlvbi9kYXRhL2N1aXNpbmVzLmNzdiIpCgojIERyb3AgaWQgY29sdW1uLCByaWNlLCBnYXJsaWMgYW5kIGdpbmdlciBmcm9tIG91ciBvcmlnaW5hbCBkYXRhIHNldApkZl9zZWxlY3QgPC0gZGYgJT4lIAogIHNlbGVjdCgtYygxLCByaWNlLCBnYXJsaWMsIGdpbmdlcikpICU+JQogICMgRW5jb2RlIGN1aXNpbmUgY29sdW1uIGFzIGNhdGVnb3JpY2FsCiAgbXV0YXRlKGN1aXNpbmUgPSBmYWN0b3IoY3Vpc2luZSkpCgoKIyBDcmVhdGUgZGF0YSBzcGxpdCBzcGVjaWZpY2F0aW9uCnNldC5zZWVkKDIwNTYpCmN1aXNpbmVzX3NwbGl0IDwtIGluaXRpYWxfc3BsaXQoZGF0YSA9IGRmX3NlbGVjdCwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBzdHJhdGEgPSBjdWlzaW5lLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHByb3AgPSAwLjcpCgojIEV4dHJhY3QgdGhlIGRhdGEgaW4gZWFjaCBzcGxpdApjdWlzaW5lc190cmFpbiA8LSB0cmFpbmluZyhjdWlzaW5lc19zcGxpdCkKY3Vpc2luZXNfdGVzdCA8LSB0ZXN0aW5nKGN1aXNpbmVzX3NwbGl0KQoKIyBEaXNwbGF5IGRpc3RyaWJ1dGlvbiBvZiBjdWlzaW5lcyBpbiB0aGUgdHJhaW5pbmcgc2V0CmN1aXNpbmVzX3RyYWluICU+JSAKICBjb3VudChjdWlzaW5lKSAlPiUgCiAgYXJyYW5nZShkZXNjKG4pKQoKCmBgYAoKIyMjIERlYWwgd2l0aCBpbWJhbGFuY2VkIGRhdGEKCkltYmFsYW5jZWQgZGF0YSBvZnRlbiBoYXMgbmVnYXRpdmUgZWZmZWN0cyBvbiB0aGUgbW9kZWwgcGVyZm9ybWFuY2UuIE1hbnkgbW9kZWxzIHBlcmZvcm0gYmVzdCB3aGVuIHRoZSBudW1iZXIgb2Ygb2JzZXJ2YXRpb25zIGlzIGVxdWFsIGFuZCwgdGh1cywgdGVuZCB0byBzdHJ1Z2dsZSB3aXRoIHVuYmFsYW5jZWQgZGF0YS4KClRoZXJlIGFyZSBtYWpvcmx5IHR3byB3YXlzIG9mIGRlYWxpbmcgd2l0aCBpbWJhbGFuY2VkIGRhdGEgc2V0czoKCi0gICBhZGRpbmcgb2JzZXJ2YXRpb25zIHRvIHRoZSBtaW5vcml0eSBjbGFzczogYE92ZXItc2FtcGxpbmdgIGUuZyB1c2luZyBhIFNNT1RFIGFsZ29yaXRobSB3aGljaCBzeW50aGV0aWNhbGx5IGdlbmVyYXRlcyBuZXcgZXhhbXBsZXMgb2YgdGhlIG1pbm9yaXR5IGNsYXNzIHVzaW5nIG5lYXJlc3QgbmVpZ2hib3JzIG9mIHRoZXNlIGNhc2VzLgoKLSAgIHJlbW92aW5nIG9ic2VydmF0aW9ucyBmcm9tIG1ham9yaXR5IGNsYXNzOiBgVW5kZXItc2FtcGxpbmdgCgpJbiBvdXIgcHJldmlvdXMgbGVzc29uLCB3ZSBkZW1vbnN0cmF0ZWQgaG93IHRvIGRlYWwgd2l0aCBpbWJhbGFuY2VkIGRhdGEgc2V0cyB1c2luZyBhIGByZWNpcGVgLiBBIHJlY2lwZSBjYW4gYmUgdGhvdWdodCBvZiBhcyBhIGJsdWVwcmludCB0aGF0IGRlc2NyaWJlcyB3aGF0IHN0ZXBzIHNob3VsZCBiZSBhcHBsaWVkIHRvIGEgZGF0YSBzZXQgaW4gb3JkZXIgdG8gZ2V0IGl0IHJlYWR5IGZvciBkYXRhIGFuYWx5c2lzLiBJbiBvdXIgY2FzZSwgd2Ugd2FudCB0byBoYXZlIGFuIGVxdWFsIGRpc3RyaWJ1dGlvbiBpbiB0aGUgbnVtYmVyIG9mIG91ciBjdWlzaW5lcyBmb3Igb3VyIGB0cmFpbmluZyBzZXRgLiBMZXQncyBnZXQgcmlnaHQgaW50byBpdC4KCmBgYHtyIHJlY2FwX2JhbGFuY2V9CiMgTG9hZCB0aGVtaXMgcGFja2FnZSBmb3IgZGVhbGluZyB3aXRoIGltYmFsYW5jZWQgZGF0YQpsaWJyYXJ5KHRoZW1pcykKCiMgQ3JlYXRlIGEgcmVjaXBlIGZvciBwcmVwcm9jZXNzaW5nIHRyYWluaW5nIGRhdGEKY3Vpc2luZXNfcmVjaXBlIDwtIHJlY2lwZShjdWlzaW5lIH4gLiwgZGF0YSA9IGN1aXNpbmVzX3RyYWluKSAlPiUKICBzdGVwX3Ntb3RlKGN1aXNpbmUpIAoKIyBQcmludCByZWNpcGUKY3Vpc2luZXNfcmVjaXBlCgpgYGAKCk5vdyB3ZSBhcmUgcmVhZHkgdG8gdHJhaW4gbW9kZWxzIPCfkanigI3wn5K78J+RqOKAjfCfkrshCgojIyAzLiBCZXlvbmQgbXVsdGlub21pYWwgcmVncmVzc2lvbiBtb2RlbHMKCkluIG91ciBwcmV2aW91cyBsZXNzb24sIHdlIGxvb2tlZCBhdCBtdWx0aW5vbWlhbCByZWdyZXNzaW9uIG1vZGVscy4gTGV0J3MgZXhwbG9yZSBzb21lIG1vcmUgZmxleGlibGUgbW9kZWxzIGZvciBjbGFzc2lmaWNhdGlvbi4KCiMjIyBTdXBwb3J0IFZlY3RvciBNYWNoaW5lcy4KCkluIHRoZSBjb250ZXh0IG9mIGNsYXNzaWZpY2F0aW9uLCBgU3VwcG9ydCBWZWN0b3IgTWFjaGluZXNgIGlzIGEgbWFjaGluZSBsZWFybmluZyB0ZWNobmlxdWUgdGhhdCB0cmllcyB0byBmaW5kIGEgKmh5cGVycGxhbmUqIHRoYXQgImJlc3QiIHNlcGFyYXRlcyB0aGUgY2xhc3Nlcy4gTGV0J3MgbG9vayBhdCBhIHNpbXBsZSBleGFtcGxlOgoKIVtCeSBVc2VyOlphY2tXZWluYmVyZzpUaGlzIGZpbGUgd2FzIGRlcml2ZWQgZnJvbTogPGh0dHBzOi8vY29tbW9ucy53aWtpbWVkaWEub3JnL3cvaW5kZXgucGhwP2N1cmlkPTIyODc3NTk4Pl0oLi4vLi4vaW1hZ2VzL3N2bS5wbmcpe3dpZHRoPSIzMDAifQoKSH4xfiBkb2VzIG5vdCBzZXBhcmF0ZSB0aGUgY2xhc3Nlcy4gSH4yfiBkb2VzLCBidXQgb25seSB3aXRoIGEgc21hbGwgbWFyZ2luLiBIfjN+IHNlcGFyYXRlcyB0aGVtIHdpdGggdGhlIG1heGltYWwgbWFyZ2luLgoKIyMjIyBMaW5lYXIgU3VwcG9ydCBWZWN0b3IgQ2xhc3NpZmllcgoKU3VwcG9ydC1WZWN0b3IgY2x1c3RlcmluZyAoU1ZDKSBpcyBhIGNoaWxkIG9mIHRoZSBTdXBwb3J0LVZlY3RvciBtYWNoaW5lcyBmYW1pbHkgb2YgTUwgdGVjaG5pcXVlcy4gSW4gU1ZDLCB0aGUgaHlwZXJwbGFuZSBpcyBjaG9zZW4gdG8gY29ycmVjdGx5IHNlcGFyYXRlIGBtb3N0YCBvZiB0aGUgdHJhaW5pbmcgb2JzZXJ2YXRpb25zLCBidXQgYG1heSBtaXNjbGFzc2lmeWAgYSBmZXcgb2JzZXJ2YXRpb25zLiBCeSBhbGxvd2luZyBzb21lIHBvaW50cyB0byBiZSBvbiB0aGUgd3Jvbmcgc2lkZSwgdGhlIFNWTSBiZWNvbWVzIG1vcmUgcm9idXN0IHRvIG91dGxpZXJzIGhlbmNlIGJldHRlciBnZW5lcmFsaXphdGlvbiB0byBuZXcgZGF0YS4gVGhlIHBhcmFtZXRlciB0aGF0IHJlZ3VsYXRlcyB0aGlzIHZpb2xhdGlvbiBpcyByZWZlcnJlZCB0byBhcyBgY29zdGAgd2hpY2ggaGFzIGEgZGVmYXVsdCB2YWx1ZSBvZiAxIChzZWUgYGhlbHAoInN2bV9wb2x5IilgKS4KCkxldCdzIGNyZWF0ZSBhIGxpbmVhciBTVkMgYnkgc2V0dGluZyBgZGVncmVlID0gMWAgaW4gYSBwb2x5bm9taWFsIFNWTSBtb2RlbC4KCmBgYHtyIHN2Y19zcGVjfQojIE1ha2UgYSBsaW5lYXIgU1ZDIHNwZWNpZmljYXRpb24Kc3ZjX2xpbmVhcl9zcGVjIDwtIHN2bV9wb2x5KGRlZ3JlZSA9IDEpICU+JSAKICBzZXRfZW5naW5lKCJrZXJubGFiIikgJT4lIAogIHNldF9tb2RlKCJjbGFzc2lmaWNhdGlvbiIpCgojIEJ1bmRsZSBzcGVjaWZpY2F0aW9uIGFuZCByZWNpcGUgaW50byBhIHdvcmtsb3cKc3ZjX2xpbmVhcl93ZiA8LSB3b3JrZmxvdygpICU+JSAKICBhZGRfcmVjaXBlKGN1aXNpbmVzX3JlY2lwZSkgJT4lIAogIGFkZF9tb2RlbChzdmNfbGluZWFyX3NwZWMpCgojIFByaW50IG91dCB3b3JrZmxvdwpzdmNfbGluZWFyX3dmCmBgYAoKTm93IHRoYXQgd2UgaGF2ZSBjYXB0dXJlZCB0aGUgcHJlcHJvY2Vzc2luZyBzdGVwcyBhbmQgbW9kZWwgc3BlY2lmaWNhdGlvbiBpbnRvIGEgKndvcmtmbG93Kiwgd2UgY2FuIGdvIGFoZWFkIGFuZCB0cmFpbiB0aGUgbGluZWFyIFNWQyBhbmQgZXZhbHVhdGUgcmVzdWx0cyB3aGlsZSBhdCBpdC4gRm9yIHBlcmZvcm1hbmNlIG1ldHJpY3MsIGxldCdzIGNyZWF0ZSBhIG1ldHJpYyBzZXQgdGhhdCB3aWxsIGV2YWx1YXRlOiBgYWNjdXJhY3lgLCBgc2Vuc2l0aXZpdHlgLCBgUG9zaXRpdmUgUHJlZGljdGVkIFZhbHVlYCBhbmQgYEYgTWVhc3VyZWAKCj4gYGF1Z21lbnQoKWAgd2lsbCBhZGQgY29sdW1uKHMpIGZvciBwcmVkaWN0aW9ucyB0byB0aGUgZ2l2ZW4gZGF0YS4KCmBgYHtyIHN2Y190cmFpbn0KIyBUcmFpbiBhIGxpbmVhciBTVkMgbW9kZWwKc3ZjX2xpbmVhcl9maXQgPC0gc3ZjX2xpbmVhcl93ZiAlPiUgCiAgZml0KGRhdGEgPSBjdWlzaW5lc190cmFpbikKCiMgQ3JlYXRlIGEgbWV0cmljIHNldApldmFsX21ldHJpY3MgPC0gbWV0cmljX3NldChwcHYsIHNlbnMsIGFjY3VyYWN5LCBmX21lYXMpCgoKIyBNYWtlIHByZWRpY3Rpb25zIGFuZCBFdmFsdWF0ZSBtb2RlbCBwZXJmb3JtYW5jZQpzdmNfbGluZWFyX2ZpdCAlPiUgCiAgYXVnbWVudChuZXdfZGF0YSA9IGN1aXNpbmVzX3Rlc3QpICU+JSAKICBldmFsX21ldHJpY3ModHJ1dGggPSBjdWlzaW5lLCBlc3RpbWF0ZSA9IC5wcmVkX2NsYXNzKQogIAoKCmBgYAoKIyMjIyAKCiMjIyMgU3VwcG9ydCBWZWN0b3IgTWFjaGluZQoKVGhlIHN1cHBvcnQgdmVjdG9yIG1hY2hpbmUgKFNWTSkgaXMgYW4gZXh0ZW5zaW9uIG9mIHRoZSBzdXBwb3J0IHZlY3RvciBjbGFzc2lmaWVyIGluIG9yZGVyIHRvIGFjY29tbW9kYXRlIGEgbm9uLWxpbmVhciBib3VuZGFyeSBiZXR3ZWVuIHRoZSBjbGFzc2VzLiBJbiBlc3NlbmNlLCBTVk1zIHVzZSB0aGUgKmtlcm5lbCB0cmljayogdG8gZW5sYXJnZSB0aGUgZmVhdHVyZSBzcGFjZSB0byBhZGFwdCB0byBub25saW5lYXIgcmVsYXRpb25zaGlwcyBiZXR3ZWVuIGNsYXNzZXMuIE9uZSBwb3B1bGFyIGFuZCBleHRyZW1lbHkgZmxleGlibGUga2VybmVsIGZ1bmN0aW9uIHVzZWQgYnkgU1ZNcyBpcyB0aGUgKlJhZGlhbCBiYXNpcyBmdW5jdGlvbi4qIExldCdzIHNlZSBob3cgaXQgd2lsbCBwZXJmb3JtIG9uIG91ciBkYXRhLgoKYGBge3Igc3ZtX3JiZn0Kc2V0LnNlZWQoMjA1NikKCiMgTWFrZSBhbiBSQkYgU1ZNIHNwZWNpZmljYXRpb24Kc3ZtX3JiZl9zcGVjIDwtIHN2bV9yYmYoKSAlPiUgCiAgc2V0X2VuZ2luZSgia2VybmxhYiIpICU+JSAKICBzZXRfbW9kZSgiY2xhc3NpZmljYXRpb24iKQoKIyBCdW5kbGUgc3BlY2lmaWNhdGlvbiBhbmQgcmVjaXBlIGludG8gYSB3b3JrbG93CnN2bV9yYmZfd2YgPC0gd29ya2Zsb3coKSAlPiUgCiAgYWRkX3JlY2lwZShjdWlzaW5lc19yZWNpcGUpICU+JSAKICBhZGRfbW9kZWwoc3ZtX3JiZl9zcGVjKQoKCiMgVHJhaW4gYW4gUkJGIG1vZGVsCnN2bV9yYmZfZml0IDwtIHN2bV9yYmZfd2YgJT4lIAogIGZpdChkYXRhID0gY3Vpc2luZXNfdHJhaW4pCgoKIyBNYWtlIHByZWRpY3Rpb25zIGFuZCBFdmFsdWF0ZSBtb2RlbCBwZXJmb3JtYW5jZQpzdm1fcmJmX2ZpdCAlPiUgCiAgYXVnbWVudChuZXdfZGF0YSA9IGN1aXNpbmVzX3Rlc3QpICU+JSAKICBldmFsX21ldHJpY3ModHJ1dGggPSBjdWlzaW5lLCBlc3RpbWF0ZSA9IC5wcmVkX2NsYXNzKQpgYGAKCk11Y2ggYmV0dGVyIPCfpKkhCgo+IOKchSBQbGVhc2Ugc2VlOgo+Cj4gLSAgIFsqU3VwcG9ydCBWZWN0b3IgTWFjaGluZXMqXShodHRwczovL2JyYWRsZXlib2VobWtlLmdpdGh1Yi5pby9IT01ML3N2bS5odG1sKSwgSGFuZHMtb24gTWFjaGluZSBMZWFybmluZyB3aXRoIFIKPgo+IC0gICBbKlN1cHBvcnQgVmVjdG9yIE1hY2hpbmVzKl0oaHR0cHM6Ly93d3cuc3RhdGxlYXJuaW5nLmNvbS8pLCBBbiBJbnRyb2R1Y3Rpb24gdG8gU3RhdGlzdGljYWwgTGVhcm5pbmcgd2l0aCBBcHBsaWNhdGlvbnMgaW4gUgo+Cj4gZm9yIGZ1cnRoZXIgcmVhZGluZy4KCiMjIyBOZWFyZXN0IE5laWdoYm9yIGNsYXNzaWZpZXJzCgoqSyotbmVhcmVzdCBuZWlnaGJvciAoS05OKSBpcyBhbiBhbGdvcml0aG0gaW4gd2hpY2ggZWFjaCBvYnNlcnZhdGlvbiBpcyBwcmVkaWN0ZWQgYmFzZWQgb24gaXRzICpzaW1pbGFyaXR5KiB0byBvdGhlciBvYnNlcnZhdGlvbnMuCgpMZXQncyBmaXQgb25lIHRvIG91ciBkYXRhLgoKYGBge3Iga25ufQojIE1ha2UgYSBLTk4gc3BlY2lmaWNhdGlvbgprbm5fc3BlYyA8LSBuZWFyZXN0X25laWdoYm9yKCkgJT4lIAogIHNldF9lbmdpbmUoImtrbm4iKSAlPiUgCiAgc2V0X21vZGUoImNsYXNzaWZpY2F0aW9uIikKCiMgQnVuZGxlIHJlY2lwZSBhbmQgbW9kZWwgc3BlY2lmaWNhdGlvbiBpbnRvIGEgd29ya2Zsb3cKa25uX3dmIDwtIHdvcmtmbG93KCkgJT4lIAogIGFkZF9yZWNpcGUoY3Vpc2luZXNfcmVjaXBlKSAlPiUgCiAgYWRkX21vZGVsKGtubl9zcGVjKQoKIyBUcmFpbiBhIGJvb3N0ZWQgdHJlZSBtb2RlbAprbm5fd2ZfZml0IDwtIGtubl93ZiAlPiUgCiAgZml0KGRhdGEgPSBjdWlzaW5lc190cmFpbikKCgojIE1ha2UgcHJlZGljdGlvbnMgYW5kIEV2YWx1YXRlIG1vZGVsIHBlcmZvcm1hbmNlCmtubl93Zl9maXQgJT4lIAogIGF1Z21lbnQobmV3X2RhdGEgPSBjdWlzaW5lc190ZXN0KSAlPiUgCiAgZXZhbF9tZXRyaWNzKHRydXRoID0gY3Vpc2luZSwgZXN0aW1hdGUgPSAucHJlZF9jbGFzcykKYGBgCgpJdCBhcHBlYXJzIHRoYXQgdGhpcyBtb2RlbCBpcyBub3QgcGVyZm9ybWluZyB0aGF0IHdlbGwuIFByb2JhYmx5IGNoYW5naW5nIHRoZSBtb2RlbCdzIGFyZ3VtZW50cyAoc2VlIGBoZWxwKCJuZWFyZXN0X25laWdoYm9yIilgIHdpbGwgaW1wcm92ZSBtb2RlbCBwZXJmb3JtYW5jZS4gQmUgc3VyZSB0byB0cnkgaXQgb3V0LgoKPiDinIUgUGxlYXNlIHNlZToKPgo+IC0gICBbSGFuZHMtb24gTWFjaGluZSBMZWFybmluZyB3aXRoIFJdKGh0dHBzOi8vYnJhZGxleWJvZWhta2UuZ2l0aHViLmlvL0hPTUwvKQo+Cj4gLSAgIFtBbiBJbnRyb2R1Y3Rpb24gdG8gU3RhdGlzdGljYWwgTGVhcm5pbmcgd2l0aCBBcHBsaWNhdGlvbnMgaW4gUl0oaHR0cHM6Ly93d3cuc3RhdGxlYXJuaW5nLmNvbS8pCj4KPiB0byBsZWFybiBtb3JlIGFib3V0ICpLKi1OZWFyZXN0IE5laWdoYm9ycyBjbGFzc2lmaWVycy4KCiMjIyBFbnNlbWJsZSBjbGFzc2lmaWVycwoKRW5zZW1ibGUgYWxnb3JpdGhtcyB3b3JrIGJ5IGNvbWJpbmluZyBtdWx0aXBsZSBiYXNlIGVzdGltYXRvcnMgdG8gcHJvZHVjZSBhbiBvcHRpbWFsIG1vZGVsIGVpdGhlciBieToKCmBiYWdnaW5nYDogYXBwbHlpbmcgYW4gKmF2ZXJhZ2luZyBmdW5jdGlvbiogdG8gYSBjb2xsZWN0aW9uIG9mIGJhc2UgbW9kZWxzCgpgYm9vc3RpbmdgOiBidWlsZGluZyBhIHNlcXVlbmNlIG9mIG1vZGVscyB0aGF0IGJ1aWxkIG9uIG9uZSBhbm90aGVyIHRvIGltcHJvdmUgcHJlZGljdGl2ZSBwZXJmb3JtYW5jZS4KCkxldCdzIHN0YXJ0IGJ5IHRyeWluZyBvdXQgYSBSYW5kb20gRm9yZXN0IG1vZGVsLCB3aGljaCBidWlsZHMgYSBsYXJnZSBjb2xsZWN0aW9uIG9mIGRlY2lzaW9uIHRyZWVzIHRoZW4gYXBwbGllcyBhbiBhdmVyYWdpbmcgZnVuY3Rpb24gdG8gZm9yIGEgYmV0dGVyIG92ZXJhbGwgbW9kZWwuCgpgYGB7ciByZn0KIyBNYWtlIGEgcmFuZG9tIGZvcmVzdCBzcGVjaWZpY2F0aW9uCnJmX3NwZWMgPC0gcmFuZF9mb3Jlc3QoKSAlPiUgCiAgc2V0X2VuZ2luZSgicmFuZ2VyIikgJT4lIAogIHNldF9tb2RlKCJjbGFzc2lmaWNhdGlvbiIpCgojIEJ1bmRsZSByZWNpcGUgYW5kIG1vZGVsIHNwZWNpZmljYXRpb24gaW50byBhIHdvcmtmbG93CnJmX3dmIDwtIHdvcmtmbG93KCkgJT4lIAogIGFkZF9yZWNpcGUoY3Vpc2luZXNfcmVjaXBlKSAlPiUgCiAgYWRkX21vZGVsKHJmX3NwZWMpCgojIFRyYWluIGEgcmFuZG9tIGZvcmVzdCBtb2RlbApyZl93Zl9maXQgPC0gcmZfd2YgJT4lIAogIGZpdChkYXRhID0gY3Vpc2luZXNfdHJhaW4pCgoKIyBNYWtlIHByZWRpY3Rpb25zIGFuZCBFdmFsdWF0ZSBtb2RlbCBwZXJmb3JtYW5jZQpyZl93Zl9maXQgJT4lIAogIGF1Z21lbnQobmV3X2RhdGEgPSBjdWlzaW5lc190ZXN0KSAlPiUgCiAgZXZhbF9tZXRyaWNzKHRydXRoID0gY3Vpc2luZSwgZXN0aW1hdGUgPSAucHJlZF9jbGFzcykKICAKCmBgYAoKR29vZCBqb2Ig8J+RjyEKCkxldCdzIGFsc28gZXhwZXJpbWVudCB3aXRoIGEgQm9vc3RlZCBUcmVlIG1vZGVsLgoKQm9vc3RlZCBUcmVlIGRlZmluZXMgYW4gZW5zZW1ibGUgbWV0aG9kIHRoYXQgY3JlYXRlcyBhIHNlcmllcyBvZiBzZXF1ZW50aWFsIGRlY2lzaW9uIHRyZWVzIHdoZXJlIGVhY2ggdHJlZSBkZXBlbmRzIG9uIHRoZSByZXN1bHRzIG9mIHByZXZpb3VzIHRyZWVzIGluIGFuIGF0dGVtcHQgdG8gaW5jcmVtZW50YWxseSByZWR1Y2UgdGhlIGVycm9yLiBJdCBmb2N1c2VzIG9uIHRoZSB3ZWlnaHRzIG9mIGluY29ycmVjdGx5IGNsYXNzaWZpZWQgaXRlbXMgYW5kIGFkanVzdHMgdGhlIGZpdCBmb3IgdGhlIG5leHQgY2xhc3NpZmllciB0byBjb3JyZWN0LgoKVGhlcmUgYXJlIGRpZmZlcmVudCB3YXlzIHRvIGZpdCB0aGlzIG1vZGVsIChzZWUgYGhlbHAoImJvb3N0X3RyZWUiKWApLiBJbiB0aGlzIGV4YW1wbGUsIHdlJ2xsIGZpdCBCb29zdGVkIHRyZWVzIHZpYSBgeGdib29zdGAgZW5naW5lLgoKYGBge3IgYm9vc3RlZF90cmVlfQojIE1ha2UgYSBib29zdGVkIHRyZWUgc3BlY2lmaWNhdGlvbgpib29zdF9zcGVjIDwtIGJvb3N0X3RyZWUodHJlZXMgPSAyMDApICU+JSAKICBzZXRfZW5naW5lKCJ4Z2Jvb3N0IikgJT4lIAogIHNldF9tb2RlKCJjbGFzc2lmaWNhdGlvbiIpCgojIEJ1bmRsZSByZWNpcGUgYW5kIG1vZGVsIHNwZWNpZmljYXRpb24gaW50byBhIHdvcmtmbG93CmJvb3N0X3dmIDwtIHdvcmtmbG93KCkgJT4lIAogIGFkZF9yZWNpcGUoY3Vpc2luZXNfcmVjaXBlKSAlPiUgCiAgYWRkX21vZGVsKGJvb3N0X3NwZWMpCgojIFRyYWluIGEgYm9vc3RlZCB0cmVlIG1vZGVsCmJvb3N0X3dmX2ZpdCA8LSBib29zdF93ZiAlPiUgCiAgZml0KGRhdGEgPSBjdWlzaW5lc190cmFpbikKCgojIE1ha2UgcHJlZGljdGlvbnMgYW5kIEV2YWx1YXRlIG1vZGVsIHBlcmZvcm1hbmNlCmJvb3N0X3dmX2ZpdCAlPiUgCiAgYXVnbWVudChuZXdfZGF0YSA9IGN1aXNpbmVzX3Rlc3QpICU+JSAKICBldmFsX21ldHJpY3ModHJ1dGggPSBjdWlzaW5lLCBlc3RpbWF0ZSA9IC5wcmVkX2NsYXNzKQpgYGAKCj4g4pyFIFBsZWFzZSBzZWU6Cj4KPiAtICAgW01hY2hpbmUgTGVhcm5pbmcgZm9yIFNvY2lhbCBTY2llbnRpc3RzXShodHRwczovL2NpbWVudGFkYWouZ2l0aHViLmlvL21sX3NvY3NjaS90cmVlLWJhc2VkLW1ldGhvZHMuaHRtbCNyYW5kb20tZm9yZXN0cykKPgo+IC0gICBbSGFuZHMtb24gTWFjaGluZSBMZWFybmluZyB3aXRoIFJdKGh0dHBzOi8vYnJhZGxleWJvZWhta2UuZ2l0aHViLmlvL0hPTUwvKQo+Cj4gLSAgIFtBbiBJbnRyb2R1Y3Rpb24gdG8gU3RhdGlzdGljYWwgTGVhcm5pbmcgd2l0aCBBcHBsaWNhdGlvbnMgaW4gUl0oaHR0cHM6Ly93d3cuc3RhdGxlYXJuaW5nLmNvbS8pCj4KPiAtICAgPGh0dHBzOi8vYWxnb3RlY2gubmV0bGlmeS5hcHAvYmxvZy94Z2Jvb3N0Lz4gLSBFeHBsb3JlcyB0aGUgQWRhQm9vc3QgbW9kZWwgd2hpY2ggaXMgYSBnb29kIGFsdGVybmF0aXZlIHRvIHhnYm9vc3QuCj4KPiB0byBsZWFybiBtb3JlIGFib3V0IEVuc2VtYmxlIGNsYXNzaWZpZXJzLgoKIyMgNC4gRXh0cmEgLSBjb21wYXJpbmcgbXVsdGlwbGUgbW9kZWxzCgpXZSBoYXZlIGZpdHRlZCBxdWl0ZSBhIG51bWJlciBvZiBtb2RlbHMgaW4gdGhpcyBsYWIg8J+ZjC4gSXQgY2FuIGJlY29tZSB0ZWRpb3VzIG9yIG9uZXJvdXMgdG8gY3JlYXRlIGEgbG90IG9mIHdvcmtmbG93cyBmcm9tIGRpZmZlcmVudCBzZXRzIG9mIHByZXByb2Nlc3NvcnMgYW5kL29yIG1vZGVsIHNwZWNpZmljYXRpb25zIGFuZCB0aGVuIGNhbGN1bGF0ZSB0aGUgcGVyZm9ybWFuY2UgbWV0cmljcyBvbmUgYnkgb25lLgoKTGV0J3Mgc2VlIGlmIHdlIGNhbiBhZGRyZXNzIHRoaXMgYnkgY3JlYXRpbmcgYSBmdW5jdGlvbiB0aGF0IGZpdHMgYSBsaXN0IG9mIHdvcmtmbG93cyBvbiB0aGUgdHJhaW5pbmcgc2V0IHRoZW4gcmV0dXJucyB0aGUgcGVyZm9ybWFuY2UgbWV0cmljcyBiYXNlZCBvbiB0aGUgdGVzdCBzZXQuIFdlJ2xsIGdldCB0byB1c2UgYG1hcCgpYCBhbmQgYG1hcF9kZnIoKWAgZnJvbSB0aGUgW3B1cnJyXShodHRwczovL3B1cnJyLnRpZHl2ZXJzZS5vcmcvKSBwYWNrYWdlIHRvIGFwcGx5IGZ1bmN0aW9ucyB0byBlYWNoIGVsZW1lbnQgaW4gbGlzdC4KCj4gW2BtYXAoKWBdKGh0dHBzOi8vcHVycnIudGlkeXZlcnNlLm9yZy9yZWZlcmVuY2UvbWFwLmh0bWwpIGZ1bmN0aW9ucyBhbGxvdyB5b3UgdG8gcmVwbGFjZSBtYW55IGZvciBsb29wcyB3aXRoIGNvZGUgdGhhdCBpcyBib3RoIG1vcmUgc3VjY2luY3QgYW5kIGVhc2llciB0byByZWFkLiBUaGUgYmVzdCBwbGFjZSB0byBsZWFybiBhYm91dCB0aGUgW2BtYXAoKWBdKGh0dHBzOi8vcHVycnIudGlkeXZlcnNlLm9yZy9yZWZlcmVuY2UvbWFwLmh0bWwpIGZ1bmN0aW9ucyBpcyB0aGUgW2l0ZXJhdGlvbiBjaGFwdGVyXShodHRwOi8vcjRkcy5oYWQuY28ubnovaXRlcmF0aW9uLmh0bWwpIGluIFIgZm9yIGRhdGEgc2NpZW5jZS4KCmBgYHtyIGNvbXBhcmVfbW9kZWxzfQpzZXQuc2VlZCgyMDU2KQoKIyBDcmVhdGUgYSBtZXRyaWMgc2V0CmV2YWxfbWV0cmljcyA8LSBtZXRyaWNfc2V0KHBwdiwgc2VucywgYWNjdXJhY3ksIGZfbWVhcykKCiMgRGVmaW5lIGEgZnVuY3Rpb24gdGhhdCByZXR1cm5zIHBlcmZvcm1hbmNlIG1ldHJpY3MKY29tcGFyZV9tb2RlbHMgPC0gZnVuY3Rpb24od29ya2Zsb3dfbGlzdCwgdHJhaW5fc2V0LCB0ZXN0X3NldCl7CiAgCiAgc3VwcHJlc3NXYXJuaW5ncygKICAgICMgRml0IGVhY2ggbW9kZWwgdG8gdGhlIHRyYWluX3NldAogICAgbWFwKHdvcmtmbG93X2xpc3QsIGZpdCwgZGF0YSA9IHRyYWluX3NldCkgJT4lIAogICAgIyBNYWtlIHByZWRpY3Rpb25zIG9uIHRoZSB0ZXN0IHNldAogICAgICBtYXBfZGZyKGF1Z21lbnQsIG5ld19kYXRhID0gdGVzdF9zZXQsIC5pZCA9ICJtb2RlbCIpICU+JQogICAgIyBTZWxlY3QgZGVzaXJlZCBjb2x1bW5zCiAgICAgIHNlbGVjdChtb2RlbCwgY3Vpc2luZSwgLnByZWRfY2xhc3MpICU+JSAKICAgICMgRXZhbHVhdGUgbW9kZWwgcGVyZm9ybWFuY2UKICAgICAgZ3JvdXBfYnkobW9kZWwpICU+JSAKICAgICAgZXZhbF9tZXRyaWNzKHRydXRoID0gY3Vpc2luZSwgZXN0aW1hdGUgPSAucHJlZF9jbGFzcykgJT4lIAogICAgICB1bmdyb3VwKCkKICApCiAgCn0gIyBFbmQgb2YgZnVuY3Rpb24KCgpgYGAKCkxldCdzIGNhbGwgb3VyIGZ1bmN0aW9uIGFuZCBjb21wYXJlIHRoZSBhY2N1cmFjeSBhY3Jvc3MgdGhlIG1vZGVscy4KCmBgYHtyIGNhbGxfZm59CiMgTWFrZSBhIGxpc3Qgb2Ygd29ya2Zsb3dzCndvcmtmbG93X2xpc3QgPC0gbGlzdCgKICAic3ZjIiA9IHN2Y19saW5lYXJfd2YsCiAgInN2bSIgPSBzdm1fcmJmX3dmLAogICJrbm4iID0ga25uX3dmLAogICJyYW5kb21fZm9yZXN0IiA9IHJmX3dmLAogICJ4Z2Jvb3N0IiA9IGJvb3N0X3dmKQoKIyBDYWxsIHRoZSBmdW5jdGlvbgpzZXQuc2VlZCgyMDU2KQpwZXJmX21ldHJpY3MgPC0gY29tcGFyZV9tb2RlbHMod29ya2Zsb3dfbGlzdCA9IHdvcmtmbG93X2xpc3QsIHRyYWluX3NldCA9IGN1aXNpbmVzX3RyYWluLCB0ZXN0X3NldCA9IGN1aXNpbmVzX3Rlc3QpCgojIFByaW50IG91dCBwZXJmb3JtYW5jZSBtZXRyaWNzCnBlcmZfbWV0cmljcyAlPiUgCiAgZ3JvdXBfYnkoLm1ldHJpYykgJT4lIAogIGFycmFuZ2UoZGVzYyguZXN0aW1hdGUpKSAlPiUgCiAgc2xpY2VfaGVhZChuPTcpCgojIENvbXBhcmUgYWNjdXJhY3kKcGVyZl9tZXRyaWNzICU+JSAKICBmaWx0ZXIoLm1ldHJpYyA9PSAiYWNjdXJhY3kiKSAlPiUgCiAgYXJyYW5nZShkZXNjKC5lc3RpbWF0ZSkpCgpgYGAKClsqKndvcmtmbG93c2V0KipdKGh0dHBzOi8vd29ya2Zsb3dzZXRzLnRpZHltb2RlbHMub3JnLykgcGFja2FnZSBhbGxvdyB1c2VycyB0byBjcmVhdGUgYW5kIGVhc2lseSBmaXQgYSBsYXJnZSBudW1iZXIgb2YgbW9kZWxzIGJ1dCBpcyBtb3N0bHkgZGVzaWduZWQgdG8gd29yayB3aXRoIHJlc2FtcGxpbmcgdGVjaG5pcXVlcyBzdWNoIGFzIGBjcm9zcy12YWxpZGF0aW9uYCwgYW4gYXBwcm9hY2ggd2UgYXJlIHlldCB0byBjb3Zlci4KCiMjICoq8J+agENoYWxsZW5nZSoqCgpFYWNoIG9mIHRoZXNlIHRlY2huaXF1ZXMgaGFzIGEgbGFyZ2UgbnVtYmVyIG9mIHBhcmFtZXRlcnMgdGhhdCB5b3UgY2FuIHR3ZWFrIGZvciBpbnN0YW5jZSBgY29zdGAgaW4gU1ZNcywgYG5laWdoYm9yc2AgaW4gS05OLCBgbXRyeWAgKFJhbmRvbWx5IFNlbGVjdGVkIFByZWRpY3RvcnMpIGluIFJhbmRvbSBGb3Jlc3QuCgpSZXNlYXJjaCBlYWNoIG9uZSdzIGRlZmF1bHQgcGFyYW1ldGVycyBhbmQgdGhpbmsgYWJvdXQgd2hhdCB0d2Vha2luZyB0aGVzZSBwYXJhbWV0ZXJzIHdvdWxkIG1lYW4gZm9yIHRoZSBtb2RlbCdzIHF1YWxpdHkuCgpUbyBmaW5kIG91dCBtb3JlIGFib3V0IGEgcGFydGljdWxhciBtb2RlbCBhbmQgaXRzIHBhcmFtZXRlcnMsIHVzZTogYGhlbHAoIm1vZGVsIilgIGUuZyBgaGVscCgicmFuZF9mb3Jlc3QiKWAKCj4gSW4gcHJhY3RpY2UsIHdlIHVzdWFsbHkgKmVzdGltYXRlKiB0aGUgKmJlc3QgdmFsdWVzKiBmb3IgdGhlc2UgYnkgdHJhaW5pbmcgbWFueSBtb2RlbHMgb24gYSBgc2ltdWxhdGVkIGRhdGEgc2V0YCBhbmQgbWVhc3VyaW5nIGhvdyB3ZWxsIGFsbCB0aGVzZSBtb2RlbHMgcGVyZm9ybS4gVGhpcyBwcm9jZXNzIGlzIGNhbGxlZCAqKnR1bmluZyoqLgoKIyMjIFsqKlBvc3QtbGVjdHVyZSBxdWl6KipdKGh0dHBzOi8vZ3JheS1zYW5kLTA3YTEwZjQwMy4xLmF6dXJlc3RhdGljYXBwcy5uZXQvcXVpei8yNC8pCgojIyMgKipSZXZpZXcgJiBTZWxmIFN0dWR5KioKClRoZXJlJ3MgYSBsb3Qgb2YgamFyZ29uIGluIHRoZXNlIGxlc3NvbnMsIHNvIHRha2UgYSBtaW51dGUgdG8gcmV2aWV3IFt0aGlzIGxpc3RdKGh0dHBzOi8vZG9jcy5taWNyb3NvZnQuY29tL2RvdG5ldC9tYWNoaW5lLWxlYXJuaW5nL3Jlc291cmNlcy9nbG9zc2FyeT9XVC5tY19pZD1hY2FkZW1pYy03Nzk1Mi1sZWVzdG90dCkgb2YgdXNlZnVsIHRlcm1pbm9sb2d5IQoKIyMjIyBUSEFOSyBZT1UgVE86CgpbYEFsbGlzb24gSG9yc3RgXShodHRwczovL3R3aXR0ZXIuY29tL2FsbGlzb25faG9yc3QvKSBmb3IgY3JlYXRpbmcgdGhlIGFtYXppbmcgaWxsdXN0cmF0aW9ucyB0aGF0IG1ha2UgUiBtb3JlIHdlbGNvbWluZyBhbmQgZW5nYWdpbmcuIEZpbmQgbW9yZSBpbGx1c3RyYXRpb25zIGF0IGhlciBbZ2FsbGVyeV0oaHR0cHM6Ly93d3cuZ29vZ2xlLmNvbS91cmw/cT1odHRwczovL2dpdGh1Yi5jb20vYWxsaXNvbmhvcnN0L3N0YXRzLWlsbHVzdHJhdGlvbnMmc2E9RCZzb3VyY2U9ZWRpdG9ycyZ1c3Q9MTYyNjM4MDc3MjUzMDAwMCZ1c2c9QU92VmF3M3pjZnlDaXpGUVpwa1NMenhpaVFFTSkuCgpbQ2Fzc2llIEJyZXZpdV0oaHR0cHM6Ly93d3cudHdpdHRlci5jb20vY2Fzc2lldmlldykgYW5kIFtKZW4gTG9vcGVyXShodHRwczovL3d3dy50d2l0dGVyLmNvbS9qZW5sb29wZXIpIGZvciBjcmVhdGluZyB0aGUgb3JpZ2luYWwgUHl0aG9uIHZlcnNpb24gb2YgdGhpcyBtb2R1bGUg4pml77iPCgpIYXBweSBMZWFybmluZywKCltFcmljXShodHRwczovL3R3aXR0ZXIuY29tL2VyaWNudGF5KSwgR29sZCBNaWNyb3NvZnQgTGVhcm4gU3R1ZGVudCBBbWJhc3NhZG9yLgoKIVtBcnR3b3JrIGJ5IFxAYWxsaXNvbl9ob3JzdF0oLi4vLi4vaW1hZ2VzL3JfbGVhcm5lcnNfc20uanBlZykK
+
+
+