From a74f75d9126c8b26e4c2bfdbbeedb27b55cea79e Mon Sep 17 00:00:00 2001 From: Jasleen Sondhi Date: Mon, 19 Jun 2023 17:14:12 +0530 Subject: [PATCH] added lesson_4.html --- .../4-Logistic/solution/R/lesson_4.Rmd | 6 +- .../4-Logistic/solution/R/lesson_4.html | 3649 +++++++++++++++++ README.md | 2 +- 3 files changed, 3653 insertions(+), 4 deletions(-) create mode 100644 2-Regression/4-Logistic/solution/R/lesson_4.html diff --git a/2-Regression/4-Logistic/solution/R/lesson_4.Rmd b/2-Regression/4-Logistic/solution/R/lesson_4.Rmd index ed6a0dc0..6bf95bf0 100644 --- a/2-Regression/4-Logistic/solution/R/lesson_4.Rmd +++ b/2-Regression/4-Logistic/solution/R/lesson_4.Rmd @@ -14,7 +14,7 @@ output: ![Infographic by Dasani Madipalli](../../images/logistic-linear.png){width="600"} -#### ** [Pre-lecture quiz](https://gray-sand-07a10f403.1.azurestaticapps.net/quiz/15/)** +#### **[Pre-lecture quiz](https://gray-sand-07a10f403.1.azurestaticapps.net/quiz/15/)** #### Introduction @@ -56,7 +56,7 @@ suppressWarnings(if (!require("pacman"))install.packages("pacman")) pacman::p_load(tidyverse, tidymodels, janitor, ggbeeswarm) ``` -## ** Define the question** +## **Define the question** For our purposes, we will express this as a binary: 'Orange' or 'Not Orange'. There is also a 'striped' category in our dataset but there are few instances of it, so we will not use it. It disappears once we remove null values from the dataset, anyway. @@ -148,7 +148,7 @@ pumpkins_select %>% The goal of data exploration is to try to understand the `relationships` between its attributes; in particular, any apparent correlation between the *features* and the *label* your model will try to predict. One way of doing this is by using data visualization. -Given our the data types of our columns, we can `encode` them and be on our way to making some visualizations. This simply involves `translating` a column with `categorical values` for example our columns of type *char*, into one or more `numeric columns` that take the place of the original. - Something we did in our [last lesson](https://github.com/microsoft/ML-For-Beginners/blob/main/2-Regression/3-Linear/solution/lesson_3-R.ipynb). +Given our the data types of our columns, we can `encode` them and be on our way to making some visualizations. This simply involves `translating` a column with `categorical values` for example our columns of type *char*, into one or more `numeric columns` that take the place of the original. - Something we did in our [last lesson](https://github.com/microsoft/ML-For-Beginners/blob/main/2-Regression/3-Linear/solution/lesson_3.html). Tidymodels provides yet another neat package: [recipes](https://recipes.tidymodels.org/)- a package for preprocessing data. We'll define a `recipe` that specifies that all predictor columns should be encoded into a set of integers , `prep` it to estimates the required quantities and statistics needed by any operations and finally `bake` to apply the computations to new data. diff --git a/2-Regression/4-Logistic/solution/R/lesson_4.html b/2-Regression/4-Logistic/solution/R/lesson_4.html new file mode 100644 index 00000000..f81e8cfa --- /dev/null +++ b/2-Regression/4-Logistic/solution/R/lesson_4.html @@ -0,0 +1,3649 @@ + + + + + + + + + + + + + +Build a regression model: logistic regression + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + +
+
+
+
+
+ +
+ + + + + + + +
+

Build a logistic regression model - Lesson 4

+
+ +

Infographic by Dasani Madipalli

+
+ +
+

Introduction

+

In this final lesson on Regression, one of the basic classic +ML techniques, we will take a look at Logistic Regression. You would use +this technique to discover patterns to predict binary +categories. Is this candy chocolate or not? Is this disease +contagious or not? Will this customer choose this product or not?

+

In this lesson, you will learn:

+
    +
  • Techniques for logistic regression
    • +
+

✅ Deepen your understanding of working with this type of regression +in this Learn +module

+
+
+

Prerequisite

+

Having worked with the pumpkin data, we are now familiar enough with +it to realize that there’s one binary category that we can work with: +Color.

+

Let’s build a logistic regression model to predict that, given some +variables, what color a given pumpkin is likely to be (orange +🎃 or white 👻).

+
+

Why are we talking about binary classification in a lesson grouping +about regression? Only for linguistic convenience, as logistic +regression is really +a classification method, albeit a linear-based one. Learn about +other ways to classify data in the next lesson group.

+
+

For this lesson, we’ll require the following packages:

+ +

You can have them installed as:

+

install.packages(c("tidyverse", "tidymodels", "janitor", "ggbeeswarm"))

+

Alternately, the script below checks whether you have the packages +required to complete this module and installs them for you in case they +are missing.

+
suppressWarnings(if (!require("pacman"))install.packages("pacman"))
+
+pacman::p_load(tidyverse, tidymodels, janitor, ggbeeswarm)
+
+
+
+

Define the question

+

For our purposes, we will express this as a binary: ‘Orange’ or ‘Not +Orange’. There is also a ‘striped’ category in our dataset but there are +few instances of it, so we will not use it. It disappears once we remove +null values from the dataset, anyway.

+
+

🎃 Fun fact, we sometimes call white pumpkins ‘ghost’ pumpkins. They +aren’t very easy to carve, so they aren’t as popular as the orange ones +but they are cool looking!

+
+
+
+

About logistic regression

+

Logistic regression differs from linear regression, which you learned +about previously, in a few important ways.

+
+

Binary classification

+

Logistic regression does not offer the same features as linear +regression. The former offers a prediction about a +binary category (“orange or not orange”) whereas the latter +is capable of predicting continual values, for example +given the origin of a pumpkin and the time of harvest, how much its +price will rise.

+
+ +

Infographic by Dasani Madipalli

+
+
+
+

Other classifications

+

There are other types of logistic regression, including multinomial +and ordinal:

+
    +

  • Multinomial, which involves having more than one +category - “Orange, White, and Striped”.

    • +

  • Ordinal, which involves ordered categories, +useful if we wanted to order our outcomes logically, like our pumpkins +that are ordered by a finite number of sizes +(mini,sm,med,lg,xl,xxl).

    • +
+
+ +

Infographic by Dasani Madipalli

+
+


+It’s still linear

+

Even though this type of Regression is all about ‘category +predictions’, it still works best when there is a clear linear +relationship between the dependent variable (color) and the other +independent variables (the rest of the dataset, like city name and +size). It’s good to get an idea of whether there is any linearity +dividing these variables or not.

+
+
+

Variables DO NOT have to correlate

+

Remember how linear regression worked better with more correlated +variables? Logistic regression is the opposite - the variables don’t +have to align. That works for this data which has somewhat weak +correlations.

+
+
+

You need a lot of clean data

+

Logistic regression will give more accurate results if you use more +data; our small dataset is not optimal for this task, so keep that in +mind.

+

✅ Think about the types of data that would lend themselves well to +logistic regression

+
+
+
+

1. Tidy the data

+

Now, the fun begins! Let’s start by importing the data, cleaning the +data a bit, dropping rows containing missing values and selecting only +some of the columns:

+
# Load the core tidyverse packages
+library(tidyverse)
+
+# Import the data and clean column names
+pumpkins <- read_csv(file = "https://raw.githubusercontent.com/microsoft/ML-For-Beginners/main/2-Regression/data/US-pumpkins.csv") %>% 
+  clean_names()
+
+# Select desired columns
+pumpkins_select <- pumpkins %>% 
+  select(c(city_name, package, variety, origin, item_size, color)) 
+
+# Drop rows containing missing values and encode color as factor (category)
+pumpkins_select <- pumpkins_select %>% 
+  drop_na() %>% 
+  mutate(color = factor(color))
+
+# View the first few rows
+pumpkins_select %>% 
+  slice_head(n = 5)
+
+ +
+

Sometimes, we may want some little more information on our data. We +can have a look at the data, its structure and +the data type of its features by using the glimpse() +function as below:

+
pumpkins_select %>% 
+  glimpse()
+
## Rows: 991
+## Columns: 6
+## $ city_name <chr> "BALTIMORE", "BALTIMORE", "BALTIMORE", "BALTIMORE", "BALTIMO~
+## $ package   <chr> "24 inch bins", "24 inch bins", "24 inch bins", "24 inch bin~
+## $ variety   <chr> "HOWDEN TYPE", "HOWDEN TYPE", "HOWDEN TYPE", "HOWDEN TYPE", ~
+## $ origin    <chr> "DELAWARE", "VIRGINIA", "MARYLAND", "MARYLAND", "MARYLAND", ~
+## $ item_size <chr> "med", "med", "lge", "lge", "med", "lge", "med", "lge", "med~
+## $ color     <fct> ORANGE, ORANGE, ORANGE, ORANGE, ORANGE, ORANGE, ORANGE, ORAN~
+

Wow! Seems that all our columns are all of type character, +further alluding that they are all categorical.

+

Let’s confirm that we will actually be doing a binary classification +problem:

+
# Subset distinct observations in outcome column
+pumpkins_select %>% 
+  distinct(color)
+
+ +
+

🥳🥳 That went down well!

+
+
+

2. Explore the data

+

The goal of data exploration is to try to understand the +relationships between its attributes; in particular, any +apparent correlation between the features and the +label your model will try to predict. One way of doing this is +by using data visualization.

+

Given our the data types of our columns, we can encode +them and be on our way to making some visualizations. This simply +involves translating a column with +categorical values for example our columns of type +char, into one or more numeric columns that take +the place of the original. - Something we did in our last +lesson.

+

Tidymodels provides yet another neat package: recipes- a package for +preprocessing data. We’ll define a recipe that specifies +that all predictor columns should be encoded into a set of integers , +prep it to estimates the required quantities and statistics +needed by any operations and finally bake to apply the +computations to new data.

+
+

Normally, recipes is usually used as a preprocessor for modelling +where it defines what steps should be applied to a data set in order to +get it ready for modelling. In that case it is highly +recommend that you use a workflow() instead of +manually estimating a recipe using prep and bake. We’ll see all this in +just a moment.

+

However for now, we are using recipes + prep + bake to specify what +steps should be applied to a data set in order to get it ready for data +analysis and then extract the preprocessed data with the steps +applied.

+
+
# Preprocess and extract data to allow some data analysis
+baked_pumpkins <- recipe(color ~ ., data = pumpkins_select) %>% 
+  # Encode all columns to a set of integers
+  step_integer(all_predictors(), zero_based = T) %>% 
+  prep() %>% 
+  bake(new_data = NULL)
+
+
+# Display the first few rows of preprocessed data
+baked_pumpkins %>% 
+  slice_head(n = 5)
+
+ +
+

Now let’s compare the feature distributions for each label value +using box plots. We’ll begin by formatting the data to a long +format to make it somewhat easier to make multiple +facets.

+
# Pivot data to long format
+baked_pumpkins_long <- baked_pumpkins %>% 
+  pivot_longer(!color, names_to = "features", values_to = "values")
+
+
+# Print out restructured data
+baked_pumpkins_long %>% 
+  slice_head(n = 10)
+
+ +
+

Now, let’s make some boxplots showing the distribution of the +predictors with respect to the outcome color!

+
theme_set(theme_light())
+#Make a box plot for each predictor feature
+baked_pumpkins_long %>% 
+  mutate(color = factor(color)) %>% 
+  ggplot(mapping = aes(x = color, y = values, fill = features)) +
+  geom_boxplot() + 
+  facet_wrap(~ features, scales = "free", ncol = 3) +
+  scale_color_viridis_d(option = "cividis", end = .8) +
+  theme(legend.position = "none")
+

+

Amazing🤩! For some of the features, there’s a noticeable difference +in the distribution for each color label. For instance, it seems the +white pumpkins can be found in smaller packages and in some particular +varieties of pumpkins. The item_size category also seems to +make a difference in the color distribution. These features may help +predict the color of a pumpkin.

+
+

Use a swarm plot

+

Color is a binary category (Orange or Not), it’s called +categorical data. There are other various ways of visualizing +categorical data.

+

Try a swarm plot to show the distribution of color with +respect to the item_size.

+

We’ll use the ggbeeswarm package +which provides methods to create beeswarm-style plots using ggplot2. +Beeswarm plots are a way of plotting points that would ordinarily +overlap so that they fall next to each other instead.

+
# Create beeswarm plots of color and item_size
+baked_pumpkins %>% 
+  mutate(color = factor(color)) %>% 
+  ggplot(mapping = aes(x = color, y = item_size, color = color)) +
+  geom_quasirandom() +
+  scale_color_brewer(palette = "Dark2", direction = -1) +
+  theme(legend.position = "none")
+

+
+
+

Violin plot

+

A ‘violin’ type plot is useful as you can easily visualize the way +that data in the two categories is distributed. Violin plots +are similar to box plots, except that they also show the probability +density of the data at different values. Violin plots don’t work so well +with smaller datasets as the distribution is displayed more +‘smoothly’.

+
# Create a violin plot of color and item_size
+baked_pumpkins %>%
+  mutate(color = factor(color)) %>% 
+  ggplot(mapping = aes(x = color, y = item_size, fill = color)) +
+  geom_violin() +
+  geom_boxplot(color = "black", fill = "white", width = 0.02) +
+  scale_fill_brewer(palette = "Dark2", direction = -1) +
+  theme(legend.position = "none")
+

+

Now that we have an idea of the relationship between the binary +categories of color and the larger group of sizes, let’s explore +logistic regression to determine a given pumpkin’s likely color.

+
+
+
+

3. Build your model

+
+

🧮 Show Me The Math

+

Remember how linear regression often used +ordinary least squares to arrive at a value? +Logistic regression relies on the concept of ‘maximum +likelihood’ using sigmoid functions. +A Sigmoid Function on a plot looks like an S shape. It +takes a value and maps it to somewhere between 0 and 1. Its curve is +also called a ‘logistic curve’. Its formula looks like this:

+

+

where the sigmoid’s midpoint finds itself at x’s 0 point, L is the +curve’s maximum value, and k is the curve’s steepness. If the outcome of +the function is more than 0.5, the label in question will be given the +class 1 of the binary choice. If not, it will be classified as 0.

+
+

Let’s begin by splitting the data into training and +test sets. The training set is used to train a classifier +so that it finds a statistical relationship between the features and the +label value.

+

It is best practice to hold out some of your data for +testing in order to get a better estimate of how your +models will perform on new data by comparing the predicted labels with +the already known labels in the test set. rsample, a package in +Tidymodels, provides infrastructure for efficient data splitting and +resampling:

+
# Split data into 80% for training and 20% for testing
+set.seed(2056)
+pumpkins_split <- pumpkins_select %>% 
+  initial_split(prop = 0.8)
+
+# Extract the data in each split
+pumpkins_train <- training(pumpkins_split)
+pumpkins_test <- testing(pumpkins_split)
+
+# Print out the first 5 rows of the training set
+pumpkins_train %>% 
+  slice_head(n = 5)
+
+ +
+

🙌 We are now ready to train a model by fitting the training features +to the training label (color).

+

We’ll begin by creating a recipe that specifies the preprocessing +steps that should be carried out on our data to get it ready for +modelling i.e: encoding categorical variables into a set of +integers.

+

There are quite a number of ways to specify a logistic regression +model in Tidymodels. See ?logistic_reg() For now, we’ll +specify a logistic regression model via the default +stats::glm() engine.

+
# Create a recipe that specifies preprocessing steps for modelling
+pumpkins_recipe <- recipe(color ~ ., data = pumpkins_train) %>% 
+  step_integer(all_predictors(), zero_based = TRUE)
+
+
+# Create a logistic model specification
+log_reg <- logistic_reg() %>% 
+  set_engine("glm") %>% 
+  set_mode("classification")
+

Now that we have a recipe and a model specification, we need to find +a way of bundling them together into an object that will first +preprocess the data (prep+bake behind the scenes), fit the model on the +preprocessed data and also allow for potential post-processing +activities.

+

In Tidymodels, this convenient object is called a workflow and +conveniently holds your modeling components.

+
# Bundle modelling components in a workflow
+log_reg_wf <- workflow() %>% 
+  add_recipe(pumpkins_recipe) %>% 
+  add_model(log_reg)
+
+# Print out the workflow
+log_reg_wf
+
## == Workflow ====================================================================
+## Preprocessor: Recipe
+## Model: logistic_reg()
+## 
+## -- Preprocessor ----------------------------------------------------------------
+## 1 Recipe Step
+## 
+## * step_integer()
+## 
+## -- Model -----------------------------------------------------------------------
+## Logistic Regression Model Specification (classification)
+## 
+## Computational engine: glm
+

After a workflow has been specified, a model can be +trained using the fit() +function. The workflow will estimate a recipe and preprocess the data +before training, so we won’t have to manually do that using prep and +bake.

+
# Train the model
+wf_fit <- log_reg_wf %>% 
+  fit(data = pumpkins_train)
+
+# Print the trained workflow
+wf_fit
+
## == Workflow [trained] ==========================================================
+## Preprocessor: Recipe
+## Model: logistic_reg()
+## 
+## -- Preprocessor ----------------------------------------------------------------
+## 1 Recipe Step
+## 
+## * step_integer()
+## 
+## -- Model -----------------------------------------------------------------------
+## 
+## Call:  stats::glm(formula = ..y ~ ., family = stats::binomial, data = data)
+## 
+## Coefficients:
+## (Intercept)    city_name      package      variety       origin    item_size  
+##    -0.52170     -0.01957     -0.58802     -0.07160      0.06825      0.14516  
+## 
+## Degrees of Freedom: 791 Total (i.e. Null);  786 Residual
+## Null Deviance:       700.7 
+## Residual Deviance: 645.3     AIC: 657.3
+

The model print out shows the coefficients learned during +training.

+

Now we’ve trained the model using the training data, we can make +predictions on the test data using parsnip::predict(). +Let’s start by using the model to predict labels for our test set and +the probabilities for each label. When the probability is more than 0.5, +the predict class is ORANGE else WHITE.

+
# Make predictions for color and corresponding probabilities
+results <- pumpkins_test %>% select(color) %>% 
+  bind_cols(wf_fit %>% 
+              predict(new_data = pumpkins_test)) %>%
+  bind_cols(wf_fit %>%
+              predict(new_data = pumpkins_test, type = "prob"))
+
+# Compare predictions
+results %>% 
+  slice_head(n = 10)
+
+ +
+

Very nice! This provides some more insights into how logistic +regression works.

+

Comparing each prediction with its corresponding “ground truth” +actual value isn’t a very efficient way to determine how well the model +is predicting. Fortunately, Tidymodels has a few more tricks up its +sleeve: yardstick - a +package used to measure the effectiveness of models using performance +metrics.

+

One performance metric associated with classification problems is the +confusion matrix. +A confusion matrix describes how well a classification model performs. A +confusion matrix tabulates how many examples in each class were +correctly classified by a model. In our case, it will show you how many +orange pumpkins were classified as orange and how many white pumpkins +were classified as white; the confusion matrix also shows you how many +were classified into the wrong categories.

+

The conf_mat() +function from yardstick calculates this cross-tabulation of observed and +predicted classes.

+
# Confusion matrix for prediction results
+conf_mat(data = results, truth = color, estimate = .pred_class)
+
##           Truth
+## Prediction ORANGE WHITE
+##     ORANGE    170    28
+##     WHITE       1     0
+

Let’s interpret the confusion matrix. Our model is asked to classify +pumpkins between two binary categories, category orange and +category not-orange

+
    +

  • If your model predicts a pumpkin as orange and it belongs to +category ‘orange’ in reality we call it a true positive, +shown by the top left number.

    • +

  • If your model predicts a pumpkin as not orange and it belongs to +category ‘orange’ in reality we call it a false negative, +shown by the bottom left number.

    • +

  • If your model predicts a pumpkin as orange and it belongs to +category ‘not-orange’ in reality we call it a +false positive, shown by the top right number.

    • +

  • If your model predicts a pumpkin as not orange and it belongs to +category ‘not-orange’ in reality we call it a +true negative, shown by the bottom right number.

    • +
+ + + + + + + + +
Truth
+ + + + + + + + + + + + + + + + + + +
PredictedORANGEWHITE
ORANGETPFP
WHITEFNTN
+

As you might have guessed it’s preferable to have a larger number of +true positives and true negatives and a lower number of false positives +and false negatives, which implies that the model performs better.

+

The confusion matrix is helpful since it gives rise to other metrics +that can help us better evaluate the performance of a classification +model. Let’s go through some of them:

+

🎓 Precision: TP/(TP + FP) defined as the proportion of +predicted positives that are actually positive. Also called positive predictive value

+

🎓 Recall: TP/(TP + FN) defined as the proportion of +positive results out of the number of samples which were actually +positive. Also known as sensitivity.

+

🎓 Specificity: TN/(TN + FP) defined as the proportion +of negative results out of the number of samples which were actually +negative.

+

🎓 Accuracy: TP + TN/(TP + TN + FP + FN) The percentage +of labels predicted accurately for a sample.

+

🎓 F Measure: A weighted average of the precision and recall, with +best being 1 and worst being 0.

+

Let’s calculate these metrics!

+
# Combine metric functions and calculate them all at once
+eval_metrics <- metric_set(ppv, recall, spec, f_meas, accuracy)
+eval_metrics(data = results, truth = color, estimate = .pred_class)
+
+ +
+
+

Visualize the ROC curve of this model

+

For a start, this is not a bad model; its precision, recall, F +measure and accuracy are in the 80% range so ideally you could use it to +predict the color of a pumpkin given a set of variables. It also seems +that our model was not really able to identify the white pumpkins 🧐. +Could you guess why? One reason could be because of the high prevalence +of ORANGE pumpkins in our training set making our model more inclined to +predict the majority class.

+

Let’s do one more visualization to see the so-called ROC score:

+
# Make a roc_curve
+results %>% 
+  roc_curve(color, .pred_ORANGE) %>% 
+  autoplot()
+

+

ROC curves are often used to get a view of the output of a classifier +in terms of its true vs. false positives. ROC curves typically feature +True Positive Rate/Sensitivity on the Y axis, and +False Positive Rate/1-Specificity on the X axis. Thus, the +steepness of the curve and the space between the midpoint line and the +curve matter: you want a curve that quickly heads up and over the line. +In our case, there are false positives to start with, and then the line +heads up and over properly.

+

Finally, let’s use yardstick::roc_auc() to calculate the +actual Area Under the Curve. One way of interpreting AUC is as the +probability that the model ranks a random positive example more highly +than a random negative example.

+
# Calculate area under curve
+results %>% 
+  roc_auc(color, .pred_ORANGE)
+
+ +
+

The result is around 0.67053. Given that the AUC ranges +from 0 to 1, you want a big score, since a model that is 100% correct in +its predictions will have an AUC of 1; in this case, the model is +pretty good.

+

In future lessons on classifications, you will learn how to improve +your model’s scores (such as dealing with imbalanced data in this +case).

+

But for now, congratulations 🎉🎉🎉! You’ve completed these +regression lessons!

+

You R awesome!

+
+ +

Artwork by @allison_horst

+
+
+
+ +
LS0tDQp0aXRsZTogJ0J1aWxkIGEgcmVncmVzc2lvbiBtb2RlbDogbG9naXN0aWMgcmVncmVzc2lvbicNCm91dHB1dDoNCiAgaHRtbF9kb2N1bWVudDoNCiAgICBkZl9wcmludDogcGFnZWQNCiAgICB0aGVtZTogZmxhdGx5DQogICAgaGlnaGxpZ2h0OiBicmVlemVkYXJrDQogICAgdG9jOiB5ZXMNCiAgICB0b2NfZmxvYXQ6IHllcw0KICAgIGNvZGVfZG93bmxvYWQ6IHllcw0KLS0tDQoNCiMjIEJ1aWxkIGEgbG9naXN0aWMgcmVncmVzc2lvbiBtb2RlbCAtIExlc3NvbiA0DQoNCiFbSW5mb2dyYXBoaWMgYnkgRGFzYW5pIE1hZGlwYWxsaV0oLi4vLi4vaW1hZ2VzL2xvZ2lzdGljLWxpbmVhci5wbmcpe3dpZHRoPSI2MDAifQ0KDQojIyMjICoqW1ByZS1sZWN0dXJlIHF1aXpdKGh0dHBzOi8vZ3JheS1zYW5kLTA3YTEwZjQwMy4xLmF6dXJlc3RhdGljYXBwcy5uZXQvcXVpei8xNS8pKioNCg0KIyMjIyAgSW50cm9kdWN0aW9uDQoNCkluIHRoaXMgZmluYWwgbGVzc29uIG9uIFJlZ3Jlc3Npb24sIG9uZSBvZiB0aGUgYmFzaWMgKmNsYXNzaWMqIE1MIHRlY2huaXF1ZXMsIHdlIHdpbGwgdGFrZSBhIGxvb2sgYXQgTG9naXN0aWMgUmVncmVzc2lvbi4gWW91IHdvdWxkIHVzZSB0aGlzIHRlY2huaXF1ZSB0byBkaXNjb3ZlciBwYXR0ZXJucyB0byBwcmVkaWN0IGBiaW5hcnlgIGBjYXRlZ29yaWVzYC4gSXMgdGhpcyBjYW5keSBjaG9jb2xhdGUgb3Igbm90PyBJcyB0aGlzIGRpc2Vhc2UgY29udGFnaW91cyBvciBub3Q/IFdpbGwgdGhpcyBjdXN0b21lciBjaG9vc2UgdGhpcyBwcm9kdWN0IG9yIG5vdD8NCg0KSW4gdGhpcyBsZXNzb24sIHlvdSB3aWxsIGxlYXJuOg0KDQotICAgVGVjaG5pcXVlcyBmb3IgbG9naXN0aWMgcmVncmVzc2lvbg0KDQrinIUgRGVlcGVuIHlvdXIgdW5kZXJzdGFuZGluZyBvZiB3b3JraW5nIHdpdGggdGhpcyB0eXBlIG9mIHJlZ3Jlc3Npb24gaW4gdGhpcyBbTGVhcm4gbW9kdWxlXShodHRwczovL2RvY3MubWljcm9zb2Z0LmNvbS9sZWFybi9tb2R1bGVzL3RyYWluLWV2YWx1YXRlLWNsYXNzaWZpY2F0aW9uLW1vZGVscz9XVC5tY19pZD1hY2FkZW1pYy03Nzk1Mi1sZWVzdG90dCkNCg0KIyMjIyAqKlByZXJlcXVpc2l0ZSoqDQoNCkhhdmluZyB3b3JrZWQgd2l0aCB0aGUgcHVtcGtpbiBkYXRhLCB3ZSBhcmUgbm93IGZhbWlsaWFyIGVub3VnaCB3aXRoIGl0IHRvIHJlYWxpemUgdGhhdCB0aGVyZSdzIG9uZSBiaW5hcnkgY2F0ZWdvcnkgdGhhdCB3ZSBjYW4gd29yayB3aXRoOiBgQ29sb3JgLg0KDQpMZXQncyBidWlsZCBhIGxvZ2lzdGljIHJlZ3Jlc3Npb24gbW9kZWwgdG8gcHJlZGljdCB0aGF0LCBnaXZlbiBzb21lIHZhcmlhYmxlcywgKndoYXQgY29sb3IgYSBnaXZlbiBwdW1wa2luIGlzIGxpa2VseSB0byBiZSogKG9yYW5nZSDwn46DIG9yIHdoaXRlIPCfkbspLg0KDQo+IFdoeSBhcmUgd2UgdGFsa2luZyBhYm91dCBiaW5hcnkgY2xhc3NpZmljYXRpb24gaW4gYSBsZXNzb24gZ3JvdXBpbmcgYWJvdXQgcmVncmVzc2lvbj8gT25seSBmb3IgbGluZ3Vpc3RpYyBjb252ZW5pZW5jZSwgYXMgbG9naXN0aWMgcmVncmVzc2lvbiBpcyBbcmVhbGx5IGEgY2xhc3NpZmljYXRpb24gbWV0aG9kXShodHRwczovL3NjaWtpdC1sZWFybi5vcmcvc3RhYmxlL21vZHVsZXMvbGluZWFyX21vZGVsLmh0bWwjbG9naXN0aWMtcmVncmVzc2lvbiksIGFsYmVpdCBhIGxpbmVhci1iYXNlZCBvbmUuIExlYXJuIGFib3V0IG90aGVyIHdheXMgdG8gY2xhc3NpZnkgZGF0YSBpbiB0aGUgbmV4dCBsZXNzb24gZ3JvdXAuDQoNCkZvciB0aGlzIGxlc3Nvbiwgd2UnbGwgcmVxdWlyZSB0aGUgZm9sbG93aW5nIHBhY2thZ2VzOg0KDQotICAgYHRpZHl2ZXJzZWA6IFRoZSBbdGlkeXZlcnNlXShodHRwczovL3d3dy50aWR5dmVyc2Uub3JnLykgaXMgYSBbY29sbGVjdGlvbiBvZiBSIHBhY2thZ2VzXShodHRwczovL3d3dy50aWR5dmVyc2Uub3JnL3BhY2thZ2VzKSBkZXNpZ25lZCB0byBtYWtlcyBkYXRhIHNjaWVuY2UgZmFzdGVyLCBlYXNpZXIgYW5kIG1vcmUgZnVuIQ0KDQotICAgYHRpZHltb2RlbHNgOiBUaGUgW3RpZHltb2RlbHNdKGh0dHBzOi8vd3d3LnRpZHltb2RlbHMub3JnLykgZnJhbWV3b3JrIGlzIGEgW2NvbGxlY3Rpb24gb2YgcGFja2FnZXNdKGh0dHBzOi8vd3d3LnRpZHltb2RlbHMub3JnL3BhY2thZ2VzLykgZm9yIG1vZGVsaW5nIGFuZCBtYWNoaW5lIGxlYXJuaW5nLg0KDQotICAgYGphbml0b3JgOiBUaGUgW2phbml0b3IgcGFja2FnZV0oaHR0cHM6Ly9naXRodWIuY29tL3NmaXJrZS9qYW5pdG9yKSBwcm92aWRlcyBzaW1wbGUgbGl0dGxlIHRvb2xzIGZvciBleGFtaW5pbmcgYW5kIGNsZWFuaW5nIGRpcnR5IGRhdGEuDQoNCi0gICBgZ2diZWVzd2FybWA6IFRoZSBbZ2diZWVzd2FybSBwYWNrYWdlXShodHRwczovL2dpdGh1Yi5jb20vZWNsYXJrZS9nZ2JlZXN3YXJtKSBwcm92aWRlcyBtZXRob2RzIHRvIGNyZWF0ZSBiZWVzd2FybS1zdHlsZSBwbG90cyB1c2luZyBnZ3Bsb3QyLg0KDQpZb3UgY2FuIGhhdmUgdGhlbSBpbnN0YWxsZWQgYXM6DQoNCmBpbnN0YWxsLnBhY2thZ2VzKGMoInRpZHl2ZXJzZSIsICJ0aWR5bW9kZWxzIiwgImphbml0b3IiLCAiZ2diZWVzd2FybSIpKWANCg0KQWx0ZXJuYXRlbHksIHRoZSBzY3JpcHQgYmVsb3cgY2hlY2tzIHdoZXRoZXIgeW91IGhhdmUgdGhlIHBhY2thZ2VzIHJlcXVpcmVkIHRvIGNvbXBsZXRlIHRoaXMgbW9kdWxlIGFuZCBpbnN0YWxscyB0aGVtIGZvciB5b3UgaW4gY2FzZSB0aGV5IGFyZSBtaXNzaW5nLg0KDQpgYGB7ciwgbWVzc2FnZT1GLCB3YXJuaW5nPUZ9DQpzdXBwcmVzc1dhcm5pbmdzKGlmICghcmVxdWlyZSgicGFjbWFuIikpaW5zdGFsbC5wYWNrYWdlcygicGFjbWFuIikpDQoNCnBhY21hbjo6cF9sb2FkKHRpZHl2ZXJzZSwgdGlkeW1vZGVscywgamFuaXRvciwgZ2diZWVzd2FybSkNCmBgYA0KDQojIyAqKkRlZmluZSB0aGUgcXVlc3Rpb24qKg0KDQpGb3Igb3VyIHB1cnBvc2VzLCB3ZSB3aWxsIGV4cHJlc3MgdGhpcyBhcyBhIGJpbmFyeTogJ09yYW5nZScgb3IgJ05vdCBPcmFuZ2UnLiBUaGVyZSBpcyBhbHNvIGEgJ3N0cmlwZWQnIGNhdGVnb3J5IGluIG91ciBkYXRhc2V0IGJ1dCB0aGVyZSBhcmUgZmV3IGluc3RhbmNlcyBvZiBpdCwgc28gd2Ugd2lsbCBub3QgdXNlIGl0LiBJdCBkaXNhcHBlYXJzIG9uY2Ugd2UgcmVtb3ZlIG51bGwgdmFsdWVzIGZyb20gdGhlIGRhdGFzZXQsIGFueXdheS4NCg0KPiDwn46DIEZ1biBmYWN0LCB3ZSBzb21ldGltZXMgY2FsbCB3aGl0ZSBwdW1wa2lucyAnZ2hvc3QnIHB1bXBraW5zLiBUaGV5IGFyZW4ndCB2ZXJ5IGVhc3kgdG8gY2FydmUsIHNvIHRoZXkgYXJlbid0IGFzIHBvcHVsYXIgYXMgdGhlIG9yYW5nZSBvbmVzIGJ1dCB0aGV5IGFyZSBjb29sIGxvb2tpbmchDQoNCiMjICoqQWJvdXQgbG9naXN0aWMgcmVncmVzc2lvbioqDQoNCkxvZ2lzdGljIHJlZ3Jlc3Npb24gZGlmZmVycyBmcm9tIGxpbmVhciByZWdyZXNzaW9uLCB3aGljaCB5b3UgbGVhcm5lZCBhYm91dCBwcmV2aW91c2x5LCBpbiBhIGZldyBpbXBvcnRhbnQgd2F5cy4NCg0KIyMjIyAqKkJpbmFyeSBjbGFzc2lmaWNhdGlvbioqDQoNCkxvZ2lzdGljIHJlZ3Jlc3Npb24gZG9lcyBub3Qgb2ZmZXIgdGhlIHNhbWUgZmVhdHVyZXMgYXMgbGluZWFyIHJlZ3Jlc3Npb24uIFRoZSBmb3JtZXIgb2ZmZXJzIGEgcHJlZGljdGlvbiBhYm91dCBhIGBiaW5hcnkgY2F0ZWdvcnlgICgib3JhbmdlIG9yIG5vdCBvcmFuZ2UiKSB3aGVyZWFzIHRoZSBsYXR0ZXIgaXMgY2FwYWJsZSBvZiBwcmVkaWN0aW5nIGBjb250aW51YWwgdmFsdWVzYCwgZm9yIGV4YW1wbGUgZ2l2ZW4gdGhlIG9yaWdpbiBvZiBhIHB1bXBraW4gYW5kIHRoZSB0aW1lIG9mIGhhcnZlc3QsICpob3cgbXVjaCBpdHMgcHJpY2Ugd2lsbCByaXNlKi4NCg0KIVtJbmZvZ3JhcGhpYyBieSBEYXNhbmkgTWFkaXBhbGxpXSguLi8uLi9pbWFnZXMvcHVtcGtpbi1jbGFzc2lmaWVyLnBuZyl7d2lkdGg9IjYwMCJ9DQoNCiMjIyMgKipPdGhlciBjbGFzc2lmaWNhdGlvbnMqKg0KDQpUaGVyZSBhcmUgb3RoZXIgdHlwZXMgb2YgbG9naXN0aWMgcmVncmVzc2lvbiwgaW5jbHVkaW5nIG11bHRpbm9taWFsIGFuZCBvcmRpbmFsOg0KDQotICAgKipNdWx0aW5vbWlhbCoqLCB3aGljaCBpbnZvbHZlcyBoYXZpbmcgbW9yZSB0aGFuIG9uZSBjYXRlZ29yeSAtICJPcmFuZ2UsIFdoaXRlLCBhbmQgU3RyaXBlZCIuDQoNCi0gICAqKk9yZGluYWwqKiwgd2hpY2ggaW52b2x2ZXMgb3JkZXJlZCBjYXRlZ29yaWVzLCB1c2VmdWwgaWYgd2Ugd2FudGVkIHRvIG9yZGVyIG91ciBvdXRjb21lcyBsb2dpY2FsbHksIGxpa2Ugb3VyIHB1bXBraW5zIHRoYXQgYXJlIG9yZGVyZWQgYnkgYSBmaW5pdGUgbnVtYmVyIG9mIHNpemVzIChtaW5pLHNtLG1lZCxsZyx4bCx4eGwpLg0KDQohW0luZm9ncmFwaGljIGJ5IERhc2FuaSBNYWRpcGFsbGldKC4uLy4uL2ltYWdlcy9tdWx0aW5vbWlhbC1vcmRpbmFsLnBuZyl7d2lkdGg9IjYwMCJ9DQoNClwNCioqSXQncyBzdGlsbCBsaW5lYXIqKg0KDQpFdmVuIHRob3VnaCB0aGlzIHR5cGUgb2YgUmVncmVzc2lvbiBpcyBhbGwgYWJvdXQgJ2NhdGVnb3J5IHByZWRpY3Rpb25zJywgaXQgc3RpbGwgd29ya3MgYmVzdCB3aGVuIHRoZXJlIGlzIGEgY2xlYXIgbGluZWFyIHJlbGF0aW9uc2hpcCBiZXR3ZWVuIHRoZSBkZXBlbmRlbnQgdmFyaWFibGUgKGNvbG9yKSBhbmQgdGhlIG90aGVyIGluZGVwZW5kZW50IHZhcmlhYmxlcyAodGhlIHJlc3Qgb2YgdGhlIGRhdGFzZXQsIGxpa2UgY2l0eSBuYW1lIGFuZCBzaXplKS4gSXQncyBnb29kIHRvIGdldCBhbiBpZGVhIG9mIHdoZXRoZXIgdGhlcmUgaXMgYW55IGxpbmVhcml0eSBkaXZpZGluZyB0aGVzZSB2YXJpYWJsZXMgb3Igbm90Lg0KDQojIyMjICoqVmFyaWFibGVzIERPIE5PVCBoYXZlIHRvIGNvcnJlbGF0ZSoqDQoNClJlbWVtYmVyIGhvdyBsaW5lYXIgcmVncmVzc2lvbiB3b3JrZWQgYmV0dGVyIHdpdGggbW9yZSBjb3JyZWxhdGVkIHZhcmlhYmxlcz8gTG9naXN0aWMgcmVncmVzc2lvbiBpcyB0aGUgb3Bwb3NpdGUgLSB0aGUgdmFyaWFibGVzIGRvbid0IGhhdmUgdG8gYWxpZ24uIFRoYXQgd29ya3MgZm9yIHRoaXMgZGF0YSB3aGljaCBoYXMgc29tZXdoYXQgd2VhayBjb3JyZWxhdGlvbnMuDQoNCiMjIyMgKipZb3UgbmVlZCBhIGxvdCBvZiBjbGVhbiBkYXRhKioNCg0KTG9naXN0aWMgcmVncmVzc2lvbiB3aWxsIGdpdmUgbW9yZSBhY2N1cmF0ZSByZXN1bHRzIGlmIHlvdSB1c2UgbW9yZSBkYXRhOyBvdXIgc21hbGwgZGF0YXNldCBpcyBub3Qgb3B0aW1hbCBmb3IgdGhpcyB0YXNrLCBzbyBrZWVwIHRoYXQgaW4gbWluZC4NCg0K4pyFIFRoaW5rIGFib3V0IHRoZSB0eXBlcyBvZiBkYXRhIHRoYXQgd291bGQgbGVuZCB0aGVtc2VsdmVzIHdlbGwgdG8gbG9naXN0aWMgcmVncmVzc2lvbg0KDQojIyAxLiBUaWR5IHRoZSBkYXRhDQoNCk5vdywgdGhlIGZ1biBiZWdpbnMhIExldCdzIHN0YXJ0IGJ5IGltcG9ydGluZyB0aGUgZGF0YSwgY2xlYW5pbmcgdGhlIGRhdGEgYSBiaXQsIGRyb3BwaW5nIHJvd3MgY29udGFpbmluZyBtaXNzaW5nIHZhbHVlcyBhbmQgc2VsZWN0aW5nIG9ubHkgc29tZSBvZiB0aGUgY29sdW1uczoNCg0KYGBge3IsIHRpZHlyLCBtZXNzYWdlPUYsIHdhcm5pbmc9Rn0NCiMgTG9hZCB0aGUgY29yZSB0aWR5dmVyc2UgcGFja2FnZXMNCmxpYnJhcnkodGlkeXZlcnNlKQ0KDQojIEltcG9ydCB0aGUgZGF0YSBhbmQgY2xlYW4gY29sdW1uIG5hbWVzDQpwdW1wa2lucyA8LSByZWFkX2NzdihmaWxlID0gImh0dHBzOi8vcmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbS9taWNyb3NvZnQvTUwtRm9yLUJlZ2lubmVycy9tYWluLzItUmVncmVzc2lvbi9kYXRhL1VTLXB1bXBraW5zLmNzdiIpICU+JSANCiAgY2xlYW5fbmFtZXMoKQ0KDQojIFNlbGVjdCBkZXNpcmVkIGNvbHVtbnMNCnB1bXBraW5zX3NlbGVjdCA8LSBwdW1wa2lucyAlPiUgDQogIHNlbGVjdChjKGNpdHlfbmFtZSwgcGFja2FnZSwgdmFyaWV0eSwgb3JpZ2luLCBpdGVtX3NpemUsIGNvbG9yKSkgDQoNCiMgRHJvcCByb3dzIGNvbnRhaW5pbmcgbWlzc2luZyB2YWx1ZXMgYW5kIGVuY29kZSBjb2xvciBhcyBmYWN0b3IgKGNhdGVnb3J5KQ0KcHVtcGtpbnNfc2VsZWN0IDwtIHB1bXBraW5zX3NlbGVjdCAlPiUgDQogIGRyb3BfbmEoKSAlPiUgDQogIG11dGF0ZShjb2xvciA9IGZhY3Rvcihjb2xvcikpDQoNCiMgVmlldyB0aGUgZmlyc3QgZmV3IHJvd3MNCnB1bXBraW5zX3NlbGVjdCAlPiUgDQogIHNsaWNlX2hlYWQobiA9IDUpDQoNCmBgYA0KDQpTb21ldGltZXMsIHdlIG1heSB3YW50IHNvbWUgbGl0dGxlIG1vcmUgaW5mb3JtYXRpb24gb24gb3VyIGRhdGEuIFdlIGNhbiBoYXZlIGEgbG9vayBhdCB0aGUgYGRhdGFgLCBgaXRzIHN0cnVjdHVyZWAgYW5kIHRoZSBgZGF0YSB0eXBlYCBvZiBpdHMgZmVhdHVyZXMgYnkgdXNpbmcgdGhlIFsqZ2xpbXBzZSgpKl0oaHR0cHM6Ly9waWxsYXIuci1saWIub3JnL3JlZmVyZW5jZS9nbGltcHNlLmh0bWwpIGZ1bmN0aW9uIGFzIGJlbG93Og0KDQpgYGB7ciBnbGltcHNlfQ0KcHVtcGtpbnNfc2VsZWN0ICU+JSANCiAgZ2xpbXBzZSgpDQpgYGANCg0KV293ISBTZWVtcyB0aGF0IGFsbCBvdXIgY29sdW1ucyBhcmUgYWxsIG9mIHR5cGUgKmNoYXJhY3RlciosIGZ1cnRoZXIgYWxsdWRpbmcgdGhhdCB0aGV5IGFyZSBhbGwgY2F0ZWdvcmljYWwuDQoNCkxldCdzIGNvbmZpcm0gdGhhdCB3ZSB3aWxsIGFjdHVhbGx5IGJlIGRvaW5nIGEgYmluYXJ5IGNsYXNzaWZpY2F0aW9uIHByb2JsZW06DQoNCmBgYHtyIGRpc3RpbmN0IGNvbG9yfQ0KIyBTdWJzZXQgZGlzdGluY3Qgb2JzZXJ2YXRpb25zIGluIG91dGNvbWUgY29sdW1uDQpwdW1wa2luc19zZWxlY3QgJT4lIA0KICBkaXN0aW5jdChjb2xvcikNCg0KYGBgDQoNCvCfpbPwn6WzIFRoYXQgd2VudCBkb3duIHdlbGwhDQoNCiMjIDIuIEV4cGxvcmUgdGhlIGRhdGENCg0KVGhlIGdvYWwgb2YgZGF0YSBleHBsb3JhdGlvbiBpcyB0byB0cnkgdG8gdW5kZXJzdGFuZCB0aGUgYHJlbGF0aW9uc2hpcHNgIGJldHdlZW4gaXRzIGF0dHJpYnV0ZXM7IGluIHBhcnRpY3VsYXIsIGFueSBhcHBhcmVudCBjb3JyZWxhdGlvbiBiZXR3ZWVuIHRoZSAqZmVhdHVyZXMqIGFuZCB0aGUgKmxhYmVsKiB5b3VyIG1vZGVsIHdpbGwgdHJ5IHRvIHByZWRpY3QuIE9uZSB3YXkgb2YgZG9pbmcgdGhpcyBpcyBieSB1c2luZyBkYXRhIHZpc3VhbGl6YXRpb24uDQoNCkdpdmVuIG91ciB0aGUgZGF0YSB0eXBlcyBvZiBvdXIgY29sdW1ucywgd2UgY2FuIGBlbmNvZGVgIHRoZW0gYW5kIGJlIG9uIG91ciB3YXkgdG8gbWFraW5nIHNvbWUgdmlzdWFsaXphdGlvbnMuIFRoaXMgc2ltcGx5IGludm9sdmVzIGB0cmFuc2xhdGluZ2AgYSBjb2x1bW4gd2l0aCBgY2F0ZWdvcmljYWwgdmFsdWVzYCBmb3IgZXhhbXBsZSBvdXIgY29sdW1ucyBvZiB0eXBlICpjaGFyKiwgaW50byBvbmUgb3IgbW9yZSBgbnVtZXJpYyBjb2x1bW5zYCB0aGF0IHRha2UgdGhlIHBsYWNlIG9mIHRoZSBvcmlnaW5hbC4gLSBTb21ldGhpbmcgd2UgZGlkIGluIG91ciBbbGFzdCBsZXNzb25dKGh0dHBzOi8vZ2l0aHViLmNvbS9taWNyb3NvZnQvTUwtRm9yLUJlZ2lubmVycy9ibG9iL21haW4vMi1SZWdyZXNzaW9uLzMtTGluZWFyL3NvbHV0aW9uL2xlc3Nvbl8zLmh0bWwpLg0KDQpUaWR5bW9kZWxzIHByb3ZpZGVzIHlldCBhbm90aGVyIG5lYXQgcGFja2FnZTogW3JlY2lwZXNdKGh0dHBzOi8vcmVjaXBlcy50aWR5bW9kZWxzLm9yZy8pLSBhIHBhY2thZ2UgZm9yIHByZXByb2Nlc3NpbmcgZGF0YS4gV2UnbGwgZGVmaW5lIGEgYHJlY2lwZWAgdGhhdCBzcGVjaWZpZXMgdGhhdCBhbGwgcHJlZGljdG9yIGNvbHVtbnMgc2hvdWxkIGJlIGVuY29kZWQgaW50byBhIHNldCBvZiBpbnRlZ2VycyAsIGBwcmVwYCBpdCB0byBlc3RpbWF0ZXMgdGhlIHJlcXVpcmVkIHF1YW50aXRpZXMgYW5kIHN0YXRpc3RpY3MgbmVlZGVkIGJ5IGFueSBvcGVyYXRpb25zIGFuZCBmaW5hbGx5IGBiYWtlYCB0byBhcHBseSB0aGUgY29tcHV0YXRpb25zIHRvIG5ldyBkYXRhLg0KDQo+IE5vcm1hbGx5LCByZWNpcGVzIGlzIHVzdWFsbHkgdXNlZCBhcyBhIHByZXByb2Nlc3NvciBmb3IgbW9kZWxsaW5nIHdoZXJlIGl0IGRlZmluZXMgd2hhdCBzdGVwcyBzaG91bGQgYmUgYXBwbGllZCB0byBhIGRhdGEgc2V0IGluIG9yZGVyIHRvIGdldCBpdCByZWFkeSBmb3IgbW9kZWxsaW5nLiBJbiB0aGF0IGNhc2UgaXQgaXMgKipoaWdobHkgcmVjb21tZW5kKiogdGhhdCB5b3UgdXNlIGEgYHdvcmtmbG93KClgIGluc3RlYWQgb2YgbWFudWFsbHkgZXN0aW1hdGluZyBhIHJlY2lwZSB1c2luZyBwcmVwIGFuZCBiYWtlLiBXZSdsbCBzZWUgYWxsIHRoaXMgaW4ganVzdCBhIG1vbWVudC4NCj4NCj4gSG93ZXZlciBmb3Igbm93LCB3ZSBhcmUgdXNpbmcgcmVjaXBlcyArIHByZXAgKyBiYWtlIHRvIHNwZWNpZnkgd2hhdCBzdGVwcyBzaG91bGQgYmUgYXBwbGllZCB0byBhIGRhdGEgc2V0IGluIG9yZGVyIHRvIGdldCBpdCByZWFkeSBmb3IgZGF0YSBhbmFseXNpcyBhbmQgdGhlbiBleHRyYWN0IHRoZSBwcmVwcm9jZXNzZWQgZGF0YSB3aXRoIHRoZSBzdGVwcyBhcHBsaWVkLg0KDQpgYGB7ciByZWNpcGVfcHJlcF9iYWtlfQ0KIyBQcmVwcm9jZXNzIGFuZCBleHRyYWN0IGRhdGEgdG8gYWxsb3cgc29tZSBkYXRhIGFuYWx5c2lzDQpiYWtlZF9wdW1wa2lucyA8LSByZWNpcGUoY29sb3IgfiAuLCBkYXRhID0gcHVtcGtpbnNfc2VsZWN0KSAlPiUgDQogICMgRW5jb2RlIGFsbCBjb2x1bW5zIHRvIGEgc2V0IG9mIGludGVnZXJzDQogIHN0ZXBfaW50ZWdlcihhbGxfcHJlZGljdG9ycygpLCB6ZXJvX2Jhc2VkID0gVCkgJT4lIA0KICBwcmVwKCkgJT4lIA0KICBiYWtlKG5ld19kYXRhID0gTlVMTCkNCg0KDQojIERpc3BsYXkgdGhlIGZpcnN0IGZldyByb3dzIG9mIHByZXByb2Nlc3NlZCBkYXRhDQpiYWtlZF9wdW1wa2lucyAlPiUgDQogIHNsaWNlX2hlYWQobiA9IDUpDQoNCmBgYA0KDQpOb3cgbGV0J3MgY29tcGFyZSB0aGUgZmVhdHVyZSBkaXN0cmlidXRpb25zIGZvciBlYWNoIGxhYmVsIHZhbHVlIHVzaW5nIGJveCBwbG90cy4gV2UnbGwgYmVnaW4gYnkgZm9ybWF0dGluZyB0aGUgZGF0YSB0byBhICpsb25nKiBmb3JtYXQgdG8gbWFrZSBpdCBzb21ld2hhdCBlYXNpZXIgdG8gbWFrZSBtdWx0aXBsZSBgZmFjZXRzYC4NCg0KYGBge3IgcGl2b3R9DQojIFBpdm90IGRhdGEgdG8gbG9uZyBmb3JtYXQNCmJha2VkX3B1bXBraW5zX2xvbmcgPC0gYmFrZWRfcHVtcGtpbnMgJT4lIA0KICBwaXZvdF9sb25nZXIoIWNvbG9yLCBuYW1lc190byA9ICJmZWF0dXJlcyIsIHZhbHVlc190byA9ICJ2YWx1ZXMiKQ0KDQoNCiMgUHJpbnQgb3V0IHJlc3RydWN0dXJlZCBkYXRhDQpiYWtlZF9wdW1wa2luc19sb25nICU+JSANCiAgc2xpY2VfaGVhZChuID0gMTApDQoNCmBgYA0KDQoNCk5vdywgbGV0J3MgbWFrZSBzb21lIGJveHBsb3RzIHNob3dpbmcgdGhlIGRpc3RyaWJ1dGlvbiBvZiB0aGUgcHJlZGljdG9ycyB3aXRoIHJlc3BlY3QgdG8gdGhlIG91dGNvbWUgY29sb3IhDQoNCmBgYHtyIGJveHBsb3RzfQ0KdGhlbWVfc2V0KHRoZW1lX2xpZ2h0KCkpDQojTWFrZSBhIGJveCBwbG90IGZvciBlYWNoIHByZWRpY3RvciBmZWF0dXJlDQpiYWtlZF9wdW1wa2luc19sb25nICU+JSANCiAgbXV0YXRlKGNvbG9yID0gZmFjdG9yKGNvbG9yKSkgJT4lIA0KICBnZ3Bsb3QobWFwcGluZyA9IGFlcyh4ID0gY29sb3IsIHkgPSB2YWx1ZXMsIGZpbGwgPSBmZWF0dXJlcykpICsNCiAgZ2VvbV9ib3hwbG90KCkgKyANCiAgZmFjZXRfd3JhcCh+IGZlYXR1cmVzLCBzY2FsZXMgPSAiZnJlZSIsIG5jb2wgPSAzKSArDQogIHNjYWxlX2NvbG9yX3ZpcmlkaXNfZChvcHRpb24gPSAiY2l2aWRpcyIsIGVuZCA9IC44KSArDQogIHRoZW1lKGxlZ2VuZC5wb3NpdGlvbiA9ICJub25lIikNCmBgYA0KDQpBbWF6aW5n8J+kqSEgRm9yIHNvbWUgb2YgdGhlIGZlYXR1cmVzLCB0aGVyZSdzIGEgbm90aWNlYWJsZSBkaWZmZXJlbmNlIGluIHRoZSBkaXN0cmlidXRpb24gZm9yIGVhY2ggY29sb3IgbGFiZWwuIEZvciBpbnN0YW5jZSwgaXQgc2VlbXMgdGhlIHdoaXRlIHB1bXBraW5zIGNhbiBiZSBmb3VuZCBpbiBzbWFsbGVyIHBhY2thZ2VzIGFuZCBpbiBzb21lIHBhcnRpY3VsYXIgdmFyaWV0aWVzIG9mIHB1bXBraW5zLiBUaGUgKml0ZW1fc2l6ZSogY2F0ZWdvcnkgYWxzbyBzZWVtcyB0byBtYWtlIGEgZGlmZmVyZW5jZSBpbiB0aGUgY29sb3IgZGlzdHJpYnV0aW9uLiBUaGVzZSBmZWF0dXJlcyBtYXkgaGVscCBwcmVkaWN0IHRoZSBjb2xvciBvZiBhIHB1bXBraW4uDQoNCiMjIyMgKipVc2UgYSBzd2FybSBwbG90KioNCg0KQ29sb3IgaXMgYSBiaW5hcnkgY2F0ZWdvcnkgKE9yYW5nZSBvciBOb3QpLCBpdCdzIGNhbGxlZCBgY2F0ZWdvcmljYWwgZGF0YWAuIFRoZXJlIGFyZSBvdGhlciB2YXJpb3VzIHdheXMgb2YgW3Zpc3VhbGl6aW5nIGNhdGVnb3JpY2FsIGRhdGFdKGh0dHBzOi8vc2VhYm9ybi5weWRhdGEub3JnL3R1dG9yaWFsL2NhdGVnb3JpY2FsLmh0bWw/aGlnaGxpZ2h0PWJhcikuDQoNClRyeSBhIGBzd2FybSBwbG90YCB0byBzaG93IHRoZSBkaXN0cmlidXRpb24gb2YgY29sb3Igd2l0aCByZXNwZWN0IHRvIHRoZSBpdGVtX3NpemUuDQoNCldlJ2xsIHVzZSB0aGUgW2dnYmVlc3dhcm0gcGFja2FnZV0oaHR0cHM6Ly9naXRodWIuY29tL2VjbGFya2UvZ2diZWVzd2FybSkgd2hpY2ggcHJvdmlkZXMgbWV0aG9kcyB0byBjcmVhdGUgYmVlc3dhcm0tc3R5bGUgcGxvdHMgdXNpbmcgZ2dwbG90Mi4gQmVlc3dhcm0gcGxvdHMgYXJlIGEgd2F5IG9mIHBsb3R0aW5nIHBvaW50cyB0aGF0IHdvdWxkIG9yZGluYXJpbHkgb3ZlcmxhcCBzbyB0aGF0IHRoZXkgZmFsbCBuZXh0IHRvIGVhY2ggb3RoZXIgaW5zdGVhZC4NCg0KYGBge3IgYmVlX3N3YXJtIHBsb3R9DQojIENyZWF0ZSBiZWVzd2FybSBwbG90cyBvZiBjb2xvciBhbmQgaXRlbV9zaXplDQpiYWtlZF9wdW1wa2lucyAlPiUgDQogIG11dGF0ZShjb2xvciA9IGZhY3Rvcihjb2xvcikpICU+JSANCiAgZ2dwbG90KG1hcHBpbmcgPSBhZXMoeCA9IGNvbG9yLCB5ID0gaXRlbV9zaXplLCBjb2xvciA9IGNvbG9yKSkgKw0KICBnZW9tX3F1YXNpcmFuZG9tKCkgKw0KICBzY2FsZV9jb2xvcl9icmV3ZXIocGFsZXR0ZSA9ICJEYXJrMiIsIGRpcmVjdGlvbiA9IC0xKSArDQogIHRoZW1lKGxlZ2VuZC5wb3NpdGlvbiA9ICJub25lIikNCmBgYA0KDQojIyMjICoqVmlvbGluIHBsb3QqKg0KDQpBICd2aW9saW4nIHR5cGUgcGxvdCBpcyB1c2VmdWwgYXMgeW91IGNhbiBlYXNpbHkgdmlzdWFsaXplIHRoZSB3YXkgdGhhdCBkYXRhIGluIHRoZSB0d28gY2F0ZWdvcmllcyBpcyBkaXN0cmlidXRlZC4gW2BWaW9saW4gcGxvdHNgXShodHRwczovL2VuLndpa2lwZWRpYS5vcmcvd2lraS9WaW9saW5fcGxvdCkgYXJlIHNpbWlsYXIgdG8gYm94IHBsb3RzLCBleGNlcHQgdGhhdCB0aGV5IGFsc28gc2hvdyB0aGUgcHJvYmFiaWxpdHkgZGVuc2l0eSBvZiB0aGUgZGF0YSBhdCBkaWZmZXJlbnQgdmFsdWVzLiBWaW9saW4gcGxvdHMgZG9uJ3Qgd29yayBzbyB3ZWxsIHdpdGggc21hbGxlciBkYXRhc2V0cyBhcyB0aGUgZGlzdHJpYnV0aW9uIGlzIGRpc3BsYXllZCBtb3JlICdzbW9vdGhseScuDQoNCmBgYHtyIHZpb2xpbl9wbG90fQ0KIyBDcmVhdGUgYSB2aW9saW4gcGxvdCBvZiBjb2xvciBhbmQgaXRlbV9zaXplDQpiYWtlZF9wdW1wa2lucyAlPiUNCiAgbXV0YXRlKGNvbG9yID0gZmFjdG9yKGNvbG9yKSkgJT4lIA0KICBnZ3Bsb3QobWFwcGluZyA9IGFlcyh4ID0gY29sb3IsIHkgPSBpdGVtX3NpemUsIGZpbGwgPSBjb2xvcikpICsNCiAgZ2VvbV92aW9saW4oKSArDQogIGdlb21fYm94cGxvdChjb2xvciA9ICJibGFjayIsIGZpbGwgPSAid2hpdGUiLCB3aWR0aCA9IDAuMDIpICsNCiAgc2NhbGVfZmlsbF9icmV3ZXIocGFsZXR0ZSA9ICJEYXJrMiIsIGRpcmVjdGlvbiA9IC0xKSArDQogIHRoZW1lKGxlZ2VuZC5wb3NpdGlvbiA9ICJub25lIikNCg0KYGBgDQoNCk5vdyB0aGF0IHdlIGhhdmUgYW4gaWRlYSBvZiB0aGUgcmVsYXRpb25zaGlwIGJldHdlZW4gdGhlIGJpbmFyeSBjYXRlZ29yaWVzIG9mIGNvbG9yIGFuZCB0aGUgbGFyZ2VyIGdyb3VwIG9mIHNpemVzLCBsZXQncyBleHBsb3JlIGxvZ2lzdGljIHJlZ3Jlc3Npb24gdG8gZGV0ZXJtaW5lIGEgZ2l2ZW4gcHVtcGtpbidzIGxpa2VseSBjb2xvci4NCg0KIyMgMy4gQnVpbGQgeW91ciBtb2RlbA0KDQo+ICoq8J+nriBTaG93IE1lIFRoZSBNYXRoKioNCj4NCj4gUmVtZW1iZXIgaG93IGBsaW5lYXIgcmVncmVzc2lvbmAgb2Z0ZW4gdXNlZCBgb3JkaW5hcnkgbGVhc3Qgc3F1YXJlc2AgdG8gYXJyaXZlIGF0IGEgdmFsdWU/IGBMb2dpc3RpYyByZWdyZXNzaW9uYCByZWxpZXMgb24gdGhlIGNvbmNlcHQgb2YgJ21heGltdW0gbGlrZWxpaG9vZCcgdXNpbmcgW2BzaWdtb2lkIGZ1bmN0aW9uc2BdKGh0dHBzOi8vd2lraXBlZGlhLm9yZy93aWtpL1NpZ21vaWRfZnVuY3Rpb24pLiBBIFNpZ21vaWQgRnVuY3Rpb24gb24gYSBwbG90IGxvb2tzIGxpa2UgYW4gYFMgc2hhcGVgLiBJdCB0YWtlcyBhIHZhbHVlIGFuZCBtYXBzIGl0IHRvIHNvbWV3aGVyZSBiZXR3ZWVuIDAgYW5kIDEuIEl0cyBjdXJ2ZSBpcyBhbHNvIGNhbGxlZCBhICdsb2dpc3RpYyBjdXJ2ZScuIEl0cyBmb3JtdWxhIGxvb2tzIGxpa2UgdGhpczoNCj4NCj4gIVtdKC4uLy4uL2ltYWdlcy9zaWdtb2lkLnBuZykNCj4NCj4gd2hlcmUgdGhlIHNpZ21vaWQncyBtaWRwb2ludCBmaW5kcyBpdHNlbGYgYXQgeCdzIDAgcG9pbnQsIEwgaXMgdGhlIGN1cnZlJ3MgbWF4aW11bSB2YWx1ZSwgYW5kIGsgaXMgdGhlIGN1cnZlJ3Mgc3RlZXBuZXNzLiBJZiB0aGUgb3V0Y29tZSBvZiB0aGUgZnVuY3Rpb24gaXMgbW9yZSB0aGFuIDAuNSwgdGhlIGxhYmVsIGluIHF1ZXN0aW9uIHdpbGwgYmUgZ2l2ZW4gdGhlIGNsYXNzIDEgb2YgdGhlIGJpbmFyeSBjaG9pY2UuIElmIG5vdCwgaXQgd2lsbCBiZSBjbGFzc2lmaWVkIGFzIDAuDQoNCkxldCdzIGJlZ2luIGJ5IHNwbGl0dGluZyB0aGUgZGF0YSBpbnRvIGB0cmFpbmluZ2AgYW5kIGB0ZXN0YCBzZXRzLiBUaGUgdHJhaW5pbmcgc2V0IGlzIHVzZWQgdG8gdHJhaW4gYSBjbGFzc2lmaWVyIHNvIHRoYXQgaXQgZmluZHMgYSBzdGF0aXN0aWNhbCByZWxhdGlvbnNoaXAgYmV0d2VlbiB0aGUgZmVhdHVyZXMgYW5kIHRoZSBsYWJlbCB2YWx1ZS4NCg0KSXQgaXMgYmVzdCBwcmFjdGljZSB0byBob2xkIG91dCBzb21lIG9mIHlvdXIgZGF0YSBmb3IgKip0ZXN0aW5nKiogaW4gb3JkZXIgdG8gZ2V0IGEgYmV0dGVyIGVzdGltYXRlIG9mIGhvdyB5b3VyIG1vZGVscyB3aWxsIHBlcmZvcm0gb24gbmV3IGRhdGEgYnkgY29tcGFyaW5nIHRoZSBwcmVkaWN0ZWQgbGFiZWxzIHdpdGggdGhlIGFscmVhZHkga25vd24gbGFiZWxzIGluIHRoZSB0ZXN0IHNldC4gW3JzYW1wbGVdKGh0dHBzOi8vcnNhbXBsZS50aWR5bW9kZWxzLm9yZy8pLCBhIHBhY2thZ2UgaW4gVGlkeW1vZGVscywgcHJvdmlkZXMgaW5mcmFzdHJ1Y3R1cmUgZm9yIGVmZmljaWVudCBkYXRhIHNwbGl0dGluZyBhbmQgcmVzYW1wbGluZzoNCg0KYGBge3Igc3BsaXRfZGF0YX0NCiMgU3BsaXQgZGF0YSBpbnRvIDgwJSBmb3IgdHJhaW5pbmcgYW5kIDIwJSBmb3IgdGVzdGluZw0Kc2V0LnNlZWQoMjA1NikNCnB1bXBraW5zX3NwbGl0IDwtIHB1bXBraW5zX3NlbGVjdCAlPiUgDQogIGluaXRpYWxfc3BsaXQocHJvcCA9IDAuOCkNCg0KIyBFeHRyYWN0IHRoZSBkYXRhIGluIGVhY2ggc3BsaXQNCnB1bXBraW5zX3RyYWluIDwtIHRyYWluaW5nKHB1bXBraW5zX3NwbGl0KQ0KcHVtcGtpbnNfdGVzdCA8LSB0ZXN0aW5nKHB1bXBraW5zX3NwbGl0KQ0KDQojIFByaW50IG91dCB0aGUgZmlyc3QgNSByb3dzIG9mIHRoZSB0cmFpbmluZyBzZXQNCnB1bXBraW5zX3RyYWluICU+JSANCiAgc2xpY2VfaGVhZChuID0gNSkNCg0KDQpgYGANCg0K8J+ZjCBXZSBhcmUgbm93IHJlYWR5IHRvIHRyYWluIGEgbW9kZWwgYnkgZml0dGluZyB0aGUgdHJhaW5pbmcgZmVhdHVyZXMgdG8gdGhlIHRyYWluaW5nIGxhYmVsIChjb2xvcikuDQoNCldlJ2xsIGJlZ2luIGJ5IGNyZWF0aW5nIGEgcmVjaXBlIHRoYXQgc3BlY2lmaWVzIHRoZSBwcmVwcm9jZXNzaW5nIHN0ZXBzIHRoYXQgc2hvdWxkIGJlIGNhcnJpZWQgb3V0IG9uIG91ciBkYXRhIHRvIGdldCBpdCByZWFkeSBmb3IgbW9kZWxsaW5nIGkuZTogZW5jb2RpbmcgY2F0ZWdvcmljYWwgdmFyaWFibGVzIGludG8gYSBzZXQgb2YgaW50ZWdlcnMuDQoNClRoZXJlIGFyZSBxdWl0ZSBhIG51bWJlciBvZiB3YXlzIHRvIHNwZWNpZnkgYSBsb2dpc3RpYyByZWdyZXNzaW9uIG1vZGVsIGluIFRpZHltb2RlbHMuIFNlZSBgP2xvZ2lzdGljX3JlZygpYCBGb3Igbm93LCB3ZSdsbCBzcGVjaWZ5IGEgbG9naXN0aWMgcmVncmVzc2lvbiBtb2RlbCB2aWEgdGhlIGRlZmF1bHQgYHN0YXRzOjpnbG0oKWAgZW5naW5lLg0KDQpgYGB7ciBsb2dfcmVnfQ0KIyBDcmVhdGUgYSByZWNpcGUgdGhhdCBzcGVjaWZpZXMgcHJlcHJvY2Vzc2luZyBzdGVwcyBmb3IgbW9kZWxsaW5nDQpwdW1wa2luc19yZWNpcGUgPC0gcmVjaXBlKGNvbG9yIH4gLiwgZGF0YSA9IHB1bXBraW5zX3RyYWluKSAlPiUgDQogIHN0ZXBfaW50ZWdlcihhbGxfcHJlZGljdG9ycygpLCB6ZXJvX2Jhc2VkID0gVFJVRSkNCg0KDQojIENyZWF0ZSBhIGxvZ2lzdGljIG1vZGVsIHNwZWNpZmljYXRpb24NCmxvZ19yZWcgPC0gbG9naXN0aWNfcmVnKCkgJT4lIA0KICBzZXRfZW5naW5lKCJnbG0iKSAlPiUgDQogIHNldF9tb2RlKCJjbGFzc2lmaWNhdGlvbiIpDQoNCg0KYGBgDQoNCk5vdyB0aGF0IHdlIGhhdmUgYSByZWNpcGUgYW5kIGEgbW9kZWwgc3BlY2lmaWNhdGlvbiwgd2UgbmVlZCB0byBmaW5kIGEgd2F5IG9mIGJ1bmRsaW5nIHRoZW0gdG9nZXRoZXIgaW50byBhbiBvYmplY3QgdGhhdCB3aWxsIGZpcnN0IHByZXByb2Nlc3MgdGhlIGRhdGEgKHByZXArYmFrZSBiZWhpbmQgdGhlIHNjZW5lcyksIGZpdCB0aGUgbW9kZWwgb24gdGhlIHByZXByb2Nlc3NlZCBkYXRhIGFuZCBhbHNvIGFsbG93IGZvciBwb3RlbnRpYWwgcG9zdC1wcm9jZXNzaW5nIGFjdGl2aXRpZXMuDQoNCkluIFRpZHltb2RlbHMsIHRoaXMgY29udmVuaWVudCBvYmplY3QgaXMgY2FsbGVkIGEgW2B3b3JrZmxvd2BdKGh0dHBzOi8vd29ya2Zsb3dzLnRpZHltb2RlbHMub3JnLykgYW5kIGNvbnZlbmllbnRseSBob2xkcyB5b3VyIG1vZGVsaW5nIGNvbXBvbmVudHMuDQoNCmBgYHtyIHdvcmtmbG93fQ0KIyBCdW5kbGUgbW9kZWxsaW5nIGNvbXBvbmVudHMgaW4gYSB3b3JrZmxvdw0KbG9nX3JlZ193ZiA8LSB3b3JrZmxvdygpICU+JSANCiAgYWRkX3JlY2lwZShwdW1wa2luc19yZWNpcGUpICU+JSANCiAgYWRkX21vZGVsKGxvZ19yZWcpDQoNCiMgUHJpbnQgb3V0IHRoZSB3b3JrZmxvdw0KbG9nX3JlZ193Zg0KDQoNCmBgYA0KDQpBZnRlciBhIHdvcmtmbG93IGhhcyBiZWVuICpzcGVjaWZpZWQqLCBhIG1vZGVsIGNhbiBiZSBgdHJhaW5lZGAgdXNpbmcgdGhlIFtgZml0KClgXShodHRwczovL3RpZHltb2RlbHMuZ2l0aHViLmlvL3BhcnNuaXAvcmVmZXJlbmNlL2ZpdC5odG1sKSBmdW5jdGlvbi4gVGhlIHdvcmtmbG93IHdpbGwgZXN0aW1hdGUgYSByZWNpcGUgYW5kIHByZXByb2Nlc3MgdGhlIGRhdGEgYmVmb3JlIHRyYWluaW5nLCBzbyB3ZSB3b24ndCBoYXZlIHRvIG1hbnVhbGx5IGRvIHRoYXQgdXNpbmcgcHJlcCBhbmQgYmFrZS4NCg0KYGBge3IgdHJhaW59DQojIFRyYWluIHRoZSBtb2RlbA0Kd2ZfZml0IDwtIGxvZ19yZWdfd2YgJT4lIA0KICBmaXQoZGF0YSA9IHB1bXBraW5zX3RyYWluKQ0KDQojIFByaW50IHRoZSB0cmFpbmVkIHdvcmtmbG93DQp3Zl9maXQNCg0KYGBgDQoNClRoZSBtb2RlbCBwcmludCBvdXQgc2hvd3MgdGhlIGNvZWZmaWNpZW50cyBsZWFybmVkIGR1cmluZyB0cmFpbmluZy4NCg0KTm93IHdlJ3ZlIHRyYWluZWQgdGhlIG1vZGVsIHVzaW5nIHRoZSB0cmFpbmluZyBkYXRhLCB3ZSBjYW4gbWFrZSBwcmVkaWN0aW9ucyBvbiB0aGUgdGVzdCBkYXRhIHVzaW5nIFtwYXJzbmlwOjpwcmVkaWN0KCldKGh0dHBzOi8vcGFyc25pcC50aWR5bW9kZWxzLm9yZy9yZWZlcmVuY2UvcHJlZGljdC5tb2RlbF9maXQuaHRtbCkuIExldCdzIHN0YXJ0IGJ5IHVzaW5nIHRoZSBtb2RlbCB0byBwcmVkaWN0IGxhYmVscyBmb3Igb3VyIHRlc3Qgc2V0IGFuZCB0aGUgcHJvYmFiaWxpdGllcyBmb3IgZWFjaCBsYWJlbC4gV2hlbiB0aGUgcHJvYmFiaWxpdHkgaXMgbW9yZSB0aGFuIDAuNSwgdGhlIHByZWRpY3QgY2xhc3MgaXMgYE9SQU5HRWAgZWxzZSBgV0hJVEVgLg0KDQpgYGB7ciB0ZXN0X3ByZWR9DQojIE1ha2UgcHJlZGljdGlvbnMgZm9yIGNvbG9yIGFuZCBjb3JyZXNwb25kaW5nIHByb2JhYmlsaXRpZXMNCnJlc3VsdHMgPC0gcHVtcGtpbnNfdGVzdCAlPiUgc2VsZWN0KGNvbG9yKSAlPiUgDQogIGJpbmRfY29scyh3Zl9maXQgJT4lIA0KICAgICAgICAgICAgICBwcmVkaWN0KG5ld19kYXRhID0gcHVtcGtpbnNfdGVzdCkpICU+JQ0KICBiaW5kX2NvbHMod2ZfZml0ICU+JQ0KICAgICAgICAgICAgICBwcmVkaWN0KG5ld19kYXRhID0gcHVtcGtpbnNfdGVzdCwgdHlwZSA9ICJwcm9iIikpDQoNCiMgQ29tcGFyZSBwcmVkaWN0aW9ucw0KcmVzdWx0cyAlPiUgDQogIHNsaWNlX2hlYWQobiA9IDEwKQ0KDQpgYGANCg0KVmVyeSBuaWNlISBUaGlzIHByb3ZpZGVzIHNvbWUgbW9yZSBpbnNpZ2h0cyBpbnRvIGhvdyBsb2dpc3RpYyByZWdyZXNzaW9uIHdvcmtzLg0KDQpDb21wYXJpbmcgZWFjaCBwcmVkaWN0aW9uIHdpdGggaXRzIGNvcnJlc3BvbmRpbmcgImdyb3VuZCB0cnV0aCIgYWN0dWFsIHZhbHVlIGlzbid0IGEgdmVyeSBlZmZpY2llbnQgd2F5IHRvIGRldGVybWluZSBob3cgd2VsbCB0aGUgbW9kZWwgaXMgcHJlZGljdGluZy4gRm9ydHVuYXRlbHksIFRpZHltb2RlbHMgaGFzIGEgZmV3IG1vcmUgdHJpY2tzIHVwIGl0cyBzbGVldmU6IFtgeWFyZHN0aWNrYF0oaHR0cHM6Ly95YXJkc3RpY2sudGlkeW1vZGVscy5vcmcvKSAtIGEgcGFja2FnZSB1c2VkIHRvIG1lYXN1cmUgdGhlIGVmZmVjdGl2ZW5lc3Mgb2YgbW9kZWxzIHVzaW5nIHBlcmZvcm1hbmNlIG1ldHJpY3MuDQoNCk9uZSBwZXJmb3JtYW5jZSBtZXRyaWMgYXNzb2NpYXRlZCB3aXRoIGNsYXNzaWZpY2F0aW9uIHByb2JsZW1zIGlzIHRoZSBbYGNvbmZ1c2lvbiBtYXRyaXhgXShodHRwczovL3dpa2lwZWRpYS5vcmcvd2lraS9Db25mdXNpb25fbWF0cml4KS4gQSBjb25mdXNpb24gbWF0cml4IGRlc2NyaWJlcyBob3cgd2VsbCBhIGNsYXNzaWZpY2F0aW9uIG1vZGVsIHBlcmZvcm1zLiBBIGNvbmZ1c2lvbiBtYXRyaXggdGFidWxhdGVzIGhvdyBtYW55IGV4YW1wbGVzIGluIGVhY2ggY2xhc3Mgd2VyZSBjb3JyZWN0bHkgY2xhc3NpZmllZCBieSBhIG1vZGVsLiBJbiBvdXIgY2FzZSwgaXQgd2lsbCBzaG93IHlvdSBob3cgbWFueSBvcmFuZ2UgcHVtcGtpbnMgd2VyZSBjbGFzc2lmaWVkIGFzIG9yYW5nZSBhbmQgaG93IG1hbnkgd2hpdGUgcHVtcGtpbnMgd2VyZSBjbGFzc2lmaWVkIGFzIHdoaXRlOyB0aGUgY29uZnVzaW9uIG1hdHJpeCBhbHNvIHNob3dzIHlvdSBob3cgbWFueSB3ZXJlIGNsYXNzaWZpZWQgaW50byB0aGUgKip3cm9uZyoqIGNhdGVnb3JpZXMuDQoNClRoZSBbKipgY29uZl9tYXQoKWAqKl0oaHR0cHM6Ly90aWR5bW9kZWxzLmdpdGh1Yi5pby95YXJkc3RpY2svcmVmZXJlbmNlL2NvbmZfbWF0Lmh0bWwpIGZ1bmN0aW9uIGZyb20geWFyZHN0aWNrIGNhbGN1bGF0ZXMgdGhpcyBjcm9zcy10YWJ1bGF0aW9uIG9mIG9ic2VydmVkIGFuZCBwcmVkaWN0ZWQgY2xhc3Nlcy4NCg0KYGBge3IgY29uZl9tYXR9DQojIENvbmZ1c2lvbiBtYXRyaXggZm9yIHByZWRpY3Rpb24gcmVzdWx0cw0KY29uZl9tYXQoZGF0YSA9IHJlc3VsdHMsIHRydXRoID0gY29sb3IsIGVzdGltYXRlID0gLnByZWRfY2xhc3MpDQoNCg0KYGBgDQoNCkxldCdzIGludGVycHJldCB0aGUgY29uZnVzaW9uIG1hdHJpeC4gT3VyIG1vZGVsIGlzIGFza2VkIHRvIGNsYXNzaWZ5IHB1bXBraW5zIGJldHdlZW4gdHdvIGJpbmFyeSBjYXRlZ29yaWVzLCBjYXRlZ29yeSBgb3JhbmdlYCBhbmQgY2F0ZWdvcnkgYG5vdC1vcmFuZ2VgDQoNCi0gICBJZiB5b3VyIG1vZGVsIHByZWRpY3RzIGEgcHVtcGtpbiBhcyBvcmFuZ2UgYW5kIGl0IGJlbG9uZ3MgdG8gY2F0ZWdvcnkgJ29yYW5nZScgaW4gcmVhbGl0eSB3ZSBjYWxsIGl0IGEgYHRydWUgcG9zaXRpdmVgLCBzaG93biBieSB0aGUgdG9wIGxlZnQgbnVtYmVyLg0KDQotICAgSWYgeW91ciBtb2RlbCBwcmVkaWN0cyBhIHB1bXBraW4gYXMgbm90IG9yYW5nZSBhbmQgaXQgYmVsb25ncyB0byBjYXRlZ29yeSAnb3JhbmdlJyBpbiByZWFsaXR5IHdlIGNhbGwgaXQgYSBgZmFsc2UgbmVnYXRpdmVgLCBzaG93biBieSB0aGUgYm90dG9tIGxlZnQgbnVtYmVyLg0KDQotICAgSWYgeW91ciBtb2RlbCBwcmVkaWN0cyBhIHB1bXBraW4gYXMgb3JhbmdlIGFuZCBpdCBiZWxvbmdzIHRvIGNhdGVnb3J5ICdub3Qtb3JhbmdlJyBpbiByZWFsaXR5IHdlIGNhbGwgaXQgYSBgZmFsc2UgcG9zaXRpdmVgLCBzaG93biBieSB0aGUgdG9wIHJpZ2h0IG51bWJlci4NCg0KLSAgIElmIHlvdXIgbW9kZWwgcHJlZGljdHMgYSBwdW1wa2luIGFzIG5vdCBvcmFuZ2UgYW5kIGl0IGJlbG9uZ3MgdG8gY2F0ZWdvcnkgJ25vdC1vcmFuZ2UnIGluIHJlYWxpdHkgd2UgY2FsbCBpdCBhIGB0cnVlIG5lZ2F0aXZlYCwgc2hvd24gYnkgdGhlIGJvdHRvbSByaWdodCBudW1iZXIuDQoNCnwgVHJ1dGggfA0KfDotLS0tLTp8DQoNCg0KfCAgICAgICAgICAgICAgIHwgICAgICAgIHwgICAgICAgfA0KfC0tLS0tLS0tLS0tLS0tLXwtLS0tLS0tLXwtLS0tLS0tfA0KfCAqKlByZWRpY3RlZCoqIHwgT1JBTkdFIHwgV0hJVEUgfA0KfCBPUkFOR0UgICAgICAgIHwgVFAgICAgIHwgRlAgICAgfA0KfCBXSElURSAgICAgICAgIHwgRk4gICAgIHwgVE4gICAgfA0KDQpBcyB5b3UgbWlnaHQgaGF2ZSBndWVzc2VkIGl0J3MgcHJlZmVyYWJsZSB0byBoYXZlIGEgbGFyZ2VyIG51bWJlciBvZiB0cnVlIHBvc2l0aXZlcyBhbmQgdHJ1ZSBuZWdhdGl2ZXMgYW5kIGEgbG93ZXIgbnVtYmVyIG9mIGZhbHNlIHBvc2l0aXZlcyBhbmQgZmFsc2UgbmVnYXRpdmVzLCB3aGljaCBpbXBsaWVzIHRoYXQgdGhlIG1vZGVsIHBlcmZvcm1zIGJldHRlci4NCg0KVGhlIGNvbmZ1c2lvbiBtYXRyaXggaXMgaGVscGZ1bCBzaW5jZSBpdCBnaXZlcyByaXNlIHRvIG90aGVyIG1ldHJpY3MgdGhhdCBjYW4gaGVscCB1cyBiZXR0ZXIgZXZhbHVhdGUgdGhlIHBlcmZvcm1hbmNlIG9mIGEgY2xhc3NpZmljYXRpb24gbW9kZWwuIExldCdzIGdvIHRocm91Z2ggc29tZSBvZiB0aGVtOg0KDQrwn46TIFByZWNpc2lvbjogYFRQLyhUUCArIEZQKWAgZGVmaW5lZCBhcyB0aGUgcHJvcG9ydGlvbiBvZiBwcmVkaWN0ZWQgcG9zaXRpdmVzIHRoYXQgYXJlIGFjdHVhbGx5IHBvc2l0aXZlLiBBbHNvIGNhbGxlZCBbcG9zaXRpdmUgcHJlZGljdGl2ZSB2YWx1ZV0oaHR0cHM6Ly9lbi53aWtpcGVkaWEub3JnL3dpa2kvUG9zaXRpdmVfcHJlZGljdGl2ZV92YWx1ZSAiUG9zaXRpdmUgcHJlZGljdGl2ZSB2YWx1ZSIpDQoNCvCfjpMgUmVjYWxsOiBgVFAvKFRQICsgRk4pYCBkZWZpbmVkIGFzIHRoZSBwcm9wb3J0aW9uIG9mIHBvc2l0aXZlIHJlc3VsdHMgb3V0IG9mIHRoZSBudW1iZXIgb2Ygc2FtcGxlcyB3aGljaCB3ZXJlIGFjdHVhbGx5IHBvc2l0aXZlLiBBbHNvIGtub3duIGFzIGBzZW5zaXRpdml0eWAuDQoNCvCfjpMgU3BlY2lmaWNpdHk6IGBUTi8oVE4gKyBGUClgIGRlZmluZWQgYXMgdGhlIHByb3BvcnRpb24gb2YgbmVnYXRpdmUgcmVzdWx0cyBvdXQgb2YgdGhlIG51bWJlciBvZiBzYW1wbGVzIHdoaWNoIHdlcmUgYWN0dWFsbHkgbmVnYXRpdmUuDQoNCvCfjpMgQWNjdXJhY3k6IGBUUCArIFROLyhUUCArIFROICsgRlAgKyBGTilgIFRoZSBwZXJjZW50YWdlIG9mIGxhYmVscyBwcmVkaWN0ZWQgYWNjdXJhdGVseSBmb3IgYSBzYW1wbGUuDQoNCvCfjpMgRiBNZWFzdXJlOiBBIHdlaWdodGVkIGF2ZXJhZ2Ugb2YgdGhlIHByZWNpc2lvbiBhbmQgcmVjYWxsLCB3aXRoIGJlc3QgYmVpbmcgMSBhbmQgd29yc3QgYmVpbmcgMC4NCg0KTGV0J3MgY2FsY3VsYXRlIHRoZXNlIG1ldHJpY3MhDQoNCmBgYHtyIG1ldHJpY19zZXR9DQojIENvbWJpbmUgbWV0cmljIGZ1bmN0aW9ucyBhbmQgY2FsY3VsYXRlIHRoZW0gYWxsIGF0IG9uY2UNCmV2YWxfbWV0cmljcyA8LSBtZXRyaWNfc2V0KHBwdiwgcmVjYWxsLCBzcGVjLCBmX21lYXMsIGFjY3VyYWN5KQ0KZXZhbF9tZXRyaWNzKGRhdGEgPSByZXN1bHRzLCB0cnV0aCA9IGNvbG9yLCBlc3RpbWF0ZSA9IC5wcmVkX2NsYXNzKQ0KYGBgDQoNCiMjIyMgKipWaXN1YWxpemUgdGhlIFJPQyBjdXJ2ZSBvZiB0aGlzIG1vZGVsKioNCg0KRm9yIGEgc3RhcnQsIHRoaXMgaXMgbm90IGEgYmFkIG1vZGVsOyBpdHMgcHJlY2lzaW9uLCByZWNhbGwsIEYgbWVhc3VyZSBhbmQgYWNjdXJhY3kgYXJlIGluIHRoZSA4MCUgcmFuZ2Ugc28gaWRlYWxseSB5b3UgY291bGQgdXNlIGl0IHRvIHByZWRpY3QgdGhlIGNvbG9yIG9mIGEgcHVtcGtpbiBnaXZlbiBhIHNldCBvZiB2YXJpYWJsZXMuIEl0IGFsc28gc2VlbXMgdGhhdCBvdXIgbW9kZWwgd2FzIG5vdCByZWFsbHkgYWJsZSB0byBpZGVudGlmeSB0aGUgd2hpdGUgcHVtcGtpbnMg8J+nkC4gQ291bGQgeW91IGd1ZXNzIHdoeT8gT25lIHJlYXNvbiBjb3VsZCBiZSBiZWNhdXNlIG9mIHRoZSBoaWdoIHByZXZhbGVuY2Ugb2YgT1JBTkdFIHB1bXBraW5zIGluIG91ciB0cmFpbmluZyBzZXQgbWFraW5nIG91ciBtb2RlbCBtb3JlIGluY2xpbmVkIHRvIHByZWRpY3QgdGhlIG1ham9yaXR5IGNsYXNzLg0KDQpMZXQncyBkbyBvbmUgbW9yZSB2aXN1YWxpemF0aW9uIHRvIHNlZSB0aGUgc28tY2FsbGVkIFtgUk9DIHNjb3JlYF0oaHR0cHM6Ly9lbi53aWtpcGVkaWEub3JnL3dpa2kvUmVjZWl2ZXJfb3BlcmF0aW5nX2NoYXJhY3RlcmlzdGljKToNCg0KYGBge3Igcm9jX2N1cnZlfQ0KIyBNYWtlIGEgcm9jX2N1cnZlDQpyZXN1bHRzICU+JSANCiAgcm9jX2N1cnZlKGNvbG9yLCAucHJlZF9PUkFOR0UpICU+JSANCiAgYXV0b3Bsb3QoKQ0KDQpgYGANCg0KUk9DIGN1cnZlcyBhcmUgb2Z0ZW4gdXNlZCB0byBnZXQgYSB2aWV3IG9mIHRoZSBvdXRwdXQgb2YgYSBjbGFzc2lmaWVyIGluIHRlcm1zIG9mIGl0cyB0cnVlIHZzLiBmYWxzZSBwb3NpdGl2ZXMuIFJPQyBjdXJ2ZXMgdHlwaWNhbGx5IGZlYXR1cmUgYFRydWUgUG9zaXRpdmUgUmF0ZWAvU2Vuc2l0aXZpdHkgb24gdGhlIFkgYXhpcywgYW5kIGBGYWxzZSBQb3NpdGl2ZSBSYXRlYC8xLVNwZWNpZmljaXR5IG9uIHRoZSBYIGF4aXMuIFRodXMsIHRoZSBzdGVlcG5lc3Mgb2YgdGhlIGN1cnZlIGFuZCB0aGUgc3BhY2UgYmV0d2VlbiB0aGUgbWlkcG9pbnQgbGluZSBhbmQgdGhlIGN1cnZlIG1hdHRlcjogeW91IHdhbnQgYSBjdXJ2ZSB0aGF0IHF1aWNrbHkgaGVhZHMgdXAgYW5kIG92ZXIgdGhlIGxpbmUuIEluIG91ciBjYXNlLCB0aGVyZSBhcmUgZmFsc2UgcG9zaXRpdmVzIHRvIHN0YXJ0IHdpdGgsIGFuZCB0aGVuIHRoZSBsaW5lIGhlYWRzIHVwIGFuZCBvdmVyIHByb3Blcmx5Lg0KDQpGaW5hbGx5LCBsZXQncyB1c2UgYHlhcmRzdGljazo6cm9jX2F1YygpYCB0byBjYWxjdWxhdGUgdGhlIGFjdHVhbCBBcmVhIFVuZGVyIHRoZSBDdXJ2ZS4gT25lIHdheSBvZiBpbnRlcnByZXRpbmcgQVVDIGlzIGFzIHRoZSBwcm9iYWJpbGl0eSB0aGF0IHRoZSBtb2RlbCByYW5rcyBhIHJhbmRvbSBwb3NpdGl2ZSBleGFtcGxlIG1vcmUgaGlnaGx5IHRoYW4gYSByYW5kb20gbmVnYXRpdmUgZXhhbXBsZS4NCg0KYGBge3Igcm9jX2FvY30NCiMgQ2FsY3VsYXRlIGFyZWEgdW5kZXIgY3VydmUNCnJlc3VsdHMgJT4lIA0KICByb2NfYXVjKGNvbG9yLCAucHJlZF9PUkFOR0UpDQoNCmBgYA0KDQpUaGUgcmVzdWx0IGlzIGFyb3VuZCBgMC42NzA1M2AuIEdpdmVuIHRoYXQgdGhlIEFVQyByYW5nZXMgZnJvbSAwIHRvIDEsIHlvdSB3YW50IGEgYmlnIHNjb3JlLCBzaW5jZSBhIG1vZGVsIHRoYXQgaXMgMTAwJSBjb3JyZWN0IGluIGl0cyBwcmVkaWN0aW9ucyB3aWxsIGhhdmUgYW4gQVVDIG9mIDE7IGluIHRoaXMgY2FzZSwgdGhlIG1vZGVsIGlzICpwcmV0dHkgZ29vZCouDQoNCkluIGZ1dHVyZSBsZXNzb25zIG9uIGNsYXNzaWZpY2F0aW9ucywgeW91IHdpbGwgbGVhcm4gaG93IHRvIGltcHJvdmUgeW91ciBtb2RlbCdzIHNjb3JlcyAoc3VjaCBhcyBkZWFsaW5nIHdpdGggaW1iYWxhbmNlZCBkYXRhIGluIHRoaXMgY2FzZSkuDQoNCkJ1dCBmb3Igbm93LCBjb25ncmF0dWxhdGlvbnMg8J+OifCfjonwn46JISBZb3UndmUgY29tcGxldGVkIHRoZXNlIHJlZ3Jlc3Npb24gbGVzc29ucyENCg0KWW91IFIgYXdlc29tZSENCg0KIVtBcnR3b3JrIGJ5IFxAYWxsaXNvbl9ob3JzdF0oLi4vLi4vaW1hZ2VzL3JfbGVhcm5lcnNfc20uanBlZykNCg0KDQo=
+ + +
+
+ +
+ + + + + + + + + + + + + + + + + diff --git a/README.md b/README.md index 706efc4e..cf354db8 100644 --- a/README.md +++ b/README.md @@ -97,7 +97,7 @@ By ensuring that the content aligns with projects, the process is made more enga | 05 | Introduction to regression | [Regression](2-Regression/README.md) | Get started with Python and Scikit-learn for regression models | | | | 06 | North American pumpkin prices 🎃 | [Regression](2-Regression/README.md) | Visualize and clean data in preparation for ML | | | | 07 | North American pumpkin prices 🎃 | [Regression](2-Regression/README.md) | Build linear and polynomial regression models | | | -| 08 | North American pumpkin prices 🎃 | [Regression](2-Regression/README.md) | Build a logistic regression model | | | +| 08 | North American pumpkin prices 🎃 | [Regression](2-Regression/README.md) | Build a logistic regression model | | | | 09 | A Web App 🔌 | [Web App](3-Web-App/README.md) | Build a web app to use your trained model | [Python](3-Web-App/1-Web-App/README.md) | Jen | | 10 | Introduction to classification | [Classification](4-Classification/README.md) | Clean, prep, and visualize your data; introduction to classification |