In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
"
+ ]
},
- "metadata": {
- "needs_background": "light"
- }
+ "metadata": {},
+ "output_type": "display_data"
}
],
"source": [
@@ -183,5 +633,32 @@
"plt.show()"
]
}
- ]
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.11.1"
+ },
+ "metadata": {
+ "interpreter": {
+ "hash": "70b38d7a306a849643e446cd70466270a13445e5987dfa1344ef2b127438fa4d"
+ }
+ },
+ "orig_nbformat": 2
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
}
diff --git a/2-Regression/2-Data/README.md b/2-Regression/2-Data/README.md
index 8427313e..298f552e 100644
--- a/2-Regression/2-Data/README.md
+++ b/2-Regression/2-Data/README.md
@@ -73,11 +73,11 @@ Open the _notebook.ipynb_ file in Visual Studio Code and import the spreadsheet
There is missing data, but maybe it won't matter for the task at hand.
-1. To make your dataframe easier to work with, drop several of its columns, using `drop()`, keeping only the columns you need:
+1. To make your dataframe easier to work with, select only the columns you need, using the `loc` function which extracts from the original dataframe a group of rows (passed as first parameter) and columns (passed as second parameter). The expression `:` in the case below means "all rows".
```python
- new_columns = ['Package', 'Month', 'Low Price', 'High Price', 'Date']
- pumpkins = pumpkins.drop([c for c in pumpkins.columns if c not in new_columns], axis=1)
+ columns_to_select = ['Package', 'Low Price', 'High Price', 'Date']
+ pumpkins = pumpkins.loc[:, columns_to_select]
```
### Second, determine average price of pumpkin
diff --git a/2-Regression/2-Data/solution/notebook.ipynb b/2-Regression/2-Data/solution/notebook.ipynb
index 7c8ec0b6..3595485d 100644
--- a/2-Regression/2-Data/solution/notebook.ipynb
+++ b/2-Regression/2-Data/solution/notebook.ipynb
@@ -304,8 +304,8 @@
"source": [
"\n",
"# A set of new columns for a new dataframe. Filter out nonmatching columns\n",
- "new_columns = ['Package', 'Month', 'Low Price', 'High Price', 'Date']\n",
- "pumpkins = pumpkins.drop([c for c in pumpkins.columns if c not in new_columns], axis=1)\n",
+ "columns_to_select = ['Package', 'Low Price', 'High Price', 'Date']\n",
+ "pumpkins = pumpkins.loc[:, columns_to_select]\n",
"\n",
"# Get an average between low and high price for the base pumpkin price\n",
"price = (pumpkins['Low Price'] + pumpkins['High Price']) / 2\n",
@@ -412,7 +412,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
- "version": "3.8.9"
+ "version": "3.11.1"
},
"metadata": {
"interpreter": {
diff --git a/2-Regression/3-Linear/notebook.ipynb b/2-Regression/3-Linear/notebook.ipynb
index b01f1ee8..2902cce8 100644
--- a/2-Regression/3-Linear/notebook.ipynb
+++ b/2-Regression/3-Linear/notebook.ipynb
@@ -38,8 +38,8 @@
"source": [
"pumpkins = pumpkins[pumpkins['Package'].str.contains('bushel', case=True, regex=True)]\n",
"\n",
- "new_columns = ['Package', 'Variety', 'City Name', 'Month', 'Low Price', 'High Price', 'Date']\n",
- "pumpkins = pumpkins.drop([c for c in pumpkins.columns if c not in new_columns], axis=1)\n",
+ "columns_to_select = ['Package', 'Variety', 'City Name', 'Low Price', 'High Price', 'Date']\n",
+ "pumpkins = pumpkins.loc[:, columns_to_select]\n",
"\n",
"price = (pumpkins['Low Price'] + pumpkins['High Price']) / 2\n",
"\n",
diff --git a/2-Regression/4-Logistic/README.md b/2-Regression/4-Logistic/README.md
index 2385269f..01790853 100644
--- a/2-Regression/4-Logistic/README.md
+++ b/2-Regression/4-Logistic/README.md
@@ -1,7 +1,7 @@
# Logistic regression to predict categories
-
-> Infographic by [Dasani Madipalli](https://twitter.com/dasani_decoded)
+
+
## [Pre-lecture quiz](https://gray-sand-07a10f403.1.azurestaticapps.net/quiz/15/)
> ### [This lesson is available in R!](./solution/R/lesson_4-R.ipynb)
@@ -26,9 +26,9 @@ Let's build a logistic regression model to predict that, given some variables, _
## Define the question
-For our purposes, we will express this as a binary: 'Orange' or 'Not Orange'. There is also a 'striped' category in our dataset but there are few instances of it, so we will not use it. It disappears once we remove null values from the dataset, anyway.
+For our purposes, we will express this as a binary: 'White' or 'Not White'. There is also a 'striped' category in our dataset but there are few instances of it, so we will not use it. It disappears once we remove null values from the dataset, anyway.
-> 🎃 Fun fact, we sometimes call white pumpkins 'ghost' pumpkins. They aren't very easy to carve, so they aren't as popular as the orange ones but they are cool looking!
+> 🎃 Fun fact, we sometimes call white pumpkins 'ghost' pumpkins. They aren't very easy to carve, so they aren't as popular as the orange ones but they are cool looking! So we could also reformulate our question as: 'Ghost' or 'Not Ghost'. 👻
## About logistic regression
@@ -47,12 +47,7 @@ There are other types of logistic regression, including multinomial and ordinal:
- **Multinomial**, which involves having more than one category - "Orange, White, and Striped".
- **Ordinal**, which involves ordered categories, useful if we wanted to order our outcomes logically, like our pumpkins that are ordered by a finite number of sizes (mini,sm,med,lg,xl,xxl).
-
-> Infographic by [Dasani Madipalli](https://twitter.com/dasani_decoded)
-
-### It's still linear
-
-Even though this type of Regression is all about 'category predictions', it still works best when there is a clear linear relationship between the dependent variable (color) and the other independent variables (the rest of the dataset, like city name and size). It's good to get an idea of whether there is any linearity dividing these variables or not.
+
### Variables DO NOT have to correlate
@@ -71,78 +66,146 @@ First, clean the data a bit, dropping null values and selecting only some of the
1. Add the following code:
```python
- from sklearn.preprocessing import LabelEncoder
-
- new_columns = ['Color','Origin','Item Size','Variety','City Name','Package']
-
- new_pumpkins = pumpkins.drop([c for c in pumpkins.columns if c not in new_columns], axis=1)
-
- new_pumpkins.dropna(inplace=True)
-
- new_pumpkins = new_pumpkins.apply(LabelEncoder().fit_transform)
+
+ columns_to_select = ['City Name','Package','Variety', 'Origin','Item Size', 'Color']
+ pumpkins = full_pumpkins.loc[:, columns_to_select]
+
+ pumpkins.dropna(inplace=True)
```
You can always take a peek at your new dataframe:
```python
- new_pumpkins.info
+ pumpkins.info
```
-### Visualization - side-by-side grid
+### Visualization - categorical plot
By now you have loaded up the [starter notebook](./notebook.ipynb) with pumpkin data once again and cleaned it so as to preserve a dataset containing a few variables, including `Color`. Let's visualize the dataframe in the notebook using a different library: [Seaborn](https://seaborn.pydata.org/index.html), which is built on Matplotlib which we used earlier.
-Seaborn offers some neat ways to visualize your data. For example, you can compare distributions of the data for each point in a side-by-side grid.
+Seaborn offers some neat ways to visualize your data. For example, you can compare distributions of the data for each `Variety` and `Color` in a categorical plot.
-1. Create such a grid by instantiating a `PairGrid`, using our pumpkin data `new_pumpkins`, followed by calling `map()`:
+1. Create such a plot by using the `catplot` function, using our pumpkin data `pumpkins`, and specifying a color mapping for each pumpkin category (orange or white):
```python
import seaborn as sns
- g = sns.PairGrid(new_pumpkins)
- g.map(sns.scatterplot)
+ palette = {
+ 'ORANGE': 'orange',
+ 'WHITE': 'wheat',
+ }
+
+ sns.catplot(
+ data=pumpkins, y="Variety", hue="Color", kind="count",
+ palette=palette,
+ )
```
- 
+ 
- By observing data side-by-side, you can see how the Color data relates to the other columns.
+ By observing the data, you can see how the Color data relates to Variety.
- ✅ Given this scatterplot grid, what are some interesting explorations you can envision?
+ ✅ Given this categorical plot, what are some interesting explorations you can envision?
-### Use a swarm plot
+### Data pre-processing: feature and label encoding
+Our pumpkins dataset contains string values for all its columns. Working with categorical data is intuitive for humans but not for machines. Machine learning algorithms work well with numbers. There's why encoding is a very important step in the data pre-processing phase, since it enables to turn categorical data into numerical data, without losing any information. A good encoding leads to build a good model.
-Since Color is a binary category (Orange or Not), it's called 'categorical data' and needs 'a more [specialized approach](https://seaborn.pydata.org/tutorial/categorical.html?highlight=bar) to visualization'. There are other ways to visualize the relationship of this category with other variables.
+For feature encoding there are two main types of encoders:
-You can visualize variables side-by-side with Seaborn plots.
+1. Ordinal encoder: it suits well for ordinal variables, which are categorical variables where their data follows a logical ordering, like the `Item Size` column in our dataset. It creates a mapping such that each category is represented by a number, which is the order of the category in the column.
-1. Try a 'swarm' plot to show the distribution of values:
+ ```python
+ from sklearn.preprocessing import OrdinalEncoder
+
+ item_size_categories = [['sml', 'med', 'med-lge', 'lge', 'xlge', 'jbo', 'exjbo']]
+ ordinal_features = ['Item Size']
+ ordinal_encoder = OrdinalEncoder(categories=item_size_categories)
+ ```
+
+2. Categorical encoder: it suits well for nominal variables, which are categorical variables where their data does not follow a logical ordering, like all the features different from `Item Size` in our dataset. It is a one-hot encoding, which means that each category is represented by a binary column: the encoded variable is equal to 1 if the pumpkin belongs to that Variety and 0 otherwise.
```python
- sns.swarmplot(x="Color", y="Item Size", data=new_pumpkins)
+ from sklearn.preprocessing import OneHotEncoder
+
+ categorical_features = ['City Name', 'Package', 'Variety', 'Origin']
+ categorical_encoder = OneHotEncoder(sparse_output=False)
```
+Then, `ColumnTransformer` is used to combine multiple encoders into a single step and apply them to the appropriate columns.
+
+```python
+ from sklearn.compose import ColumnTransformer
+
+ ct = ColumnTransformer(transformers=[
+ ('ord', ordinal_encoder, ordinal_features),
+ ('cat', categorical_encoder, categorical_features)
+ ])
+
+ ct.set_output(transform='pandas')
+ encoded_features = ct.fit_transform(pumpkins)
+```
+On the other hand, to encode the label, we use the scikit-learn `LabelEncoder` class, which is a utility class to help normalize labels such that they contain only values between 0 and n_classes-1 (here, 0 and 1).
+
+```python
+ from sklearn.preprocessing import LabelEncoder
- 
+ label_encoder = LabelEncoder()
+ encoded_label = label_encoder.fit_transform(pumpkins['Color'])
+```
+Once we have encoded the features and the label, we can merge them into a new dataframe `encoded_pumpkins`.
-### Violin plot
+```python
+ encoded_pumpkins = encoded_features.assign(Color=encoded_label)
+```
+✅ What are the advantages of using an ordinal encoder for the `Item Size` column?
+
+### Analyse relationships between variables
-A 'violin' type plot is useful as you can easily visualize the way that data in the two categories is distributed. Violin plots don't work so well with smaller datasets as the distribution is displayed more 'smoothly'.
+Now that we have pre-processed our data, we can analyse the relationships between the features and the label to grasp an idea of how well the model will be able to predict the label given the features.
+The best way to perform this kind of analysis is plotting the data. We'll be using again the Seaborn `catplot` function, to visualize the relationships between `Item Size`, `Variety` and `Color` in a categorical plot. To better plot the data we'll be using the encoded `Item Size` column and the unencoded `Variety` column.
-1. As parameters `x=Color`, `kind="violin"` and call `catplot()`:
+```python
+ palette = {
+ 'ORANGE': 'orange',
+ 'WHITE': 'wheat',
+ }
+ pumpkins['Item Size'] = encoded_pumpkins['ord__Item Size']
+
+ g = sns.catplot(
+ data=pumpkins,
+ x="Item Size", y="Color", row='Variety',
+ kind="box", orient="h",
+ sharex=False, margin_titles=True,
+ height=1.8, aspect=4, palette=palette,
+ )
+ g.set(xlabel="Item Size", ylabel="").set(xlim=(0,6))
+ g.set_titles(row_template="{row_name}")
+```
+
+
+### Use a swarm plot
+
+Since Color is a binary category (White or Not), it needs 'a [specialized approach](https://seaborn.pydata.org/tutorial/categorical.html?highlight=bar) to visualization'. There are other ways to visualize the relationship of this category with other variables.
+
+You can visualize variables side-by-side with Seaborn plots.
+
+1. Try a 'swarm' plot to show the distribution of values:
```python
- sns.catplot(x="Color", y="Item Size",
- kind="violin", data=new_pumpkins)
+ palette = {
+ 0: 'orange',
+ 1: 'wheat'
+ }
+ sns.swarmplot(x="Color", y="ord__Item Size", data=encoded_pumpkins, palette=palette)
```
- 
+ 
- ✅ Try creating this plot, and other Seaborn plots, using other variables.
+**Watch Out**: the code above might generate a warning, since seaborn fails to represent such amount of datapoints into a swam plot. A possible solution is decreasing the size of the marker, by using the 'size' parameter. However, be aware that this affects the readability of the plot.
-Now that we have an idea of the relationship between the binary categories of color and the larger group of sizes, let's explore logistic regression to determine a given pumpkin's likely color.
> **🧮 Show Me The Math**
>
-> Remember how linear regression often used ordinary least squares to arrive at a value? Logistic regression relies on the concept of 'maximum likelihood' using [sigmoid functions](https://wikipedia.org/wiki/Sigmoid_function). A 'Sigmoid Function' on a plot looks like an 'S' shape. It takes a value and maps it to somewhere between 0 and 1. Its curve is also called a 'logistic curve'. Its formula looks like this:
+> Logistic regression relies on the concept of 'maximum likelihood' using [sigmoid functions](https://wikipedia.org/wiki/Sigmoid_function). A 'Sigmoid Function' on a plot looks like an 'S' shape. It takes a value and maps it to somewhere between 0 and 1. Its curve is also called a 'logistic curve'. Its formula looks like this:
>
> 
>
@@ -157,49 +220,47 @@ Building a model to find these binary classification is surprisingly straightfor
```python
from sklearn.model_selection import train_test_split
- Selected_features = ['Origin','Item Size','Variety','City Name','Package']
-
- X = new_pumpkins[Selected_features]
- y = new_pumpkins['Color']
-
+ X = encoded_pumpkins[encoded_pumpkins.columns.difference(['Color'])]
+ y = encoded_pumpkins['Color']
+
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=0)
```
-1. Now you can train your model, by calling `fit()` with your training data, and print out its result:
+2. Now you can train your model, by calling `fit()` with your training data, and print out its result:
```python
- from sklearn.model_selection import train_test_split
- from sklearn.metrics import accuracy_score, classification_report
+ from sklearn.metrics import f1_score, classification_report
from sklearn.linear_model import LogisticRegression
-
+
model = LogisticRegression()
model.fit(X_train, y_train)
predictions = model.predict(X_test)
-
+
print(classification_report(y_test, predictions))
print('Predicted labels: ', predictions)
- print('Accuracy: ', accuracy_score(y_test, predictions))
+ print('F1-score: ', f1_score(y_test, predictions))
```
- Take a look at your model's scoreboard. It's not too bad, considering you have only about 1000 rows of data:
+ Take a look at your model's scoreboard. It's not bad, considering you have only about 1000 rows of data:
```output
precision recall f1-score support
- 0 0.85 0.95 0.90 166
- 1 0.38 0.15 0.22 33
+ 0 0.94 0.98 0.96 166
+ 1 0.85 0.67 0.75 33
- accuracy 0.82 199
- macro avg 0.62 0.55 0.56 199
- weighted avg 0.77 0.82 0.78 199
+ accuracy 0.92 199
+ macro avg 0.89 0.82 0.85 199
+ weighted avg 0.92 0.92 0.92 199
- Predicted labels: [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0
- 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
- 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1
- 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1
- 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
- 0 0 0 1 0 1 0 0 1 0 0 0 1 0]
+ Predicted labels: [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0
+ 0 0 0 0 0 1 0 1 0 0 1 0 0 0 0 0 1 0 1 0 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0
+ 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0 0 0 1 0
+ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 1 1 0
+ 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1
+ 0 0 0 1 0 0 0 0 0 0 0 0 1 1]
+ F1-score: 0.7457627118644068
```
## Better comprehension via a confusion matrix
@@ -219,7 +280,7 @@ While you can get a scoreboard report [terms](https://scikit-learn.org/stable/mo
```output
array([[162, 4],
- [ 33, 0]])
+ [ 11, 22]])
```
In Scikit-learn, confusion matrices Rows (axis 0) are actual labels and columns (axis 1) are predicted labels.
@@ -229,22 +290,22 @@ In Scikit-learn, confusion matrices Rows (axis 0) are actual labels and columns
| 0 | TN | FP |
| 1 | FN | TP |
-What's going on here? Let's say our model is asked to classify pumpkins between two binary categories, category 'orange' and category 'not-orange'.
+What's going on here? Let's say our model is asked to classify pumpkins between two binary categories, category 'white' and category 'not-white'.
-- If your model predicts a pumpkin as not orange and it belongs to category 'not-orange' in reality we call it a true negative, shown by the top left number.
-- If your model predicts a pumpkin as orange and it belongs to category 'not-orange' in reality we call it a false negative, shown by the bottom left number.
-- If your model predicts a pumpkin as not orange and it belongs to category 'orange' in reality we call it a false positive, shown by the top right number.
-- If your model predicts a pumpkin as orange and it belongs to category 'orange' in reality we call it a true positive, shown by the bottom right number.
+- If your model predicts a pumpkin as not white and it belongs to category 'not-white' in reality we call it a true negative, shown by the top left number.
+- If your model predicts a pumpkin as white and it belongs to category 'not-white' in reality we call it a false negative, shown by the bottom left number.
+- If your model predicts a pumpkin as not white and it belongs to category 'white' in reality we call it a false positive, shown by the top right number.
+- If your model predicts a pumpkin as white and it belongs to category 'white' in reality we call it a true positive, shown by the bottom right number.
As you might have guessed it's preferable to have a larger number of true positives and true negatives and a lower number of false positives and false negatives, which implies that the model performs better.
-How does the confusion matrix relate to precision and recall? Remember, the classification report printed above showed precision (0.83) and recall (0.98).
+How does the confusion matrix relate to precision and recall? Remember, the classification report printed above showed precision (0.85) and recall (0.67).
-Precision = tp / (tp + fp) = 162 / (162 + 33) = 0.8307692307692308
+Precision = tp / (tp + fp) = 22 / (22 + 4) = 0.8461538461538461
-Recall = tp / (tp + fn) = 162 / (162 + 4) = 0.9759036144578314
+Recall = tp / (tp + fn) = 22 / (22 + 11) = 0.6666666666666666
-✅ Q: According to the confusion matrix, how did the model do? A: Not too bad; there are a good number of true negatives but also several false negatives.
+✅ Q: According to the confusion matrix, how did the model do? A: Not bad; there are a good number of true negatives but also a few false negatives.
Let's revisit the terms we saw earlier with the help of the confusion matrix's mapping of TP/TN and FP/FN:
@@ -266,22 +327,28 @@ Let's revisit the terms we saw earlier with the help of the confusion matrix's m
## Visualize the ROC curve of this model
-This is not a bad model; its accuracy is in the 80% range so ideally you could use it to predict the color of a pumpkin given a set of variables.
-
-Let's do one more visualization to see the so-called 'ROC' score:
+Let's do one more visualization to see the so-called 'ROC' curve:
```python
from sklearn.metrics import roc_curve, roc_auc_score
+import matplotlib
+import matplotlib.pyplot as plt
+%matplotlib inline
y_scores = model.predict_proba(X_test)
-# calculate ROC curve
fpr, tpr, thresholds = roc_curve(y_test, y_scores[:,1])
-sns.lineplot(([0, 1], [0, 1]))
-sns.lineplot((fpr, tpr))
+
+fig = plt.figure(figsize=(6, 6))
+plt.plot([0, 1], [0, 1], 'k--')
+plt.plot(fpr, tpr)
+plt.xlabel('False Positive Rate')
+plt.ylabel('True Positive Rate')
+plt.title('ROC Curve')
+plt.show()
```
-Using Seaborn again, plot the model's [Receiving Operating Characteristic](https://scikit-learn.org/stable/auto_examples/model_selection/plot_roc.html?highlight=roc) or ROC. ROC curves are often used to get a view of the output of a classifier in terms of its true vs. false positives. "ROC curves typically feature true positive rate on the Y axis, and false positive rate on the X axis." Thus, the steepness of the curve and the space between the midpoint line and the curve matter: you want a curve that quickly heads up and over the line. In our case, there are false positives to start with, and then the line heads up and over properly:
+Using Matplotlib, plot the model's [Receiving Operating Characteristic](https://scikit-learn.org/stable/auto_examples/model_selection/plot_roc.html?highlight=roc) or ROC. ROC curves are often used to get a view of the output of a classifier in terms of its true vs. false positives. "ROC curves typically feature true positive rate on the Y axis, and false positive rate on the X axis." Thus, the steepness of the curve and the space between the midpoint line and the curve matter: you want a curve that quickly heads up and over the line. In our case, there are false positives to start with, and then the line heads up and over properly:
-
+
Finally, use Scikit-learn's [`roc_auc_score` API](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.roc_auc_score.html?highlight=roc_auc#sklearn.metrics.roc_auc_score) to compute the actual 'Area Under the Curve' (AUC):
@@ -289,7 +356,7 @@ Finally, use Scikit-learn's [`roc_auc_score` API](https://scikit-learn.org/stabl
auc = roc_auc_score(y_test,y_scores[:,1])
print(auc)
```
-The result is `0.6976998904709748`. Given that the AUC ranges from 0 to 1, you want a big score, since a model that is 100% correct in its predictions will have an AUC of 1; in this case, the model is _pretty good_.
+The result is `0.9749908725812341`. Given that the AUC ranges from 0 to 1, you want a big score, since a model that is 100% correct in its predictions will have an AUC of 1; in this case, the model is _pretty good_.
In future lessons on classifications, you will learn how to iterate to improve your model's scores. But for now, congratulations! You've completed these regression lessons!
diff --git a/2-Regression/4-Logistic/images/ROC_2.png b/2-Regression/4-Logistic/images/ROC_2.png
new file mode 100644
index 00000000..aa629fb1
Binary files /dev/null and b/2-Regression/4-Logistic/images/ROC_2.png differ
diff --git a/2-Regression/4-Logistic/images/linear-vs-logistic.png b/2-Regression/4-Logistic/images/linear-vs-logistic.png
new file mode 100644
index 00000000..0cfe2209
Binary files /dev/null and b/2-Regression/4-Logistic/images/linear-vs-logistic.png differ
diff --git a/2-Regression/4-Logistic/images/multinomial-vs-ordinal.png b/2-Regression/4-Logistic/images/multinomial-vs-ordinal.png
new file mode 100644
index 00000000..1d4fb16f
Binary files /dev/null and b/2-Regression/4-Logistic/images/multinomial-vs-ordinal.png differ
diff --git a/2-Regression/4-Logistic/images/pumpkins_catplot_1.png b/2-Regression/4-Logistic/images/pumpkins_catplot_1.png
new file mode 100644
index 00000000..1c646732
Binary files /dev/null and b/2-Regression/4-Logistic/images/pumpkins_catplot_1.png differ
diff --git a/2-Regression/4-Logistic/images/pumpkins_catplot_2.png b/2-Regression/4-Logistic/images/pumpkins_catplot_2.png
new file mode 100644
index 00000000..fd4112bc
Binary files /dev/null and b/2-Regression/4-Logistic/images/pumpkins_catplot_2.png differ
diff --git a/2-Regression/4-Logistic/images/swarm_2.png b/2-Regression/4-Logistic/images/swarm_2.png
new file mode 100644
index 00000000..b44e7c71
Binary files /dev/null and b/2-Regression/4-Logistic/images/swarm_2.png differ
diff --git a/2-Regression/4-Logistic/notebook.ipynb b/2-Regression/4-Logistic/notebook.ipynb
index c151ea78..7c212763 100644
--- a/2-Regression/4-Logistic/notebook.ipynb
+++ b/2-Regression/4-Logistic/notebook.ipynb
@@ -1,41 +1,15 @@
{
- "metadata": {
- "language_info": {
- "codemirror_mode": {
- "name": "ipython",
- "version": 3
- },
- "file_extension": ".py",
- "mimetype": "text/x-python",
- "name": "python",
- "nbconvert_exporter": "python",
- "pygments_lexer": "ipython3",
- "version": "3.7.0"
- },
- "orig_nbformat": 2,
- "kernelspec": {
- "name": "python37364bit8d3b438fb5fc4430a93ac2cb74d693a7",
- "display_name": "Python 3.7.0 64-bit ('3.7')"
- },
- "metadata": {
- "interpreter": {
- "hash": "70b38d7a306a849643e446cd70466270a13445e5987dfa1344ef2b127438fa4d"
- }
- }
- },
- "nbformat": 4,
- "nbformat_minor": 2,
"cells": [
{
+ "cell_type": "markdown",
+ "metadata": {},
"source": [
"## Pumpkin Varieties and Color\n",
"\n",
"Load up required libraries and dataset. Convert the data to a dataframe containing a subset of the data: \n",
"\n",
"Let's look at the relationship between color and variety"
- ],
- "cell_type": "markdown",
- "metadata": {}
+ ]
},
{
"cell_type": "code",
@@ -43,8 +17,175 @@
"metadata": {},
"outputs": [
{
- "output_type": "execute_result",
"data": {
+ "text/html": [
+ "
\n",
+ "\n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
\n",
+ "
City Name
\n",
+ "
Type
\n",
+ "
Package
\n",
+ "
Variety
\n",
+ "
Sub Variety
\n",
+ "
Grade
\n",
+ "
Date
\n",
+ "
Low Price
\n",
+ "
High Price
\n",
+ "
Mostly Low
\n",
+ "
...
\n",
+ "
Unit of Sale
\n",
+ "
Quality
\n",
+ "
Condition
\n",
+ "
Appearance
\n",
+ "
Storage
\n",
+ "
Crop
\n",
+ "
Repack
\n",
+ "
Trans Mode
\n",
+ "
Unnamed: 24
\n",
+ "
Unnamed: 25
\n",
+ "
\n",
+ " \n",
+ " \n",
+ "
\n",
+ "
0
\n",
+ "
BALTIMORE
\n",
+ "
NaN
\n",
+ "
24 inch bins
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
4/29/17
\n",
+ "
270.0
\n",
+ "
280.0
\n",
+ "
270.0
\n",
+ "
...
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
E
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
\n",
+ "
\n",
+ "
1
\n",
+ "
BALTIMORE
\n",
+ "
NaN
\n",
+ "
24 inch bins
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
5/6/17
\n",
+ "
270.0
\n",
+ "
280.0
\n",
+ "
270.0
\n",
+ "
...
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
E
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
\n",
+ "
\n",
+ "
2
\n",
+ "
BALTIMORE
\n",
+ "
NaN
\n",
+ "
24 inch bins
\n",
+ "
HOWDEN TYPE
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
9/24/16
\n",
+ "
160.0
\n",
+ "
160.0
\n",
+ "
160.0
\n",
+ "
...
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
N
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
\n",
+ "
\n",
+ "
3
\n",
+ "
BALTIMORE
\n",
+ "
NaN
\n",
+ "
24 inch bins
\n",
+ "
HOWDEN TYPE
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
9/24/16
\n",
+ "
160.0
\n",
+ "
160.0
\n",
+ "
160.0
\n",
+ "
...
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
N
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
\n",
+ "
\n",
+ "
4
\n",
+ "
BALTIMORE
\n",
+ "
NaN
\n",
+ "
24 inch bins
\n",
+ "
HOWDEN TYPE
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
11/5/16
\n",
+ "
90.0
\n",
+ "
100.0
\n",
+ "
90.0
\n",
+ "
...
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
N
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
5 rows × 26 columns
\n",
+ "
"
+ ],
"text/plain": [
" City Name Type Package Variety Sub Variety Grade Date \\\n",
"0 BALTIMORE NaN 24 inch bins NaN NaN NaN 4/29/17 \n",
@@ -68,28 +209,48 @@
"4 NaN NaN NaN N NaN NaN NaN \n",
"\n",
"[5 rows x 26 columns]"
- ],
- "text/html": "
\n\n
\n \n
\n
\n
City Name
\n
Type
\n
Package
\n
Variety
\n
Sub Variety
\n
Grade
\n
Date
\n
Low Price
\n
High Price
\n
Mostly Low
\n
...
\n
Unit of Sale
\n
Quality
\n
Condition
\n
Appearance
\n
Storage
\n
Crop
\n
Repack
\n
Trans Mode
\n
Unnamed: 24
\n
Unnamed: 25
\n
\n \n \n
\n
0
\n
BALTIMORE
\n
NaN
\n
24 inch bins
\n
NaN
\n
NaN
\n
NaN
\n
4/29/17
\n
270.0
\n
280.0
\n
270.0
\n
...
\n
NaN
\n
NaN
\n
NaN
\n
NaN
\n
NaN
\n
NaN
\n
E
\n
NaN
\n
NaN
\n
NaN
\n
\n
\n
1
\n
BALTIMORE
\n
NaN
\n
24 inch bins
\n
NaN
\n
NaN
\n
NaN
\n
5/6/17
\n
270.0
\n
280.0
\n
270.0
\n
...
\n
NaN
\n
NaN
\n
NaN
\n
NaN
\n
NaN
\n
NaN
\n
E
\n
NaN
\n
NaN
\n
NaN
\n
\n
\n
2
\n
BALTIMORE
\n
NaN
\n
24 inch bins
\n
HOWDEN TYPE
\n
NaN
\n
NaN
\n
9/24/16
\n
160.0
\n
160.0
\n
160.0
\n
...
\n
NaN
\n
NaN
\n
NaN
\n
NaN
\n
NaN
\n
NaN
\n
N
\n
NaN
\n
NaN
\n
NaN
\n
\n
\n
3
\n
BALTIMORE
\n
NaN
\n
24 inch bins
\n
HOWDEN TYPE
\n
NaN
\n
NaN
\n
9/24/16
\n
160.0
\n
160.0
\n
160.0
\n
...
\n
NaN
\n
NaN
\n
NaN
\n
NaN
\n
NaN
\n
NaN
\n
N
\n
NaN
\n
NaN
\n
NaN
\n
\n
\n
4
\n
BALTIMORE
\n
NaN
\n
24 inch bins
\n
HOWDEN TYPE
\n
NaN
\n
NaN
\n
11/5/16
\n
90.0
\n
100.0
\n
90.0
\n
...
\n
NaN
\n
NaN
\n
NaN
\n
NaN
\n
NaN
\n
NaN
\n
N
\n
NaN
\n
NaN
\n
NaN
\n
\n \n
\n
5 rows × 26 columns
\n
"
+ ]
},
+ "execution_count": 1,
"metadata": {},
- "execution_count": 1
+ "output_type": "execute_result"
}
],
"source": [
"import pandas as pd\n",
"import numpy as np\n",
"\n",
- "pumpkins = pd.read_csv('../data/US-pumpkins.csv')\n",
+ "full_pumpkins = pd.read_csv('../data/US-pumpkins.csv')\n",
"\n",
- "pumpkins.head()\n"
+ "full_pumpkins.head()\n"
]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": []
}
- ]
-}
\ No newline at end of file
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.11.1"
+ },
+ "metadata": {
+ "interpreter": {
+ "hash": "70b38d7a306a849643e446cd70466270a13445e5987dfa1344ef2b127438fa4d"
+ }
+ },
+ "orig_nbformat": 2
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}
diff --git a/2-Regression/4-Logistic/solution/notebook.ipynb b/2-Regression/4-Logistic/solution/notebook.ipynb
index be24dd5a..0fdceae9 100644
--- a/2-Regression/4-Logistic/solution/notebook.ipynb
+++ b/2-Regression/4-Logistic/solution/notebook.ipynb
@@ -6,27 +6,194 @@
"source": [
"## Logistic Regression - Lesson 4\n",
"\n",
- "Load up required libraries and dataset. Convert the data to a dataframe containing a subset of the data"
+ "Load up required libraries and dataset. Convert the data to a dataframe containing a subset of the data:"
]
},
{
"cell_type": "code",
- "execution_count": 1,
+ "execution_count": 63,
"metadata": {},
"outputs": [
{
- "output_type": "execute_result",
"data": {
+ "text/html": [
+ "
\n",
+ "\n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
\n",
+ "
City Name
\n",
+ "
Type
\n",
+ "
Package
\n",
+ "
Variety
\n",
+ "
Sub Variety
\n",
+ "
Grade
\n",
+ "
Date
\n",
+ "
Low Price
\n",
+ "
High Price
\n",
+ "
Mostly Low
\n",
+ "
...
\n",
+ "
Unit of Sale
\n",
+ "
Quality
\n",
+ "
Condition
\n",
+ "
Appearance
\n",
+ "
Storage
\n",
+ "
Crop
\n",
+ "
Repack
\n",
+ "
Trans Mode
\n",
+ "
Unnamed: 24
\n",
+ "
Unnamed: 25
\n",
+ "
\n",
+ " \n",
+ " \n",
+ "
\n",
+ "
0
\n",
+ "
BALTIMORE
\n",
+ "
NaN
\n",
+ "
24 inch bins
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
4/29/17
\n",
+ "
270.0
\n",
+ "
280.0
\n",
+ "
270.0
\n",
+ "
...
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
E
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
\n",
+ "
\n",
+ "
1
\n",
+ "
BALTIMORE
\n",
+ "
NaN
\n",
+ "
24 inch bins
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
5/6/17
\n",
+ "
270.0
\n",
+ "
280.0
\n",
+ "
270.0
\n",
+ "
...
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
E
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
\n",
+ "
\n",
+ "
2
\n",
+ "
BALTIMORE
\n",
+ "
NaN
\n",
+ "
24 inch bins
\n",
+ "
HOWDEN TYPE
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
9/24/16
\n",
+ "
160.0
\n",
+ "
160.0
\n",
+ "
160.0
\n",
+ "
...
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
N
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
\n",
+ "
\n",
+ "
3
\n",
+ "
BALTIMORE
\n",
+ "
NaN
\n",
+ "
24 inch bins
\n",
+ "
HOWDEN TYPE
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
9/24/16
\n",
+ "
160.0
\n",
+ "
160.0
\n",
+ "
160.0
\n",
+ "
...
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
N
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
\n",
+ "
\n",
+ "
4
\n",
+ "
BALTIMORE
\n",
+ "
NaN
\n",
+ "
24 inch bins
\n",
+ "
HOWDEN TYPE
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
11/5/16
\n",
+ "
90.0
\n",
+ "
100.0
\n",
+ "
90.0
\n",
+ "
...
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
N
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
NaN
\n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
5 rows × 26 columns
\n",
+ "
"
+ ],
"text/plain": [
- " City Name Type Package Variety Sub Variety Grade Date \\\n",
- "0 BALTIMORE NaN 24 inch bins NaN NaN NaN 4/29/17 \n",
+ " City Name Type Package Variety Sub Variety Grade Date \n",
+ "0 BALTIMORE NaN 24 inch bins NaN NaN NaN 4/29/17 \\\n",
"1 BALTIMORE NaN 24 inch bins NaN NaN NaN 5/6/17 \n",
"2 BALTIMORE NaN 24 inch bins HOWDEN TYPE NaN NaN 9/24/16 \n",
"3 BALTIMORE NaN 24 inch bins HOWDEN TYPE NaN NaN 9/24/16 \n",
"4 BALTIMORE NaN 24 inch bins HOWDEN TYPE NaN NaN 11/5/16 \n",
"\n",
- " Low Price High Price Mostly Low ... Unit of Sale Quality Condition \\\n",
- "0 270.0 280.0 270.0 ... NaN NaN NaN \n",
+ " Low Price High Price Mostly Low ... Unit of Sale Quality Condition \n",
+ "0 270.0 280.0 270.0 ... NaN NaN NaN \\\n",
"1 270.0 280.0 270.0 ... NaN NaN NaN \n",
"2 160.0 160.0 160.0 ... NaN NaN NaN \n",
"3 160.0 160.0 160.0 ... NaN NaN NaN \n",
@@ -40,266 +207,947 @@
"4 NaN NaN NaN N NaN NaN NaN \n",
"\n",
"[5 rows x 26 columns]"
- ],
- "text/html": "