Clarify missing height correlation example

copilot/update-notebook-for-learners
copilot-swe-agent[bot] 2 weeks ago committed by GitHub
parent 9f161b6d53
commit 999c2c4ad7
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194

@ -487,18 +487,18 @@
"metadata": {},
"outputs": [],
"source": [
"np.corrcoef(df['Height'].ffill(),df['Weight'])"
"np.corrcoef(df['Height'], df['Weight'])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Unfortunately, we did not get any results - only some strange `nan` values. This is due to the fact that some of the values in our series are undefined, represented as `nan`, which causes the result of the operation to be undefined as well. By looking at the matrix we can see that `Weight` is the problematic column, because self-correlation between `Height` values has been computed.\n",
"Unfortunately, we did not get a correlation result—only `nan` values involving `Height`. This happens because `Height` contains a missing value, represented as `nan`, so calculations that use that column are also undefined. The `1` in the bottom-right of the matrix is the self-correlation of `Weight`, which has no missing values.\n",
"\n",
"> This example shows the importance of **data preparation** and **cleaning**. Without proper data we cannot compute anything.\n",
"\n",
"Let's use `fillna` method to fill the missing values, and compute the correlation: "
"For this example, let's use `ffill` to replace the missing `Height` with the previous valid height, and then compute the correlation:"
]
},
{

Loading…
Cancel
Save