diff --git a/assets/1606031210722.png b/assets/1606031210722.png new file mode 100644 index 0000000..28267e8 Binary files /dev/null and b/assets/1606031210722.png differ diff --git a/assets/1606031429856.png b/assets/1606031429856.png new file mode 100644 index 0000000..4798022 Binary files /dev/null and b/assets/1606031429856.png differ diff --git a/assets/1606031463635.png b/assets/1606031463635.png new file mode 100644 index 0000000..84a8335 Binary files /dev/null and b/assets/1606031463635.png differ diff --git a/notebook_必备数学基础/方差分析/.ipynb_checkpoints/方差分析-checkpoint.ipynb b/notebook_必备数学基础/方差分析/.ipynb_checkpoints/方差分析-checkpoint.ipynb new file mode 100644 index 0000000..08cfdbb --- /dev/null +++ b/notebook_必备数学基础/方差分析/.ipynb_checkpoints/方差分析-checkpoint.ipynb @@ -0,0 +1,272 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 方差分析" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### 方差分析概述\n", + "检验多个总体均值是否相等,通过分析察数据的误差判断各总体均值是否相等\n", + "\n", + "下图,所有的样本都在一个相似的正态分布区间\n", + "\n", + "下图,所有的样本都是正态分布,但不在同一分布区间\n", + "" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "**实例:**\n", + "\n", + "为了对几个行业的服务消费者协会在四个行业分别抽取了不同的企业作为样本。最近一年中消费者对总共23家企业投诉的次数如下表\n", + "\n", + "**要做的事:**\n", + "\n", + "分析四个行业之间的服务质量是否有显著差异,也就是要判断“行业”对“投诉次数”是否有显著影响\n", + "\n", + "如果它们的均值相等,就意味着“行业”对投诉次数是没有影响的,即它们之间的服务质量没有显著差异:如果均值不全相等,则意味着“行业”对投诉次数是有影响的,它们之间的服务质量有显著差异" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "**相关概念:**\n", + "\n", + "\n", + "**散点图观察**\n", + "\n", + "\n", + "\n", + "**但是**\n", + "\n", + "\n", + "### 基本思想:\n", + "\n", + "\n", + "### 随机误差:\n", + "\n", + "\n", + "### 系统误差:\n", + "\n", + "\n", + "### 组内方差:\n", + "\n", + "\n", + "### 组间方差:\n", + "\n", + "\n", + "### 方差的比较:\n", + "" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 方差分析计算方法\n", + "\n", + "### 方差分析的前提:\n", + "\n", + "**每个总体都应服从正态分布**\n", + "\n", + "\n", + "**各个总体的方差必须相同**\n", + "\n", + "\n", + "**观察值是独立**\n", + "\n", + "\n", + "**在上述假定条件下,判断行业对投诉次数是否有显著影响,实际上也就是检验具有同方差的四个正态总体的均值是否相等**\n", + "\n", + "**原假设成立,即H0:μ1=μ2=μ3=μ4**\n", + "
四个行业被投诉次数的均值都相等意味着每个样本都来自均值为μ、方差为σ^2的同一正态总体\n", + "\n", + "**备择假设成立,即H1:μ1=μ2=μ3=μ4不完全相等**\n", + "
至少有一个总体的均值是不同的,四个样本分别来自均值不同的四个正态总体\n", + "" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### 单因素方差分析\n", + "模型中有一个自变量(因素)和一个观测变量其实就是关干在一个影响因素的不同水平下,观测变量均值差异的显著性检验。\n", + "\n", + "**提出假设**\n", + "
HO: μ1= μ2=...=μk,自变量对因变量没有显著影响\n", + "
即H1:μ1μ2...u4不完全相等,自变量对因变量有显著影响\n", + "\n", + "拒绝原假设,只表明至少有两个总体的均值不相等,并不意味着所有的均值都不相等\n", + "\n", + "#### 检验的统计量\n", + "\n", + "\n", + "**水平的均值:**\n", + "\n", + "定从第i个总体中抽取一个容量为ni的简单随机样本,第ⅰ个总体的样本均值为该样本的全部观察值总和除以观察值的个数\n", + " (i=1,2,...,k)\n", + "\n", + "式中:ni为第i个总体的样本观察值个数\n", + " xij为第i个总体的第j个观察值\n", + " \n", + "**全部观察值的总均值:**\n", + "\n", + "全部观察值的总和除以观察值的总个数\n", + "\n", + "式中:n=n1+n2+...+nk\n", + "" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "**总误差平方和SST**\n", + "\n", + "全部观察值与总平均值的离差平方和,反映全部观察值的离散状况\n", + "\n", + "\n", + "\n", + "**水平项平方和SSA**\n", + "\n", + "各组平均值与总平均值的离差平方和,反映各总体的样本均值之间的差异程度,又称组间平方和,该平方和既包括随机误差,也包括系统误差\n", + "\n", + "\n", + "**误差项平方和SSE**\n", + "\n", + "每个水平或组的各样本数据与其组平均值的离差平方和,反映每个样本各观察值的离散状况,又称组内平方和,该平方和反映的是随机误差的大小\n", + "\n", + "\n", + "**平方和之间的关系**\n", + "\n", + "总离差平方和(SST、误差项离差平方和(SSE)、水平项离差平方和(SSA)之间的关系\n", + "\n", + "\n", + "**SST反映全部数据总的误差程度;SSE反映随机误差的大小;SSA反映随机误差和系统误差的大小**\n", + "\n", + "如果原假设成立,则表明没有系统误差,组间平方和SSA除以自由度后的均方与组内平方和SSE和除以自由度后的均方差异就不会太大;如果组间均方显著地大于组内均方,说明各水平(总体)之间的差异不仅有随机误差,还有系统误差,判断因素的水平是否对其观察值有影响,实际上就是比较组间方差与组内方差之间差异的大小\n", + "\n", + "**均方MS**\n", + "\n", + "各误差平方和的大小与观察值的多少有关,为消除观察值多少对误差平方和大小的影响,需要将其平均,这就是均方,也称为方差,计算方法是用误差平方和除以相应的自由度\n", + "\n", + "**各自自由度**\n", + "\n", + "\n", + "**F统计量**\n", + "\n", + "将MSA(组间方差,SSA的均方,记为MSA)和MSE(组内方差,SSE的均方,记为MSE)进行对比,即得到所需要的检验统计量F\n", + "\n", + "\n", + "**F分布**\n", + "\n", + "\n", + "根据给定的显著性水平,在F分布表中查找与第一自由度df1=k-1、第二自由度df2=n-k相应的临界值\n", + "\n", + "\n", + "**方差分析表:**\n", + "" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.3" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/notebook_必备数学基础/方差分析/assets/1606031210722.png b/notebook_必备数学基础/方差分析/assets/1606031210722.png new file mode 100644 index 0000000..28267e8 Binary files /dev/null and b/notebook_必备数学基础/方差分析/assets/1606031210722.png differ diff --git a/notebook_必备数学基础/方差分析/assets/1606031429856.png b/notebook_必备数学基础/方差分析/assets/1606031429856.png new file mode 100644 index 0000000..4798022 Binary files /dev/null and b/notebook_必备数学基础/方差分析/assets/1606031429856.png differ diff --git a/notebook_必备数学基础/方差分析/assets/1606031463635.png b/notebook_必备数学基础/方差分析/assets/1606031463635.png new file mode 100644 index 0000000..84a8335 Binary files /dev/null and b/notebook_必备数学基础/方差分析/assets/1606031463635.png differ diff --git a/notebook_必备数学基础/方差分析/assets/20201122155647.png b/notebook_必备数学基础/方差分析/assets/20201122155647.png new file mode 100644 index 0000000..2f0c1f4 Binary files /dev/null and b/notebook_必备数学基础/方差分析/assets/20201122155647.png differ diff --git a/notebook_必备数学基础/方差分析/assets/20201122164855.png b/notebook_必备数学基础/方差分析/assets/20201122164855.png new file mode 100644 index 0000000..f469919 Binary files /dev/null and b/notebook_必备数学基础/方差分析/assets/20201122164855.png differ diff --git a/notebook_必备数学基础/方差分析/assets/20201122170827.png b/notebook_必备数学基础/方差分析/assets/20201122170827.png new file mode 100644 index 0000000..b55cf29 Binary files /dev/null and b/notebook_必备数学基础/方差分析/assets/20201122170827.png differ diff --git a/notebook_必备数学基础/方差分析/assets/20201122170905.png b/notebook_必备数学基础/方差分析/assets/20201122170905.png new file mode 100644 index 0000000..a6134c9 Binary files /dev/null and b/notebook_必备数学基础/方差分析/assets/20201122170905.png differ diff --git a/notebook_必备数学基础/方差分析/assets/20201122171118.png b/notebook_必备数学基础/方差分析/assets/20201122171118.png new file mode 100644 index 0000000..b05380f Binary files /dev/null and b/notebook_必备数学基础/方差分析/assets/20201122171118.png differ diff --git a/notebook_必备数学基础/方差分析/assets/20201122171813.png b/notebook_必备数学基础/方差分析/assets/20201122171813.png new file mode 100644 index 0000000..d8658c8 Binary files /dev/null and b/notebook_必备数学基础/方差分析/assets/20201122171813.png differ diff --git a/notebook_必备数学基础/方差分析/assets/20201122172021.png b/notebook_必备数学基础/方差分析/assets/20201122172021.png new file mode 100644 index 0000000..be76bb7 Binary files /dev/null and b/notebook_必备数学基础/方差分析/assets/20201122172021.png differ diff --git a/notebook_必备数学基础/方差分析/assets/20201122172125.png b/notebook_必备数学基础/方差分析/assets/20201122172125.png new file mode 100644 index 0000000..d86e07b Binary files /dev/null and b/notebook_必备数学基础/方差分析/assets/20201122172125.png differ diff --git a/notebook_必备数学基础/方差分析/assets/20201122172417.png b/notebook_必备数学基础/方差分析/assets/20201122172417.png new file mode 100644 index 0000000..b1989af Binary files /dev/null and b/notebook_必备数学基础/方差分析/assets/20201122172417.png differ diff --git a/notebook_必备数学基础/方差分析/assets/20201122172514.png b/notebook_必备数学基础/方差分析/assets/20201122172514.png new file mode 100644 index 0000000..3cac1ba Binary files /dev/null and b/notebook_必备数学基础/方差分析/assets/20201122172514.png differ diff --git a/notebook_必备数学基础/方差分析/assets/20201122172804.png b/notebook_必备数学基础/方差分析/assets/20201122172804.png new file mode 100644 index 0000000..999ac62 Binary files /dev/null and b/notebook_必备数学基础/方差分析/assets/20201122172804.png differ diff --git a/notebook_必备数学基础/方差分析/assets/20201122172952.png b/notebook_必备数学基础/方差分析/assets/20201122172952.png new file mode 100644 index 0000000..e178abe Binary files /dev/null and b/notebook_必备数学基础/方差分析/assets/20201122172952.png differ diff --git a/notebook_必备数学基础/方差分析/assets/20201122173416.png b/notebook_必备数学基础/方差分析/assets/20201122173416.png new file mode 100644 index 0000000..89486df Binary files /dev/null and b/notebook_必备数学基础/方差分析/assets/20201122173416.png differ diff --git a/notebook_必备数学基础/方差分析/assets/20201122173432.png b/notebook_必备数学基础/方差分析/assets/20201122173432.png new file mode 100644 index 0000000..f4d75e6 Binary files /dev/null and b/notebook_必备数学基础/方差分析/assets/20201122173432.png differ diff --git a/notebook_必备数学基础/方差分析/assets/20201122173507.png b/notebook_必备数学基础/方差分析/assets/20201122173507.png new file mode 100644 index 0000000..03d0ef4 Binary files /dev/null and b/notebook_必备数学基础/方差分析/assets/20201122173507.png differ diff --git a/notebook_必备数学基础/方差分析/assets/20201122173755.png b/notebook_必备数学基础/方差分析/assets/20201122173755.png new file mode 100644 index 0000000..9ecab33 Binary files /dev/null and b/notebook_必备数学基础/方差分析/assets/20201122173755.png differ diff --git a/notebook_必备数学基础/方差分析/方差分析.ipynb b/notebook_必备数学基础/方差分析/方差分析.ipynb new file mode 100644 index 0000000..08cfdbb --- /dev/null +++ b/notebook_必备数学基础/方差分析/方差分析.ipynb @@ -0,0 +1,272 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 方差分析" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### 方差分析概述\n", + "检验多个总体均值是否相等,通过分析察数据的误差判断各总体均值是否相等\n", + "\n", + "下图,所有的样本都在一个相似的正态分布区间\n", + "\n", + "下图,所有的样本都是正态分布,但不在同一分布区间\n", + "" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "**实例:**\n", + "\n", + "为了对几个行业的服务消费者协会在四个行业分别抽取了不同的企业作为样本。最近一年中消费者对总共23家企业投诉的次数如下表\n", + "\n", + "**要做的事:**\n", + "\n", + "分析四个行业之间的服务质量是否有显著差异,也就是要判断“行业”对“投诉次数”是否有显著影响\n", + "\n", + "如果它们的均值相等,就意味着“行业”对投诉次数是没有影响的,即它们之间的服务质量没有显著差异:如果均值不全相等,则意味着“行业”对投诉次数是有影响的,它们之间的服务质量有显著差异" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "**相关概念:**\n", + "\n", + "\n", + "**散点图观察**\n", + "\n", + "\n", + "\n", + "**但是**\n", + "\n", + "\n", + "### 基本思想:\n", + "\n", + "\n", + "### 随机误差:\n", + "\n", + "\n", + "### 系统误差:\n", + "\n", + "\n", + "### 组内方差:\n", + "\n", + "\n", + "### 组间方差:\n", + "\n", + "\n", + "### 方差的比较:\n", + "" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 方差分析计算方法\n", + "\n", + "### 方差分析的前提:\n", + "\n", + "**每个总体都应服从正态分布**\n", + "\n", + "\n", + "**各个总体的方差必须相同**\n", + "\n", + "\n", + "**观察值是独立**\n", + "\n", + "\n", + "**在上述假定条件下,判断行业对投诉次数是否有显著影响,实际上也就是检验具有同方差的四个正态总体的均值是否相等**\n", + "\n", + "**原假设成立,即H0:μ1=μ2=μ3=μ4**\n", + "
四个行业被投诉次数的均值都相等意味着每个样本都来自均值为μ、方差为σ^2的同一正态总体\n", + "\n", + "**备择假设成立,即H1:μ1=μ2=μ3=μ4不完全相等**\n", + "
至少有一个总体的均值是不同的,四个样本分别来自均值不同的四个正态总体\n", + "" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### 单因素方差分析\n", + "模型中有一个自变量(因素)和一个观测变量其实就是关干在一个影响因素的不同水平下,观测变量均值差异的显著性检验。\n", + "\n", + "**提出假设**\n", + "
HO: μ1= μ2=...=μk,自变量对因变量没有显著影响\n", + "
即H1:μ1μ2...u4不完全相等,自变量对因变量有显著影响\n", + "\n", + "拒绝原假设,只表明至少有两个总体的均值不相等,并不意味着所有的均值都不相等\n", + "\n", + "#### 检验的统计量\n", + "\n", + "\n", + "**水平的均值:**\n", + "\n", + "定从第i个总体中抽取一个容量为ni的简单随机样本,第ⅰ个总体的样本均值为该样本的全部观察值总和除以观察值的个数\n", + " (i=1,2,...,k)\n", + "\n", + "式中:ni为第i个总体的样本观察值个数\n", + " xij为第i个总体的第j个观察值\n", + " \n", + "**全部观察值的总均值:**\n", + "\n", + "全部观察值的总和除以观察值的总个数\n", + "\n", + "式中:n=n1+n2+...+nk\n", + "" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "**总误差平方和SST**\n", + "\n", + "全部观察值与总平均值的离差平方和,反映全部观察值的离散状况\n", + "\n", + "\n", + "\n", + "**水平项平方和SSA**\n", + "\n", + "各组平均值与总平均值的离差平方和,反映各总体的样本均值之间的差异程度,又称组间平方和,该平方和既包括随机误差,也包括系统误差\n", + "\n", + "\n", + "**误差项平方和SSE**\n", + "\n", + "每个水平或组的各样本数据与其组平均值的离差平方和,反映每个样本各观察值的离散状况,又称组内平方和,该平方和反映的是随机误差的大小\n", + "\n", + "\n", + "**平方和之间的关系**\n", + "\n", + "总离差平方和(SST、误差项离差平方和(SSE)、水平项离差平方和(SSA)之间的关系\n", + "\n", + "\n", + "**SST反映全部数据总的误差程度;SSE反映随机误差的大小;SSA反映随机误差和系统误差的大小**\n", + "\n", + "如果原假设成立,则表明没有系统误差,组间平方和SSA除以自由度后的均方与组内平方和SSE和除以自由度后的均方差异就不会太大;如果组间均方显著地大于组内均方,说明各水平(总体)之间的差异不仅有随机误差,还有系统误差,判断因素的水平是否对其观察值有影响,实际上就是比较组间方差与组内方差之间差异的大小\n", + "\n", + "**均方MS**\n", + "\n", + "各误差平方和的大小与观察值的多少有关,为消除观察值多少对误差平方和大小的影响,需要将其平均,这就是均方,也称为方差,计算方法是用误差平方和除以相应的自由度\n", + "\n", + "**各自自由度**\n", + "\n", + "\n", + "**F统计量**\n", + "\n", + "将MSA(组间方差,SSA的均方,记为MSA)和MSE(组内方差,SSE的均方,记为MSE)进行对比,即得到所需要的检验统计量F\n", + "\n", + "\n", + "**F分布**\n", + "\n", + "\n", + "根据给定的显著性水平,在F分布表中查找与第一自由度df1=k-1、第二自由度df2=n-k相应的临界值\n", + "\n", + "\n", + "**方差分析表:**\n", + "" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.3" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/必备数学基础.md b/必备数学基础.md index 0f0d5c0..d2cf37f 100644 --- a/必备数学基础.md +++ b/必备数学基础.md @@ -1203,3 +1203,28 @@ https://scikit-learn.org/stable/auto_examples/ ##### T检验的基本原理 + + +### 相关分析 + +notebook已更新,markdown待更新 + + + +### 方差分析 + +#### 方差分析概述 + +检验多个总体均值是否相等,通过分析察数据的误差判断各总体均值是否相等 + +![1606031210722](assets/1606031210722.png) + +下图,所有的样本都在一个相似的正态分布区间 + +![1606031429856](assets/1606031429856.png) + +下图,所有的样本都是正态分布,但不在同一分布区间 + +![1606031463635](assets/1606031463635.png) + +待更新,notebook已更新 \ No newline at end of file