显著性检验python

本文主要是介绍显著性检验python，希望对大家解决编程问题提供一定的参考价值，需要的开发者们随着小编来一起学习吧！

Table of Contents

信用特征检验/模型稳健性检验的代码实现

目的:

(1)让大家掌握**区域/所有权等信用特征检验的方法
**

(2)让大家掌握$Fridman检验$和$Nemenyi检验$ 这两种常见的精度对比校验方法

代码: 师兄写了现成的信用特征检验Excel输出的代码。详见：https://github.com/AnyBrother/Significance_character_test_ykp

reference

.. [1] J. Demsar (2006), Statistical comparisons of classifiers overmultiple data sets, Journal of Machine Learning Research, 7, 1-30... [2] P. Nemenyi (1963) Distribution-free Multiple Comparisons. Ph.D.thesis, Princeton University... [3] L. Sachs (1997), Angewandte Statistik. Berlin: Springer.Pages: 668-675.

import pandas as pd
df=pd.read_excel("model_performance.xlsx", header=0, index_col=0)
df

	Model_1	Model_2	Model_3
dataset_1	0.45	0.85	0.95
dataset_2	0.67	0.87	0.97
dataset_3	0.46	0.86	0.96
dataset_4	0.56	0.86	0.96
dataset_5	0.47	0.87	0.97

　　 分析工作者常常用标准方法与自己所用的分析方法进行对照试验，然后用统计学方法检验两种结果是否存在显著性差异。若存在显著性差异而又肯定测定过程中没有错误，可以认定自己所用的方法有不完善之处，即存在较大的系统误差。

　　因此分析结果的差异需进行统计检验或显著性检验。

常用的检验实现

设第一个总体的均值为 $u_1$，第二个总体的均值为 $u_2$，则有：

**单侧检验：**有先验知识，一个是否比另一个好/差

1）Ho：$u_1$ ≤ $u_2$，H1：$u_1$ >$u_2$ if Z< -Za, 拒绝 Ho；

2）Ho：$u_1$ ≥ $u_2$，H1：$u_1$ < $u_2$ if Z> -Za, 拒绝 Ho；

**双侧检验：**两样本是否存在显著差异，常用

3）Ho： $u_1$ = $u_2$, H1：$u_1$ != $u_2$ if Z> -Za / 2，拒绝 Ho。

P值	碰巧的概率	对无效假设	统计意义
P＞0.1	碰巧出现的可能性大于5%	不能否定无效假设	两组差别无显著意义
P＜0.05	碰巧出现的可能性小于5%	可以否定无效假设	两组差别有显著意义
P ＜0.01	碰巧出现的可能性小于1%	可以否定无效假设	两者差别有非常显著意义

ttest_ind

Calculates the $T-test$ for the means of TWO INDEPENDENT samples of scores.

计算两个独立样本得分的平均值的T检验。

这是针对零假设（两个独立样本具有相同的平均（预期）值）的原边检验。 $假设两样本正态分布且具有相同的方差。$

from scipy import stats
statistic, pvalue=stats.mstats.ttest_ind(df["Model_1"],df["Model_2"])
print(statistic)
print(pvalue)

-8.086075400626394
4.042721798234637e-05

import numpy as np
np.random.seed(12345678)
#Test with sample with identical means:rvs1 = stats.norm.rvs(loc=5,scale=10,size=500)
rvs2 = stats.norm.rvs(loc=5,scale=10,size=400)
statistic, pvalue=stats.ttest_ind(rvs1,rvs2)
print(statistic)
print(pvalue)

0.4119830500614155
0.6804501671011296

曼-惠特尼U检验（Mann-Whitney U test）

$每组样本量必须大于20$

$H_0：u_1=u_2,H_1：u_1!=u_2$

$ if Z> -Za / 2，拒绝 H_0$。

group1=[28,31,36,35,32,33,21,12,12,23,19,13,20,17,14,19]
group2=[12,18,19,14,20,19,12,11,8,9,10,15,16,17,10,16]statistic, pvalue= stats.mannwhitneyu(group1, group2)
print(statistic)
print(pvalue)

46.5
0.001107347927116896

KS_检验

This tests whether 2 samples are drawn from the same distribution. Note that, like in the case of the one-sample K-S test, the distribution is assumed to be continuous.

The test uses the two-sided asymptotic $Kolmogorov-Smirnov$ distribution.

If the K-S statistic is small or the p-value is high, then we cannot reject the hypothesis that the distributions of the two samples are the same.

from scipy import stats
np.random.seed(12345678)  #fix random seed to get the same result
n1 = 200  # size of first sample
n2 = 300  # size of second sample
#For a different distribution, we can reject the null hypothesis since the pvalue is below 1%:rvs1 = stats.norm.rvs(size=n1, loc=0., scale=1)
rvs2 = stats.norm.rvs(size=n2, loc=0.5, scale=1.5)
statistic, pvalue=stats.ks_2samp(rvs1, rvs2)
print(statistic)
print(pvalue)

0.20833333333333334
5.129279597815284e-05

非参数统计Wald-Wolfowitz游程检验

$非参数统计Wald-Wolfowitz游程检验$

from statsmodels.sandbox.stats.runs import runstest_2samp
x=[104,253,300,308,315,323,331,396,414,452]
y=[184,196,197,248,260,279,355,386,393,432,450]
statistic, pvalue=runstest_2samp(x,y)
print(statistic)
print(pvalue)

-0.8870032598620701
0.37507714541523396

Wilcoxon rank-sum statistic

Compute the Wilcoxon rank-sum statistic for two samples.

$TheWilcoxonrank-sumtest$ tests the null hypothesis that two sets of measurements are drawn from the same distribution. The alternative hypothesis is that values in one sample are more likely to be larger than the values in the other sample.**

from scipy.stats import ranksums
sample1 = np.random.uniform(-1, 1, 200)
print(sample1[:10])
sample2 = np.random.uniform(-0.5, 1.5, 300) # a shifted distribution
print(sample2[:10])
statistic, pvalue=ranksums(sample1, sample2)
print(statistic)
print(pvalue)

[-0.57746919 -0.05972207  0.89157307 -0.47111938  0.21487712  0.21566889-0.09707397 -0.67379604 -0.77341795 -0.75565369]
[ 1.22562954 -0.02125675  0.79309106  0.36379193  0.9209503   0.82417966-0.06000881  0.69224626 -0.20661069 -0.08388529]
-8.42221423467549
3.694347239802868e-17

chi-square test

from scipy.stats import chi2
import numpy as npT = np.array([[36, 14], [30, 25]])
def chi2_get_p_value_sl(T):det = T[0,0]*T[1,1] - T[0,1]*T[1,0]c2 = float(det) / T[0].sum() * det / T[1].sum() * T.sum() / T[:,0].sum() / T[:,1].sum()p = 1 - chi2.cdf(x=c2, df=1)return p
chi2_get_p_value_sl(T)

0.06450186480705422

Fridman检验

Due to the assumption that the test statistic has a chi squared distribution, the p-value is only reliable for n > 10 and more than 6 repeated measurements.

FriedmanchisquareResult = stats.friedmanchisquare(df.iloc[:,0], df.iloc[:,1], df.iloc[:,2])
print('Friedmanchisquare Result: stat:{}, p-value:{}'.format(FriedmanchisquareResult[0], FriedmanchisquareResult[1]))

Friedmanchisquare Result: stat:10.0, p-value:0.006737946999085468

Nemenyi检验

说明: Fridman检验只能说明模型精度之间存在差别, 但不能说明那个模型更好。因此，需要Nemenyi检验进一步验证两两模型之间的精度是否$有显著差异$。

import scikit_posthocs as spresult=sp.posthoc_nemenyi_friedman(df)
print(result)
result.to_excel("result.xlsx")#结果输出到result.xlsx中

          Model_1   Model_2   Model_3
Model_1  1.000000  0.254114  0.004467
Model_2  0.254114  1.000000  0.254114
Model_3  0.004467  0.254114  1.000000

信用特征检验

# 运行这个代码框前需要将excel中的数据替换即可
import osos.system("python ./Significance_character_test_Regions.py")#区域的信用特征检验
os.system("python ./Significance_character_test_Provinces.py")#省份的信用特征检验
os.system("python ./Significance_character_test_Industries.py")#行业的信用特征检验
#所有权的信用特征检验

$好用就给个三连吧！！！$
$好用就给个三连吧！！！$
$好用就给个三连吧！！！$

这篇关于显著性检验python的文章就介绍到这儿，希望我们推荐的文章对编程师们有所帮助！

---