python 相关性分析切点寻找,Python自然平滑样条线

2023-10-24 15:30

本文主要是介绍python 相关性分析切点寻找,Python自然平滑样条线,希望对大家解决编程问题提供一定的参考价值,需要的开发者们随着小编来一起学习吧!

I am trying to find a python package that would give an option to fit natural smoothing splines with user selectable smoothing factor. Is there an implementation for that? If not, how would you use what is available to implement it yourself?

By natural spline I mean that there should be a condition that the second derivative of the fitted function at the endpoints is zero (linear).

By smoothing spline I mean that the spline should not be 'interpolating' (passing through all the datapoints). I would like to decide the correct smoothing factor lambda (see the Wikipedia page for smoothing splines) myself.

What I have found

scipy.interpolate.CubicSpline [link]: Does natural (cubic) spline fitting. Does interpolation, and there is no way to smooth the data.

scipy.interpolate.UnivariateSpline [link]: Does spline fitting with user selectable smoothing factor. However, there is no option to make the splines natural.

解决方案

After hours of investigation, I did not find any pip installable packages which could fit a natural cubic spline with user-controllable smoothness. However, after deciding to write one myself, while reading about the topic I stumbled upon a blog post by github user madrury. He has written python code capable of producing natural cubic spline models.

The model code is available here (NaturalCubicSpline) with a BSD-licence. He has also written some examples in an IPython notebook.

But since this is the Internet and links tend to die, I will copy the relevant parts of the source code here + a helper function (get_natural_cubic_spline_model) written by me, and show an example of how to use it. The smoothness of the fit can be controlled by using different number of knots. The position of the knots can be also specified by the user.

Example

from matplotlib import pyplot as plt

import numpy as np

def func(x):

return 1/(1+25*x**2)

# make example data

x = np.linspace(-1,1,300)

y = func(x) + np.random.normal(0, 0.2, len(x))

# The number of knots can be used to control the amount of smoothness

model_6 = get_natural_cubic_spline_model(x, y, minval=min(x), maxval=max(x), n_knots=6)

model_15 = get_natural_cubic_spline_model(x, y, minval=min(x), maxval=max(x), n_knots=15)

y_est_6 = model_6.predict(x)

y_est_15 = model_15.predict(x)

plt.plot(x, y, ls='', marker='.', label='originals')

plt.plot(x, y_est_6, marker='.', label='n_knots = 6')

plt.plot(x, y_est_15, marker='.', label='n_knots = 15')

plt.legend(); plt.show()

3f3a5b82f015ba8333b227668f2b60af.png

The source code for get_natural_cubic_spline_model

import numpy as np

import pandas as pd

from sklearn.base import BaseEstimator, TransformerMixin

from sklearn.linear_model import LinearRegression

from sklearn.pipeline import Pipeline

def get_natural_cubic_spline_model(x, y, minval=None, maxval=None, n_knots=None, knots=None):

"""

Get a natural cubic spline model for the data.

For the knots, give (a) `knots` (as an array) or (b) minval, maxval and n_knots.

If the knots are not directly specified, the resulting knots are equally

space within the *interior* of (max, min). That is, the endpoints are

*not* included as knots.

Parameters

----------

x: np.array of float

The input data

y: np.array of float

The outpur data

minval: float

Minimum of interval containing the knots.

maxval: float

Maximum of the interval containing the knots.

n_knots: positive integer

The number of knots to create.

knots: array or list of floats

The knots.

Returns

--------

model: a model object

The returned model will have following method:

- predict(x):

x is a numpy array. This will return the predicted y-values.

"""

if knots:

spline = NaturalCubicSpline(knots=knots)

else:

spline = NaturalCubicSpline(max=maxval, min=minval, n_knots=n_knots)

p = Pipeline([

('nat_cubic', spline),

('regression', LinearRegression(fit_intercept=True))

])

p.fit(x, y)

return p

class AbstractSpline(BaseEstimator, TransformerMixin):

"""Base class for all spline basis expansions."""

def __init__(self, max=None, min=None, n_knots=None, n_params=None, knots=None):

if knots is None:

if not n_knots:

n_knots = self._compute_n_knots(n_params)

knots = np.linspace(min, max, num=(n_knots + 2))[1:-1]

max, min = np.max(knots), np.min(knots)

self.knots = np.asarray(knots)

@property

def n_knots(self):

return len(self.knots)

def fit(self, *args, **kwargs):

return self

class NaturalCubicSpline(AbstractSpline):

"""Apply a natural cubic basis expansion to an array.

The features created with this basis expansion can be used to fit a

piecewise cubic function under the constraint that the fitted curve is

linear *outside* the range of the knots.. The fitted curve is continuously

differentiable to the second order at all of the knots.

This transformer can be created in two ways:

- By specifying the maximum, minimum, and number of knots.

- By specifying the cutpoints directly.

If the knots are not directly specified, the resulting knots are equally

space within the *interior* of (max, min). That is, the endpoints are

*not* included as knots.

Parameters

----------

min: float

Minimum of interval containing the knots.

max: float

Maximum of the interval containing the knots.

n_knots: positive integer

The number of knots to create.

knots: array or list of floats

The knots.

"""

def _compute_n_knots(self, n_params):

return n_params

@property

def n_params(self):

return self.n_knots - 1

def transform(self, X, **transform_params):

X_spl = self._transform_array(X)

if isinstance(X, pd.Series):

col_names = self._make_names(X)

X_spl = pd.DataFrame(X_spl, columns=col_names, index=X.index)

return X_spl

def _make_names(self, X):

first_name = "{}_spline_linear".format(X.name)

rest_names = ["{}_spline_{}".format(X.name, idx)

for idx in range(self.n_knots - 2)]

return [first_name] + rest_names

def _transform_array(self, X, **transform_params):

X = X.squeeze()

try:

X_spl = np.zeros((X.shape[0], self.n_knots - 1))

except IndexError: # For arrays with only one element

X_spl = np.zeros((1, self.n_knots - 1))

X_spl[:, 0] = X.squeeze()

def d(knot_idx, x):

def ppart(t): return np.maximum(0, t)

def cube(t): return t*t*t

numerator = (cube(ppart(x - self.knots[knot_idx]))

- cube(ppart(x - self.knots[self.n_knots - 1])))

denominator = self.knots[self.n_knots - 1] - self.knots[knot_idx]

return numerator / denominator

for i in range(0, self.n_knots - 2):

X_spl[:, i+1] = (d(i, X) - d(self.n_knots - 2, X)).squeeze()

return X_spl

这篇关于python 相关性分析切点寻找,Python自然平滑样条线的文章就介绍到这儿,希望我们推荐的文章对编程师们有所帮助!


原文地址:https://blog.csdn.net/weixin_39625975/article/details/118907388
本文来自互联网用户投稿,该文观点仅代表作者本人,不代表本站立场。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如若转载,请注明出处:http://www.chinasem.cn/article/276122

相关文章

SpringBoot中六种批量更新Mysql的方式效率对比分析

《SpringBoot中六种批量更新Mysql的方式效率对比分析》文章比较了MySQL大数据量批量更新的多种方法,指出REPLACEINTO和ONDUPLICATEKEY效率最高但存在数据风险,MyB... 目录效率比较测试结构数据库初始化测试数据批量修改方案第一种 for第二种 case when第三种

python生成随机唯一id的几种实现方法

《python生成随机唯一id的几种实现方法》在Python中生成随机唯一ID有多种方法,根据不同的需求场景可以选择最适合的方案,文中通过示例代码介绍的非常详细,需要的朋友们下面随着小编来一起学习学习... 目录方法 1:使用 UUID 模块(推荐)方法 2:使用 Secrets 模块(安全敏感场景)方法

解决1093 - You can‘t specify target table报错问题及原因分析

《解决1093-Youcan‘tspecifytargettable报错问题及原因分析》MySQL1093错误因UPDATE/DELETE语句的FROM子句直接引用目标表或嵌套子查询导致,... 目录报js错原因分析具体原因解决办法方法一:使用临时表方法二:使用JOIN方法三:使用EXISTS示例总结报错原

使用Python删除Excel中的行列和单元格示例详解

《使用Python删除Excel中的行列和单元格示例详解》在处理Excel数据时,删除不需要的行、列或单元格是一项常见且必要的操作,本文将使用Python脚本实现对Excel表格的高效自动化处理,感兴... 目录开发环境准备使用 python 删除 Excphpel 表格中的行删除特定行删除空白行删除含指定

MySQL中的LENGTH()函数用法详解与实例分析

《MySQL中的LENGTH()函数用法详解与实例分析》MySQLLENGTH()函数用于计算字符串的字节长度,区别于CHAR_LENGTH()的字符长度,适用于多字节字符集(如UTF-8)的数据验证... 目录1. LENGTH()函数的基本语法2. LENGTH()函数的返回值2.1 示例1:计算字符串

Python通用唯一标识符模块uuid使用案例详解

《Python通用唯一标识符模块uuid使用案例详解》Pythonuuid模块用于生成128位全局唯一标识符,支持UUID1-5版本,适用于分布式系统、数据库主键等场景,需注意隐私、碰撞概率及存储优... 目录简介核心功能1. UUID版本2. UUID属性3. 命名空间使用场景1. 生成唯一标识符2. 数

Python办公自动化实战之打造智能邮件发送工具

《Python办公自动化实战之打造智能邮件发送工具》在数字化办公场景中,邮件自动化是提升工作效率的关键技能,本文将演示如何使用Python的smtplib和email库构建一个支持图文混排,多附件,多... 目录前言一、基础配置:搭建邮件发送框架1.1 邮箱服务准备1.2 核心库导入1.3 基础发送函数二、

Android kotlin中 Channel 和 Flow 的区别和选择使用场景分析

《Androidkotlin中Channel和Flow的区别和选择使用场景分析》Kotlin协程中,Flow是冷数据流,按需触发,适合响应式数据处理;Channel是热数据流,持续发送,支持... 目录一、基本概念界定FlowChannel二、核心特性对比数据生产触发条件生产与消费的关系背压处理机制生命周期

Python包管理工具pip的升级指南

《Python包管理工具pip的升级指南》本文全面探讨Python包管理工具pip的升级策略,从基础升级方法到高级技巧,涵盖不同操作系统环境下的最佳实践,我们将深入分析pip的工作原理,介绍多种升级方... 目录1. 背景介绍1.1 目的和范围1.2 预期读者1.3 文档结构概述1.4 术语表1.4.1 核

基于Python实现一个图片拆分工具

《基于Python实现一个图片拆分工具》这篇文章主要为大家详细介绍了如何基于Python实现一个图片拆分工具,可以根据需要的行数和列数进行拆分,感兴趣的小伙伴可以跟随小编一起学习一下... 简单介绍先自己选择输入的图片,默认是输出到项目文件夹中,可以自己选择其他的文件夹,选择需要拆分的行数和列数,可以通过