Pytorch深度学习实践笔记5（b站刘二大人）

本文主要是介绍Pytorch深度学习实践笔记5（b站刘二大人），希望对大家解决编程问题提供一定的参考价值，需要的开发者们随着小编来一起学习吧！

🎬个人简介：一个全栈工程师的升级之路！
📋个人专栏：pytorch深度学习
🎀CSDN主页发狂的小花
🌄人生秘诀：学习的本质就是极致重复!

视频来自【b站刘二大人】

1 Linear Regression

2 Dataloader 数据读取机制

3 代码

1 Linear Regression

使用Pytorch实现，步骤如下：
PyTorch Fashion(风格)

prepare dataset
design model using Class ，前向传播，计算y_pred
Construct loss and optimizer，计算loss，Optimizer 更新w
Training cycle (forward,backward,update)

2 Dataloader 数据读取机制

Pytorch数据读取机制

一文搞懂Pytorch数据读取机制！_pytorch的batch读取数据-CSDN博客

小批量数据读取

import torch  
import torch.utils.data as Data  BATCH_SIZE = 3x_data = torch.tensor([[1.0],[2.0],[3.0],[4.0],[5.0],[6.0],[7.0],[8.0],[9.0]])
y_data = torch.tensor([[2.0],[4.0],[6.0],[8.0],[10.0],[12.0],[14.0],[16.0],[18.0]])dataset = Data.TensorDataset(x_data,y_data)loader = Data.DataLoader(  dataset=dataset,  batch_size=BATCH_SIZE,  shuffle=True,  num_workers=0  
)for epoch in range(3):  for step, (batch_x, batch_y) in enumerate(loader):  print('epoch', epoch,  '| step:', step,  '| batch_x', batch_x,  '| batch_y:', batch_y)

3 代码

import torch
import torch.utils.data as Data 
import matplotlib.pyplot as plt 
# prepare datasetBATCH_SIZE = 3epoch_list = []
loss_list = []x_data = torch.tensor([[1.0],[2.0],[3.0],[4.0],[5.0],[6.0],[7.0],[8.0],[9.0]])
y_data = torch.tensor([[2.0],[4.0],[6.0],[8.0],[10.0],[12.0],[14.0],[16.0],[18.0]])dataset = Data.TensorDataset(x_data,y_data)loader = Data.DataLoader(  dataset=dataset,  batch_size=BATCH_SIZE,  shuffle=True,  num_workers=0  
)#design model using class
"""
our model class should be inherit from nn.Module, which is base class for all neural network modules.
member methods __init__() and forward() have to be implemented
class nn.linear contain two member Tensors: weight and bias
class nn.Linear has implemented the magic method __call__(),which enable the instance of the class can
be called just like a function.Normally the forward() will be called 
"""
class LinearModel(torch.nn.Module):def __init__(self):super(LinearModel, self).__init__()# (1,1)是指输入x和输出y的特征维度，这里数据集中的x和y的特征都是1维的# 该线性层需要学习的参数是w和b  获取w/b的方式分别是~linear.weight/linear.biasself.linear = torch.nn.Linear(1, 1)def forward(self, x):y_pred = self.linear(x)return y_predmodel = LinearModel()# construct loss and optimizer
# criterion = torch.nn.MSELoss(size_average = False)
criterion = torch.nn.MSELoss(reduction = 'sum')
optimizer = torch.optim.SGD(model.parameters(), lr = 0.01) # training cycle forward, backward, update
for epoch in range(1000):  for iteration, (batch_x, batch_y) in enumerate(loader):  y_pred = model(batch_x) # forwardloss = criterion(y_pred, batch_y) # backward# print("epoch: ",epoch, " iteration: ",iteration," loss: ",loss.item())optimizer.zero_grad() # the grad computer by .backward() will be accumulated. so before backward, remember set the grad to zeroloss.backward() # backward: autograd，自动计算梯度optimizer.step() # update 参数，即更新w和b的值print("epoch: ",epoch, " loss: ",loss.item())epoch_list.append(epoch)loss_list.append(loss.data.item())if (loss.data.item() < 1e-7):print("Epoch: ",epoch+1,"loss is: ",loss.data.item(),"(w,b): ","(",model.linear.weight.item(),",",model.linear.bias.item(),")")breakprint('w = ', model.linear.weight.item())
print('b = ', model.linear.bias.item())x_test = torch.tensor([[10.0]])
y_test = model(x_test)
print('y_pred = ', y_test.data)plt.plot(epoch_list,loss_list)
plt.title("SGD")
plt.xlabel("epoch")
plt.ylabel("loss")
plt.savefig("./data/pytorch4.png")

几种不同的优化器对应的结果：

Pytorch优化器全总结（三）牛顿法、BFGS、L-BFGS 含代码

pytorch LBFGS_lbfgs优化器-CSDN博客

scg.step() missing 1 required positiona-CSDN博客

LFBGS 代码

import torch
import torch.utils.data as Data 
import matplotlib.pyplot as plt 
# prepare datasetBATCH_SIZE = 3epoch_list = []
loss_list = []x_data = torch.tensor([[1.0],[2.0],[3.0],[4.0],[5.0],[6.0],[7.0],[8.0],[9.0]])
y_data = torch.tensor([[2.0],[4.0],[6.0],[8.0],[10.0],[12.0],[14.0],[16.0],[18.0]])dataset = Data.TensorDataset(x_data,y_data)loader = Data.DataLoader(  dataset=dataset,  batch_size=BATCH_SIZE,  shuffle=True,  num_workers=0  
)#design model using class
"""
our model class should be inherit from nn.Module, which is base class for all neural network modules.
member methods __init__() and forward() have to be implemented
class nn.linear contain two member Tensors: weight and bias
class nn.Linear has implemented the magic method __call__(),which enable the instance of the class can
be called just like a function.Normally the forward() will be called 
"""
class LinearModel(torch.nn.Module):def __init__(self):super(LinearModel, self).__init__()# (1,1)是指输入x和输出y的特征维度，这里数据集中的x和y的特征都是1维的# 该线性层需要学习的参数是w和b  获取w/b的方式分别是~linear.weight/linear.biasself.linear = torch.nn.Linear(1, 1)def forward(self, x):y_pred = self.linear(x)return y_predmodel = LinearModel()# construct loss and optimizer
# criterion = torch.nn.MSELoss(size_average = False)
criterion = torch.nn.MSELoss(reduction = 'sum')
optimizer = torch.optim.LBFGS(model.parameters(), lr = 0.1) # model.parameters()自动完成参数的初始化操作，这个地方我可能理解错了loss = torch.Tensor([1000.])
# training cycle forward, backward, update
for epoch in range(1000):  for iteration, (batch_x, batch_y) in enumerate(loader):def closure():y_pred = model(batch_x) # forwardloss = criterion(y_pred, batch_y) # backward# print("epoch: ",epoch, " iteration: ",iteration," loss: ",loss.item())optimizer.zero_grad() # the grad computer by .backward() will be accumulated. so before backward, remember set the grad to zeroloss.backward() # backward: autograd，自动计算梯度return lossloss = closure()optimizer.step(closure) # update 参数，即更新w和b的值print("epoch: ",epoch, " loss: ",loss.item())epoch_list.append(epoch)loss_list.append(loss.data.item())if (loss.data.item() < 1e-7):print("Epoch: ",epoch+1,"loss is: ",loss.data.item(),"(w,b): ","(",model.linear.weight.item(),",",model.linear.bias.item(),")")breakprint('w = ', model.linear.weight.item())
print('b = ', model.linear.bias.item())x_test = torch.tensor([[10.0]])
y_test = model(x_test)
print('y_pred = ', y_test.data)plt.plot(epoch_list,loss_list)
plt.title("LBFGS(lr = 0.1)")
plt.xlabel("epoch")
plt.ylabel("loss")
plt.savefig("./data/pytorch4.png")

Rprop:

Rprop 优化方法(弹性反向传播)，适用于 full-batch，不适用于 mini-batch，因而在 mini-batch 大行其道的时代里，很少见到。
优点：它可以自动调节学习率，不需要人为调节
缺点：仍依赖于人工设置一个全局学习率,随着迭代次数增多，学习率会越来越小，最终会趋近于0
结果：修改学习率和epoch均不能使其表现良好，无法满足1e-7精度条件下收敛