2.9-tf2-数据增强-tf_flowers

2023-11-07 19:51
文章标签 数据 tf 2.9 增强 flowers tf2

本文主要是介绍2.9-tf2-数据增强-tf_flowers,希望对大家解决编程问题提供一定的参考价值,需要的开发者们随着小编来一起学习吧!

文章目录

    • 1.导入包
    • 2.加载数据
    • 3.数据预处理
    • 4.数据增强
    • 5.预处理层的两种方法
    • 6.把与处理层用在数据集上
    • 7.训练模型
    • 8.自定义数据增强
    • 9.Using tf.image

tf_flowers数据集
在这里插入图片描述

1.导入包

import matplotlib.pyplot as plt
import numpy as np
import tensorflow as tf
import tensorflow_datasets as tfdsfrom tensorflow.keras import layers
from tensorflow.keras.datasets import mnist

2.加载数据

(train_ds, val_ds, test_ds), metadata = tfds.load('tf_flowers',split=['train[:80%]', 'train[80%:90%]', 'train[90%:]'],with_info=True,as_supervised=True,
)
#The flowers dataset has five classes.num_classes = metadata.features['label'].num_classes
print(num_classes)
5

复现数据:

#Let's retrieve an image from the dataset and use it to demonstrate data augmentation.get_label_name = metadata.features['label'].int2strimage, label = next(iter(train_ds))
_ = plt.imshow(image)
_ = plt.title(get_label_name(label))

在这里插入图片描述

3.数据预处理

Use Keras preprocessing layers

#Resizing and rescaling
#You can use preprocessing layers to resize your images to a consistent shape, and to rescale pixel values.IMG_SIZE = 180resize_and_rescale = tf.keras.Sequential([layers.experimental.preprocessing.Resizing(IMG_SIZE, IMG_SIZE),layers.experimental.preprocessing.Rescaling(1./255)
])
#Note: the rescaling layer above standardizes pixel values to [0,1]. If instead you wanted [-1,1], you would write Rescaling(1./127.5, offset=-1).
result = resize_and_rescale(image)
_ = plt.imshow(result)

在这里插入图片描述

#You can verify the pixels are in [0-1].print("Min and max pixel values:", result.numpy().min(), result.numpy().max())
Min and max pixel values: 0.0 1.0

4.数据增强

#Data augmentation
#You can use preprocessing layers for data augmentation as well.#Let's create a few preprocessing layers and apply them repeatedly to the same image.data_augmentation = tf.keras.Sequential([layers.experimental.preprocessing.RandomFlip("horizontal_and_vertical"),layers.experimental.preprocessing.RandomRotation(0.2),
])
# Add the image to a batch
image = tf.expand_dims(image, 0)
plt.figure(figsize=(10, 10))
for i in range(9):augmented_image = data_augmentation(image)ax = plt.subplot(3, 3, i + 1)plt.imshow(augmented_image[0])plt.axis("off")

在这里插入图片描述

#There are a variety of preprocessing layers you can use for data augmentation including layers.RandomContrast, layers.RandomCrop, layers.RandomZoom, and others.

5.预处理层的两种方法

There are two ways you can use these preprocessing layers, with important tradeoffs.

  1. 第一种方法
Option 1: Make the preprocessing layers part of your model
model = tf.keras.Sequential([resize_and_rescale,data_augmentation,layers.Conv2D(16, 3, padding='same', activation='relu'),layers.MaxPooling2D(),# Rest of your model
])
There are two important points to be aware of in this case:Data augmentation will run on-device, synchronously with the rest of your layers, and benefit from GPU acceleration.When you export your model using model.save, the preprocessing layers will be saved along with the rest of your model. If you later deploy this model, it will automatically standardize images (according to the configuration of your layers). This can save you from the effort of having to reimplement that logic server-side.Note: Data augmentation is inactive at test time so input images will only be augmented during calls to model.fit (not model.evaluate or model.predict).
  1. 第二种方法:
#Option 2: Apply the preprocessing layers to your dataset
aug_ds = train_ds.map(lambda x, y: (resize_and_rescale(x, training=True), y))
With this approach, you use Dataset.map to create a dataset that yields batches of augmented images. In this case:Data augmentation will happen asynchronously on the CPU, and is non-blocking. You can overlap the training of your model on the GPU with data preprocessing, using Dataset.prefetch, shown below.
In this case the prepreprocessing layers will not be exported with the model when you call model.save. You will need to attach them to your model before saving it or reimplement them server-side. After training, you can attach the preprocessing layers before export.

6.把与处理层用在数据集上

Configure the train, validation, and test datasets with the preprocessing layers you created above. You will also configure the datasets for performance, using parallel reads and buffered prefetching to yield batches from disk without I/O become blocking. 
Note: data augmentation should only be applied to the training set.
batch_size = 32
AUTOTUNE = tf.data.experimental.AUTOTUNEdef prepare(ds, shuffle=False, augment=False):# Resize and rescale all datasetsds = ds.map(lambda x, y: (resize_and_rescale(x), y), num_parallel_calls=AUTOTUNE)if shuffle:ds = ds.shuffle(1000)# Batch all datasetsds = ds.batch(batch_size)# Use data augmentation only on the training setif augment:ds = ds.map(lambda x, y: (data_augmentation(x, training=True), y), num_parallel_calls=AUTOTUNE)# Use buffered prefecting on all datasetsreturn ds.prefetch(buffer_size=AUTOTUNE)
train_ds = prepare(train_ds, shuffle=True, augment=True)
val_ds = prepare(val_ds)
test_ds = prepare(test_ds)

7.训练模型

model = tf.keras.Sequential([layers.Conv2D(16, 3, padding='same', activation='relu'),layers.MaxPooling2D(),layers.Conv2D(32, 3, padding='same', activation='relu'),layers.MaxPooling2D(),layers.Conv2D(64, 3, padding='same', activation='relu'),layers.MaxPooling2D(),layers.Flatten(),layers.Dense(128, activation='relu'),layers.Dense(num_classes)
])
model.compile(optimizer='adam',loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),metrics=['accuracy'])
epochs=5
history = model.fit(train_ds,validation_data=val_ds,epochs=epochs
)
Epoch 1/5
92/92 [==============================] - 30s 315ms/step - loss: 1.5078 - accuracy: 0.3428 - val_loss: 1.0809 - val_accuracy: 0.6240
Epoch 2/5
92/92 [==============================] - 28s 303ms/step - loss: 1.0781 - accuracy: 0.5724 - val_loss: 0.9762 - val_accuracy: 0.6322
Epoch 3/5
92/92 [==============================] - 28s 295ms/step - loss: 1.0083 - accuracy: 0.5900 - val_loss: 0.9570 - val_accuracy: 0.6376
Epoch 4/5
92/92 [==============================] - 28s 300ms/step - loss: 0.9537 - accuracy: 0.6116 - val_loss: 0.9081 - val_accuracy: 0.6485
Epoch 5/5
92/92 [==============================] - 28s 301ms/step - loss: 0.8816 - accuracy: 0.6525 - val_loss: 0.8353 - val_accuracy: 0.6594
loss, acc = model.evaluate(test_ds)
print("Accuracy", acc)
12/12 [==============================] - 1s 83ms/step - loss: 0.8226 - accuracy: 0.6567
Accuracy 0.6566757559776306

8.自定义数据增强

First, you will create a layers.Lambda layer. This is a good way to write concise code. Next, you will write a new layer via subclassing, which gives you more control. Both layers will randomly invert the colors in an image, accoring to some probability.
def random_invert_img(x, p=0.5):if  tf.random.uniform([]) < p:x = (255-x)else:xreturn xdef random_invert(factor=0.5):return layers.Lambda(lambda x: random_invert_img(x, factor))random_invert = random_invert()plt.figure(figsize=(10, 10))
for i in range(9):augmented_image = random_invert(image)ax = plt.subplot(3, 3, i + 1)plt.imshow(augmented_image[0].numpy().astype("uint8"))plt.axis("off")

在这里插入图片描述

#Next, implement a custom layer by subclassing.class RandomInvert(layers.Layer):def __init__(self, factor=0.5, **kwargs):super().__init__(**kwargs)self.factor = factordef call(self, x):return random_invert_img(x)_ = plt.imshow(RandomInvert()(image)[0])

在这里插入图片描述

9.Using tf.image

Since the flowers dataset was previously configured with data augmentation, let's reimport it to start fresh.(train_ds, val_ds, test_ds), metadata = tfds.load('tf_flowers',split=['train[:80%]', 'train[80%:90%]', 'train[90%:]'],with_info=True,as_supervised=True,
)
#Retrieve an image to work with.image, label = next(iter(train_ds))
_ = plt.imshow(image)
_ = plt.title(get_label_name(label))

在这里插入图片描述

Let's use the following function to visualize and compare the original and augmented images side-by-side.def visualize(original, augmented):fig = plt.figure()plt.subplot(1,2,1)plt.title('Original image')plt.imshow(original)plt.subplot(1,2,2)plt.title('Augmented image')plt.imshow(augmented)
#Data augmentation
#Flipping the image
3Flip the image either vertically or horizontally.flipped = tf.image.flip_left_right(image)
visualize(image, flipped)

在这里插入图片描述

#Grayscale an image.grayscaled = tf.image.rgb_to_grayscale(image)
visualize(image, tf.squeeze(grayscaled))
_ = plt.colorbar()

在这里插入图片描述

#Saturate an image by providing a saturation factor.saturated = tf.image.adjust_saturation(image, 3)
visualize(image, saturated)

在这里插入图片描述

#Change image brightness
#Change the brightness of image by providing a brightness factor.bright = tf.image.adjust_brightness(image, 0.4)
visualize(image, bright)

在这里插入图片描述

#Center crop the image
#Crop the image from center up to the image part you desire.cropped = tf.image.central_crop(image, central_fraction=0.5)
visualize(image,cropped)

在这里插入图片描述

#Rotate the image
#Rotate an image by 90 degrees.rotated = tf.image.rot90(image)
visualize(image, rotated)

在这里插入图片描述

#Apply augmentation to a dataset
#As before, apply data augmentation to a dataset using Dataset.map.def resize_and_rescale(image, label):image = tf.cast(image, tf.float32)image = tf.image.resize(image, [IMG_SIZE, IMG_SIZE])image = (image / 255.0)return image, labeldef augment(image,label):image, label = resize_and_rescale(image, label)# Add 6 pixels of paddingimage = tf.image.resize_with_crop_or_pad(image, IMG_SIZE + 6, IMG_SIZE + 6) # Random crop back to the original sizeimage = tf.image.random_crop(image, size=[IMG_SIZE, IMG_SIZE, 3])image = tf.image.random_brightness(image, max_delta=0.5) # Random brightnessimage = tf.clip_by_value(image, 0, 1)return image, label
#Configure the datasets
train_ds = (train_ds.shuffle(1000).map(augment, num_parallel_calls=AUTOTUNE).batch(batch_size).prefetch(AUTOTUNE)
) val_ds = (val_ds.map(resize_and_rescale, num_parallel_calls=AUTOTUNE).batch(batch_size).prefetch(AUTOTUNE)
)test_ds = (test_ds.map(resize_and_rescale, num_parallel_calls=AUTOTUNE).batch(batch_size).prefetch(AUTOTUNE)
)#These datasets can now be used to train a model as shown previously.

这篇关于2.9-tf2-数据增强-tf_flowers的文章就介绍到这儿,希望我们推荐的文章对编程师们有所帮助!



http://www.chinasem.cn/article/365916

相关文章

MyBatis-plus处理存储json数据过程

《MyBatis-plus处理存储json数据过程》文章介绍MyBatis-Plus3.4.21处理对象与集合的差异:对象可用内置Handler配合autoResultMap,集合需自定义处理器继承F... 目录1、如果是对象2、如果需要转换的是List集合总结对象和集合分两种情况处理,目前我用的MP的版本

GSON框架下将百度天气JSON数据转JavaBean

《GSON框架下将百度天气JSON数据转JavaBean》这篇文章主要为大家详细介绍了如何在GSON框架下实现将百度天气JSON数据转JavaBean,文中的示例代码讲解详细,感兴趣的小伙伴可以了解下... 目录前言一、百度天气jsON1、请求参数2、返回参数3、属性映射二、GSON属性映射实战1、类对象映

C# LiteDB处理时间序列数据的高性能解决方案

《C#LiteDB处理时间序列数据的高性能解决方案》LiteDB作为.NET生态下的轻量级嵌入式NoSQL数据库,一直是时间序列处理的优选方案,本文将为大家大家简单介绍一下LiteDB处理时间序列数... 目录为什么选择LiteDB处理时间序列数据第一章:LiteDB时间序列数据模型设计1.1 核心设计原则

Java+AI驱动实现PDF文件数据提取与解析

《Java+AI驱动实现PDF文件数据提取与解析》本文将和大家分享一套基于AI的体检报告智能评估方案,详细介绍从PDF上传、内容提取到AI分析、数据存储的全流程自动化实现方法,感兴趣的可以了解下... 目录一、核心流程:从上传到评估的完整链路二、第一步:解析 PDF,提取体检报告内容1. 引入依赖2. 封装

MySQL中查询和展示LONGBLOB类型数据的技巧总结

《MySQL中查询和展示LONGBLOB类型数据的技巧总结》在MySQL中LONGBLOB是一种二进制大对象(BLOB)数据类型,用于存储大量的二进制数据,:本文主要介绍MySQL中查询和展示LO... 目录前言1. 查询 LONGBLOB 数据的大小2. 查询并展示 LONGBLOB 数据2.1 转换为十

使用SpringBoot+InfluxDB实现高效数据存储与查询

《使用SpringBoot+InfluxDB实现高效数据存储与查询》InfluxDB是一个开源的时间序列数据库,特别适合处理带有时间戳的监控数据、指标数据等,下面详细介绍如何在SpringBoot项目... 目录1、项目介绍2、 InfluxDB 介绍3、Spring Boot 配置 InfluxDB4、I

Java整合Protocol Buffers实现高效数据序列化实践

《Java整合ProtocolBuffers实现高效数据序列化实践》ProtocolBuffers是Google开发的一种语言中立、平台中立、可扩展的结构化数据序列化机制,类似于XML但更小、更快... 目录一、Protocol Buffers简介1.1 什么是Protocol Buffers1.2 Pro

Python实现数据可视化图表生成(适合新手入门)

《Python实现数据可视化图表生成(适合新手入门)》在数据科学和数据分析的新时代,高效、直观的数据可视化工具显得尤为重要,下面:本文主要介绍Python实现数据可视化图表生成的相关资料,文中通过... 目录前言为什么需要数据可视化准备工作基本图表绘制折线图柱状图散点图使用Seaborn创建高级图表箱线图热

MySQL数据脱敏的实现方法

《MySQL数据脱敏的实现方法》本文主要介绍了MySQL数据脱敏的实现方法,包括字符替换、加密等方法,通过工具类和数据库服务整合,确保敏感信息在查询结果中被掩码处理,感兴趣的可以了解一下... 目录一. 数据脱敏的方法二. 字符替换脱敏1. 创建数据脱敏工具类三. 整合到数据库操作1. 创建服务类进行数据库

MySQL中处理数据的并发一致性的实现示例

《MySQL中处理数据的并发一致性的实现示例》在MySQL中处理数据的并发一致性是确保多个用户或应用程序同时访问和修改数据库时,不会导致数据冲突、数据丢失或数据不一致,MySQL通过事务和锁机制来管理... 目录一、事务(Transactions)1. 事务控制语句二、锁(Locks)1. 锁类型2. 锁粒