Unity 工具 之 Azure 微软连续语音识别ASR的简单整理

2024-01-12 10:36

本文主要是介绍Unity 工具 之 Azure 微软连续语音识别ASR的简单整理,希望对大家解决编程问题提供一定的参考价值,需要的开发者们随着小编来一起学习吧!

Unity 工具 之 Azure 微软连续语音识别ASR的简单整理

目录

Unity 工具 之 Azure 微软连续语音识别ASR的简单整理

一、简单介绍

二、实现原理

三、注意实现

四、实现步骤

 五、关键脚本


一、简单介绍

Unity 工具类,自己整理的一些游戏开发可能用到的模块,单独独立使用,方便游戏开发。

本节介绍,这里在使用微软的Azure 进行语音合成的两个方法的做简单整理,这里简单说明,如果你有更好的方法,欢迎留言交流。

官网注册:

面向学生的 Azure - 免费帐户额度 | Microsoft Azure

官网技术文档网址:

技术文档 | Microsoft Learn

官网的TTS:

语音转文本快速入门 - 语音服务 - Azure AI services | Microsoft Learn

Azure Unity SDK  包官网:

安装语音 SDK - Azure Cognitive Services | Microsoft Learn

SDK具体链接:

https://aka.ms/csspeech/unitypackage

二、实现原理

1、官网申请得到语音识别对应的 SPEECH_KEY 和 SPEECH_REGION

2、因为语音识别需要用到麦克风,移动端需要申请麦克风权限

3、开启语音识别,监听语音识别对应事件,即可获取到识别结果

三、注意实现

1、注意如果有卡顿什么的,注意主子线程切换,可能可以适当解决你的卡顿现象

2、注意电脑端(例如windows)运行可以不申请麦克风权限,但是移动端(例如Android)运行要申请麦克风权限,不然无法开启识别成功,可能会报错:Exception with an error code: 0x15

<uses-permission android:name="android.permission.RECORD_AUDIO"/>

System.ApplicationException: Exception with an error code: 0x15 at Microsoft.CognitiveServices.Speech.Internal.SpxExceptionThrower.ThrowIfFail (System.IntPtr hr) [0x00000] in <00000000000000000000000000000000>:0 at Microsoft.CognitiveServices.Speech.Recognizer.StartContinuousRecognition () [0x00000] in <00000000000000000000000000000000>:0 at Microsoft.CognitiveServices.Speech.Recognizer.DoAsyncRecognitionAction (System.Action recoImplAction) [0x00000] in <00000000000000000000000000000000>:0 at System.Threading.Tasks.Task.Execute () [0x00000] in <00000000000000000000000000000000>:0 at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) [0x00000] in <00000000000000000000000000000000>:0 at System.Threading.Tasks.Task.ExecuteWithThreadLocal (System.Threading.Tasks.Task& currentTaskSlot) [0x00000] in

四、实现步骤

1、下载好SDK 导入

2、简单的搭建场景

3、编写对应脚本,测试语音识别功能

4、把测试脚本添加到场景中,并赋值

5、如果移动端,例如 Android 端,勾选如下,添加麦克风权限

<uses-permission android:name="android.permission.RECORD_AUDIO"/>

5、运行,点击对应按钮,开始识别,Console 中可以看到识别结果

 五、关键脚本

1、TestSpeechRecognitionHandler


using UnityEngine;
using UnityEngine.Android;
using UnityEngine.UI;public class TestSpeechRecognitionHandler : MonoBehaviour
{#region Data/// <summary>/// 按钮,文本/// </summary>public Button QuitButton;public Button ASRButton;public Button StopASRButton;public Text ASRText;/// <summary>/// m_SpeechAndKeywordRecognitionHandler/// </summary>SpeechRecognitionHandler m_SpeechAndKeywordRecognitionHandler;#endregion#region Liefecycle function/// <summary>/// Start/// </summary>void Start(){QuitButton.onClick.AddListener(OnClickQuitButton);ASRButton.onClick.AddListener(OnClickASRButton);StopASRButton.onClick.AddListener(OnClickStopASRButton);// 请求麦克风权限RequestMicrophonePermission();}/// <summary>/// 应用退出/// </summary>async void OnApplicationQuit() {await m_SpeechAndKeywordRecognitionHandler.StopContinuousRecognizer();}#endregion#region Private function/// <summary>/// RequestMicrophonePermission/// </summary>void RequestMicrophonePermission(){// 检查当前平台是否为 Androidif (Application.platform == RuntimePlatform.Android){// 检查是否已经授予麦克风权限if (!Permission.HasUserAuthorizedPermission(Permission.Microphone)){// 如果没有权限,请求用户授权Permission.RequestUserPermission(Permission.Microphone);}}else{// 在其他平台上,可以执行其他平台特定的逻辑Debug.LogWarning("Microphone permission is not needed on this platform.");}SpeechInitialized();}/// <summary>/// SpeechInitialized/// </summary>private void SpeechInitialized() {ASRText.text = "";m_SpeechAndKeywordRecognitionHandler = new SpeechRecognitionHandler();m_SpeechAndKeywordRecognitionHandler.onRecognizingAction = (str) => { Debug.Log("onRecognizingAction: " + str); };m_SpeechAndKeywordRecognitionHandler.onRecognizedSpeechAction = (str) => { Loom.QueueOnMainThread(() => ASRText.text += str);  Debug.Log("onRecognizedSpeechAction: " + str); };m_SpeechAndKeywordRecognitionHandler.onErrorAction = (str) => { Debug.Log("onErrorAction: " + str); };m_SpeechAndKeywordRecognitionHandler.Initialized();}/// <summary>/// OnClickQuitButton/// </summary>private void OnClickQuitButton() {
#if UNITY_EDITORUnityEditor.EditorApplication.isPlaying = false;#elseApplication.Quit();
#endif }/// <summary>/// OnClickASRButton/// </summary>private void OnClickASRButton() {m_SpeechAndKeywordRecognitionHandler.StartContinuousRecognizer();}/// <summary>/// OnClickStopASRButton/// </summary>private async void OnClickStopASRButton(){await m_SpeechAndKeywordRecognitionHandler.StopContinuousRecognizer();}#endregion
}

2、SpeechRecognitionHandler

using UnityEngine;
using Microsoft.CognitiveServices.Speech;
using Microsoft.CognitiveServices.Speech.Audio;
using System;
using Task = System.Threading.Tasks.Task;/// <summary>
/// 语音识别转文本和关键词识别
/// </summary>
public class SpeechRecognitionHandler
{#region Data/// <summary>/// /// </summary>const string TAG = "[SpeechAndKeywordRecognitionHandler] ";/// <summary>/// 识别配置/// </summary>private SpeechConfig m_SpeechConfig;/// <summary>/// 音频配置/// </summary>private AudioConfig m_AudioConfig;/// <summary>/// 语音识别/// </summary>private SpeechRecognizer m_SpeechRecognizer;/// <summary>/// LLM 大模型配置/// </summary>private ASRConfig m_ASRConfig;/// <summary>/// 识别的事件/// </summary>public Action<string> onRecognizingAction;public Action<string> onRecognizedSpeechAction;public Action<string> onErrorAction;public Action<string> onSessionStoppedAction;#endregion#region Public function/// <summary>/// 初始化/// </summary>/// <returns></returns>public async void Initialized(){m_ASRConfig = new ASRConfig();Debug.Log(TAG + "m_LLMConfig.AZURE_SPEECH_RECOGNITION_LANGUAGE " + m_ASRConfig.AZURE_SPEECH_RECOGNITION_LANGUAGE);Debug.Log(TAG + "m_LLMConfig.AZURE_SPEECH_REGION " + m_ASRConfig.AZURE_SPEECH_REGION);m_SpeechConfig = SpeechConfig.FromSubscription(m_ASRConfig.AZURE_SPEECH_KEY, m_ASRConfig.AZURE_SPEECH_REGION);m_SpeechConfig.SpeechRecognitionLanguage = m_ASRConfig.AZURE_SPEECH_RECOGNITION_LANGUAGE;m_AudioConfig = AudioConfig.FromDefaultMicrophoneInput();Debug.Log(TAG + " Initialized 2 ====");// 根据自己需要处理(不需要也行)await Task.Delay(100);}#endregion#region Private function/// <summary>/// 设置识别回调事件/// </summary>private void SetRecoginzeCallback(){Debug.Log(TAG + " SetRecoginzeCallback == ");if (m_SpeechRecognizer != null){m_SpeechRecognizer.Recognizing += OnRecognizing;m_SpeechRecognizer.Recognized += OnRecognized;m_SpeechRecognizer.Canceled += OnCanceled;m_SpeechRecognizer.SessionStopped += OnSessionStopped;Debug.Log(TAG+" SetRecoginzeCallback OK ");}}#endregion#region Callback/// <summary>/// 正在识别/// </summary>/// <param name="s"></param>/// <param name="e"></param>private void OnRecognizing(object s, SpeechRecognitionEventArgs e){Debug.Log(TAG + "RecognizingSpeech:" + e.Result.Text + " :[e.Result.Reason]:" + e.Result.Reason);if (e.Result.Reason == ResultReason.RecognizingSpeech ){Debug.Log(TAG + " Trigger onRecognizingAction is null :" + onRecognizingAction == null);onRecognizingAction?.Invoke(e.Result.Text);}}/// <summary>/// 识别结束/// </summary>/// <param name="s"></param>/// <param name="e"></param>private void OnRecognized(object s, SpeechRecognitionEventArgs e){Debug.Log(TAG + "RecognizedSpeech:" + e.Result.Text + " :[e.Result.Reason]:" + e.Result.Reason);if (e.Result.Reason == ResultReason.RecognizedSpeech ){bool tmp = onRecognizedSpeechAction == null;Debug.Log(TAG + " Trigger onRecognizedSpeechAction is null :" + tmp);onRecognizedSpeechAction?.Invoke(e.Result.Text);}}/// <summary>/// 识别取消/// </summary>/// <param name="s"></param>/// <param name="e"></param>private void OnCanceled(object s, SpeechRecognitionCanceledEventArgs e){Debug.LogFormat(TAG+"Canceled: Reason={0}", e.Reason );if (e.Reason == CancellationReason.Error){onErrorAction?.Invoke(e.ErrorDetails);}}/// <summary>/// 会话结束/// </summary>/// <param name="s"></param>/// <param name="e"></param>private void OnSessionStopped(object s, SessionEventArgs e){Debug.Log(TAG+"Session stopped event." );onSessionStoppedAction?.Invoke("Session stopped event.");}#endregion#region 连续语音识别转文本/// <summary>/// 开启连续语音识别转文本/// </summary>public void StartContinuousRecognizer(){Debug.LogWarning(TAG + "StartContinuousRecognizer");try{// 转到异步中(根据自己需要处理)Loom.RunAsync(async () => {try{if (m_SpeechRecognizer != null){m_SpeechRecognizer.Dispose();m_SpeechRecognizer = null;}if (m_SpeechRecognizer == null){m_SpeechRecognizer = new SpeechRecognizer(m_SpeechConfig, m_AudioConfig);SetRecoginzeCallback();}await m_SpeechRecognizer.StartContinuousRecognitionAsync().ConfigureAwait(false);Loom.QueueOnMainThread(() => {Debug.LogWarning(TAG + "StartContinuousRecognizer QueueOnMainThread ok");});Debug.LogWarning(TAG + "StartContinuousRecognizer RunAsync ok");}catch (Exception e){Loom.QueueOnMainThread(() =>{Debug.LogError(TAG + " StartContinuousRecognizer 0 " + e);});}});}catch (Exception e){Debug.LogError(TAG + " StartContinuousRecognizer 1 " + e);}}/// <summary>/// 结束连续语音识别转文本/// </summary>public async Task StopContinuousRecognizer(){try{if (m_SpeechRecognizer != null){await m_SpeechRecognizer.StopContinuousRecognitionAsync().ConfigureAwait(false);//m_SpeechRecognizer.Dispose();//m_SpeechRecognizer = null;Debug.LogWarning(TAG + " StopContinuousRecognizer");}}catch (Exception e){Debug.LogError(TAG + " StopContinuousRecognizer Exception : " + e);}}#endregion}

3、ASRConfig

public class ASRConfig 
{#region Azure ASR/// <summary>/// AZURE_SPEECH_KEY/// </summary>public virtual string AZURE_SPEECH_KEY { get; } = @"098dc5aa21a14758a347dcac36d7eb66";/// <summary>/// AZURE_SPEECH_REGION/// </summary>public virtual string AZURE_SPEECH_REGION { get; } = @"eastasia";/// <summary>/// AZURE_SPEECH_RECOGNITION_LANGUAGE/// </summary>public virtual string AZURE_SPEECH_RECOGNITION_LANGUAGE { get; } = @"zh-CN";#endregion
}

这篇关于Unity 工具 之 Azure 微软连续语音识别ASR的简单整理的文章就介绍到这儿,希望我们推荐的文章对编程师们有所帮助!



http://www.chinasem.cn/article/597589

相关文章

Python使用OpenCV实现获取视频时长的小工具

《Python使用OpenCV实现获取视频时长的小工具》在处理视频数据时,获取视频的时长是一项常见且基础的需求,本文将详细介绍如何使用Python和OpenCV获取视频时长,并对每一行代码进行深入解析... 目录一、代码实现二、代码解析1. 导入 OpenCV 库2. 定义获取视频时长的函数3. 打开视频文

Spring IoC 容器的使用详解(最新整理)

《SpringIoC容器的使用详解(最新整理)》文章介绍了Spring框架中的应用分层思想与IoC容器原理,通过分层解耦业务逻辑、数据访问等模块,IoC容器利用@Component注解管理Bean... 目录1. 应用分层2. IoC 的介绍3. IoC 容器的使用3.1. bean 的存储3.2. 方法注

MySQL 删除数据详解(最新整理)

《MySQL删除数据详解(最新整理)》:本文主要介绍MySQL删除数据的相关知识,本文通过实例代码给大家介绍的非常详细,对大家的学习或工作具有一定的参考借鉴价值,需要的朋友参考下吧... 目录一、前言二、mysql 中的三种删除方式1.DELETE语句✅ 基本语法: 示例:2.TRUNCATE语句✅ 基本语

Linux中压缩、网络传输与系统监控工具的使用完整指南

《Linux中压缩、网络传输与系统监控工具的使用完整指南》在Linux系统管理中,压缩与传输工具是数据备份和远程协作的桥梁,而系统监控工具则是保障服务器稳定运行的眼睛,下面小编就来和大家详细介绍一下它... 目录引言一、压缩与解压:数据存储与传输的优化核心1. zip/unzip:通用压缩格式的便捷操作2.

sqlite3 命令行工具使用指南

《sqlite3命令行工具使用指南》本文系统介绍sqlite3CLI的启动、数据库操作、元数据查询、数据导入导出及输出格式化命令,涵盖文件管理、备份恢复、性能统计等实用功能,并说明命令分类、SQL语... 目录一、启动与退出二、数据库与文件操作三、元数据查询四、数据操作与导入导出五、查询输出格式化六、实用功

Python变量与数据类型全解析(最新整理)

《Python变量与数据类型全解析(最新整理)》文章介绍Python变量作为数据载体,命名需遵循字母数字下划线规则,不可数字开头,大小写敏感,避免关键字,本文给大家介绍Python变量与数据类型全解析... 目录1、变量变量命名规范python数据类型1、基本数据类型数值类型(Number):布尔类型(bo

基于Python开发Windows屏幕控制工具

《基于Python开发Windows屏幕控制工具》在数字化办公时代,屏幕管理已成为提升工作效率和保护眼睛健康的重要环节,本文将分享一个基于Python和PySide6开发的Windows屏幕控制工具,... 目录概述功能亮点界面展示实现步骤详解1. 环境准备2. 亮度控制模块3. 息屏功能实现4. 息屏时间

Python中图片与PDF识别文本(OCR)的全面指南

《Python中图片与PDF识别文本(OCR)的全面指南》在数据爆炸时代,80%的企业数据以非结构化形式存在,其中PDF和图像是最主要的载体,本文将深入探索Python中OCR技术如何将这些数字纸张转... 目录一、OCR技术核心原理二、python图像识别四大工具库1. Pytesseract - 经典O

SQLite3命令行工具最佳实践指南

《SQLite3命令行工具最佳实践指南》SQLite3是轻量级嵌入式数据库,无需服务器支持,具备ACID事务与跨平台特性,适用于小型项目和学习,sqlite3.exe作为命令行工具,支持SQL执行、数... 目录1. SQLite3简介和特点2. sqlite3.exe使用概述2.1 sqlite3.exe

基于Python实现一个Windows Tree命令工具

《基于Python实现一个WindowsTree命令工具》今天想要在Windows平台的CMD命令终端窗口中使用像Linux下的tree命令,打印一下目录结构层级树,然而还真有tree命令,但是发现... 目录引言实现代码使用说明可用选项示例用法功能特点添加到环境变量方法一:创建批处理文件并添加到PATH1