SwiftSpeech

Speech Recognition Made Simple

Recognize your user's voice elegantly without having to figure out authorization and audio engines.

• SwiftSpeech Examples
• Features
• Installation
• Getting Started
• SwiftSpeech.Session
• Customized View Components
• Support SwiftSpeech Modifiers
• License

SwiftSpeech Examples

Aside from the readme, the best way to learn more about SwiftSpeech and how speech recognition capabilities are implemented in apps like WeChat is to check out my new project SwiftSpeech Examples. For now, it contains a WeChat voice message interface mock and the three demos in SwiftSpeech.

Features

SwiftSpeech is a wrapper for Apple's Speech framework with deep SwiftUI and Combine integration.

• [x] UI control + speech recognition functionality in just several lines of code.
• [x] Customizable cancelling.
• [x] SwiftUI style reactive APIs and Combine support.
• [x] Highly customizable but also keeping your code highly reusable via a composable structure.
• [x] Fully open low-level APIs.

Installation

SwiftSpeech is available through Swift Package Manager. To use it, add a package dependency using URL:

html
https://github.com/Cay-Zhang/SwiftSpeech.git

Getting Started

1. Authorization

Although SwiftSpeech takes care of all the verbose stuff of authorization for you, you still have to state the usage descriptions and specify where you want the authorization process to happen before you start to use it.

Usage Descriptions in Info.plist

If you haven't, add these two rows in your

Info.plist

NSSpeechRecognitionUsageDescription

NSMicrophoneUsageDescription

These are the messages your users will see on their first use, in the alerts that ask them for permission to use speech recognition and to access the microphone.

Here's an exmample:

xml
NSSpeechRecognitionUsageDescription
This app uses speech recognition to convert your speech into text.
NSMicrophoneUsageDescription
This app uses the mircrophone to record audio for speech recognition.

Request Authorization

Place

SwiftSpeech.requestSpeechRecognitionAuthorization()

onAppear

swift
.onAppear {
    SwiftSpeech.requestSpeechRecognitionAuthorization()
}

2. Try some demos

You can now start to try out some light-weight demos bundled with the framework using Xcode preview. Click the "Preview on Device" button to try the demo on your device. ``

swift
static var previews: some View {
    // Two of the demo views below can take a

Group {
    SwiftSpeech.Demos.Basic(localeIdentifier: yourLocaleString)
    SwiftSpeech.Demos.Colors()
    SwiftSpeech.Demos.List(localeIdentifier: yourLocaleString)
}

} ```

Here are the "previews" of your

previews

3. Build it yourself

Knowing what this framework can do, you can now start to learn about the concepts in SwiftSpeech.

Inspect the source code of

SwiftSpeech.Demos.Basic

swift
SwiftSpeech.RecordButton()                                        // 1. The View Component
    .swiftSpeechRecordOnHold(sessionConfiguration:animation:distanceToCancel:)  // 2. The Functional Component
    .onRecognizeLatest(update: $text)                             // 3. SwiftSpeech Modifier(s)

View

.swiftSpeechToggleRecordingOnTap(sessionConfiguration:animation:)

For now, you can just use the built-in View Component and Functional Component. Let's explore some SwiftSpeech Modifiers first since every app handles its data differently:

Important: Chaining multiple or identical SwiftSpeech Modifiers together doesn't override any behavior. All actions of the modifiers will be executed in the order where the closest to the Functional Component executes first and the farthest executes last.

// 1
// All three demos use these modifiers.
// Inspect the source code of them if you want examples!
.onRecognizeLatest(
    includePartialResults: Bool = true,
    handleResult: (SwiftSpeech.Session, SFSpeechRecognitionResult) -> Void,
    handleError: (SwiftSpeech.Session, Error) -> Void
)

.onRecognize(
    includePartialResults: Bool = true,
    handleResult: (SwiftSpeech.Session, SFSpeechRecognitionResult) -> Void,
    handleError: (SwiftSpeech.Session, Error) -> Void
)
// This one simply assigns the recognized text to the binding in handleResult and ignores errors.
.onRecognizeLatest(
    includePartialResults: Bool = true,
    update: Binding
)
// This one prints the recognized text and ignores errors.
.printRecognizedText(includePartialResults: Bool = true)

The first group of modifiers encapsulates the core value of SwiftSpeech. It does all the publisher transformation and subscription for you and calls the closures with enough information to facilitate a sophisticated task when a recognition result is yielded.

onRecognizeLatest

onRecognize

In

handleResult

SwiftSpeech.Session

id

The second is a

SFSpeechRecognitionResult

result.bestTranscription.formattedString

In

handleError

// 2
.onStartRecording(appendAction: (SwiftSpeech.Session) -> Void)
.onStopRecording(appendAction: (SwiftSpeech.Session) -> Void)
.onCancelRecording(appendAction: (SwiftSpeech.Session) -> Void)

The second group gives you utter control over the whole lifespan of a

SwiftSpeech.Session

SwiftSpeech.Session

// 3
// `SwiftSpeech.ViewModifiers.OnRecognize` uses these modifiers.
// Inspect the source code of it if you want examples!
.onStartRecording(sendSessionTo: Subject)
.onStopRecording(sendSessionTo: Subject)
.onCancelRecording(sendSessionTo: Subject)

The third group might be useful if you prefer a reactive programming style. The only new argument here is a

Combine.Subject

CurrentValueSubject

PassthroughSubject

SwiftSpeech.Session

Subject

SwiftSpeech.Session

Configuration

A session can be configured using a

SwiftSpeech.Session.Configuration

SwiftSpeech.Session.Configuration

Customized Subscription to Recognition Results

If the built-in

onRecognize(Latest)

onStart/Stop/CancelRecording

A

Session

resultPublisher

Output

SFSpeechRecognitionResult

Failure

Error

You will receive a completion event when the

Session

result.isFinal == true

cancelRecording()

A

Session

stringPublisher

Independent Use

Here's an example of using

Session

swift
let session = SwiftSpeech.Session(configuration: SwiftSpeech.Session.Configuration(locale: Locale(identifier: "en-US"), contextualStrings: ["SwiftSpeech"]))
try session.startRecording()
session.stringPublisher?
    .sink { text in
        // do something with the text
    }
    .store(in: &cancelBag)

SwiftSpeech.Session

Customized View Components

A View Component is a dedicated

View

Inspect the source code of

SwiftSpeech.RecordButton

Button

@Environment(\.swiftSpeechState) var state: SwiftSpeech.State
@SpeechRecognitionAuthStatus var authStatus

Both are pretty self-explanatory: the first one represents its current state of recording, and the second one indicates the authorization status of speech recognition.

Here are more details of

SwiftSpeech.State

enum SwiftSpeech.State {
    /// Indicating there is no recording in progress.
    /// - Note: It's the default value for `@Environment(\.swiftSpeechState)`.
    case pending
    /// Indicating there is a recording in progress and the user does not intend to cancel it.
    case recording
    /// Indicating there is a recording in progress and the user intends to cancel it.
    case cancelling
}

authStatus

SFSpeechRecognizerAuthorizationStatus

$authStatus

authStatus == .authorized

Combined with a Functional Component and some SwiftSpeech Modifiers, hopefully, you can build your own fancy record systems now!

Support SwiftSpeech Modifiers

The library provides two general functional components that add a gesture to the view it modifies and perform speech recognition for you: ```swift // They already support SwiftSpeech Modifiers. func swiftSpeechRecordOnHold( sessionConfiguration: SwiftSpeech.Session.Configuration = SwiftSpeech.Session.Configuration(), animation: Animation = SwiftSpeech.defaultAnimation, distanceToCancel: CGFloat = 50.0 ) -> some View

func swiftSpeechToggleRecordingOnTap( sessionConfiguration: SwiftSpeech.Session.Configuration = SwiftSpeech.Session.Configuration(), animation: Animation = SwiftSpeech.defaultAnimation )

If you decide to implement a view that involves a custom gesture other than a hold or a tap, you can also support SwiftSpeech Modifiers by adding a delegate and calling its methods at the appropriate time:

For guidance on how to implement a custom view for speech recognition, refer to

License

SwiftSpeech is available under the MIT license.