Chapter 30

whisper

Notebookvenv18 cells

Azure audio whisper (preview) example

The example shows how to use the Azure OpenAI Whisper model to transcribe audio files.

Setup

First, we install the necessary dependencies and import the libraries we will be using.

In [ ]python · cell 3

python

! pip install "openai>=1.0.0,<2.0.0"
! pip install python-dotenv

In [ ]python · cell 4

python

import os
import openai
import dotenv

dotenv.load_dotenv()

Authentication

The Azure OpenAI service supports multiple authentication mechanisms that include API keys and Azure Active Directory token credentials.

In [4]python · cell 6

python

use_azure_active_directory = False  # Set this flag to True if you are using Azure Active Directory

Authentication using API key

To set up the OpenAI SDK to use an Azure API Key, we need to set api_key to a key associated with your endpoint (you can find this key in "Keys and Endpoints" under "Resource Management" in the Azure Portal). You'll also find the endpoint for your resource here.

In [5]python · cell 8

python

if not use_azure_active_directory:
    endpoint = os.environ["AZURE_OPENAI_ENDPOINT"]
    api_key = os.environ["AZURE_OPENAI_API_KEY"]

    client = openai.AzureOpenAI(
        azure_endpoint=endpoint,
        api_key=api_key,
        api_version="2023-09-01-preview"
    )

Authentication using Azure Active Directory

Let's now see how we can authenticate via Azure Active Directory. We'll start by installing the azure-identity library. This library will provide the token credentials we need to authenticate and help us build a token credential provider through the get_bearer_token_provider helper function. It's recommended to use get_bearer_token_provider over providing a static token to AzureOpenAI because this API will automatically cache and refresh tokens for you.

For more information on how to set up Azure Active Directory authentication with Azure OpenAI, see the documentation.

In [ ]python · cell 10

python

! pip install "azure-identity>=1.15.0"

In [5]python · cell 11

python

from azure.identity import DefaultAzureCredential, get_bearer_token_provider

if use_azure_active_directory:
    endpoint = os.environ["AZURE_OPENAI_ENDPOINT"]
    api_key = os.environ["AZURE_OPENAI_API_KEY"]

    client = openai.AzureOpenAI(
        azure_endpoint=endpoint,
        azure_ad_token_provider=get_bearer_token_provider(DefaultAzureCredential(), "https://cognitiveservices.azure.com/.default"),
        api_version="2023-09-01-preview"
    )

Note: the AzureOpenAI infers the following arguments from their corresponding environment variables if they are not provided:

api_key from AZURE_OPENAI_API_KEY
azure_ad_token from AZURE_OPENAI_AD_TOKEN
api_version from OPENAI_API_VERSION
azure_endpoint from AZURE_OPENAI_ENDPOINT

Deployments

In this section we are going to create a deployment using the whisper-1 model to transcribe audio files.

Deployments: Create in the Azure OpenAI Studio

Let's deploy a model to use with whisper. Go to https://portal.azure.com, find your Azure OpenAI resource, and then navigate to the Azure OpenAI Studio. Click on the "Deployments" tab and then create a deployment for the model you want to use for whisper. The deployment name that you give the model will be used in the code below.

In [6]python · cell 15

python

deployment = "whisper-deployment" # Fill in the deployment name from the portal here

Audio transcription

Audio transcription, or speech-to-text, is the process of converting spoken words into text. Use the openai.Audio.transcribe method to transcribe an audio file stream to text.

You can get sample audio files from the Azure AI Speech SDK repository at GitHub.

In [7]python · cell 17

python

# download sample audio file
import requests

sample_audio_url = "https://github.com/Azure-Samples/cognitive-services-speech-sdk/raw/master/sampledata/audiofiles/wikipediaOcelot.wav"
audio_file = requests.get(sample_audio_url)
with open("wikipediaOcelot.wav", "wb") as f:
    f.write(audio_file.content)

In [ ]python · cell 18

python

transcription = client.audio.transcriptions.create(
    file=open("wikipediaOcelot.wav", "rb"),
    model=deployment,
)
print(transcription.text)

Chapter 30

whisper

Notebookvenv18 cells

Azure audio whisper (preview) example

The example shows how to use the Azure OpenAI Whisper model to transcribe audio files.

Setup

First, we install the necessary dependencies and import the libraries we will be using.

In [ ]python · cell 3

python

! pip install "openai>=1.0.0,<2.0.0"
! pip install python-dotenv

In [ ]python · cell 4

python

import os
import openai
import dotenv

dotenv.load_dotenv()

Authentication

The Azure OpenAI service supports multiple authentication mechanisms that include API keys and Azure Active Directory token credentials.

In [4]python · cell 6

python

use_azure_active_directory = False  # Set this flag to True if you are using Azure Active Directory

Authentication using API key

In [5]python · cell 8

python

if not use_azure_active_directory:
    endpoint = os.environ["AZURE_OPENAI_ENDPOINT"]
    api_key = os.environ["AZURE_OPENAI_API_KEY"]

    client = openai.AzureOpenAI(
        azure_endpoint=endpoint,
        api_key=api_key,
        api_version="2023-09-01-preview"
    )

Authentication using Azure Active Directory

For more information on how to set up Azure Active Directory authentication with Azure OpenAI, see the documentation.

In [ ]python · cell 10

python

! pip install "azure-identity>=1.15.0"

In [5]python · cell 11

python

from azure.identity import DefaultAzureCredential, get_bearer_token_provider

if use_azure_active_directory:
    endpoint = os.environ["AZURE_OPENAI_ENDPOINT"]
    api_key = os.environ["AZURE_OPENAI_API_KEY"]

    client = openai.AzureOpenAI(
        azure_endpoint=endpoint,
        azure_ad_token_provider=get_bearer_token_provider(DefaultAzureCredential(), "https://cognitiveservices.azure.com/.default"),
        api_version="2023-09-01-preview"
    )

Note: the AzureOpenAI infers the following arguments from their corresponding environment variables if they are not provided:

api_key from AZURE_OPENAI_API_KEY
azure_ad_token from AZURE_OPENAI_AD_TOKEN
api_version from OPENAI_API_VERSION
azure_endpoint from AZURE_OPENAI_ENDPOINT

Deployments

In this section we are going to create a deployment using the whisper-1 model to transcribe audio files.

Deployments: Create in the Azure OpenAI Studio

In [6]python · cell 15

python

deployment = "whisper-deployment" # Fill in the deployment name from the portal here

Audio transcription

Audio transcription, or speech-to-text, is the process of converting spoken words into text. Use the openai.Audio.transcribe method to transcribe an audio file stream to text.

You can get sample audio files from the Azure AI Speech SDK repository at GitHub.

In [7]python · cell 17

python

# download sample audio file
import requests

sample_audio_url = "https://github.com/Azure-Samples/cognitive-services-speech-sdk/raw/master/sampledata/audiofiles/wikipediaOcelot.wav"
audio_file = requests.get(sample_audio_url)
with open("wikipediaOcelot.wav", "wb") as f:
    f.write(audio_file.content)

In [ ]python · cell 18

python

transcription = client.audio.transcriptions.create(
    file=open("wikipediaOcelot.wav", "rb"),
    model=deployment,
)
print(transcription.text)