diberry / summarization-openai-csharp-prompty

This solution converts speech to text and then processes and summarizes the text based on the prompt scenario.

name	description	languages	products	page_type	urlFragment
Ticket Processing with Azure AI	Process tickets automatically with Azure AI LLMs and Speech Service.	DotNet bicep azdeveloper Prompty	azure-openai azure-cognitive-search azure-app-service azure	sample	summarization-openai-csharp-prompty

Automated Ticket Processing using Azure AI

---

Table of Contents

• What is this sample?
• Features
• Architecture Diagram
• Security
• Getting Started
• Costs
• Security Guidelines
• Resources

What is this sample

In this sample we recieve issues reported by field and shop floor workers at a company called Contoso Manufacturing, a manufacturing company that makes car batteries. The issues are shared by the workers either live through microphone input or pre-recorded as audio files. We translate the input from speech to text and then use an LLM and Prompty or Promptflow to summarize the issue and return the results in a format we specify.

This sample uses the Azure AI Speech Service and Python SDk to translate the users speech into text. It leverages Azure OpenAI to summarize the text and Prompty/Prompt Flow to manage and insert the prompt into our code, and to evaluate prompt/LLM performance.

By the end of deploying this template you should be able to:

1. Describe what Azure AI Speech Service Python SDK provides.
2. Explain prompt creation with Prompty/Prompt Flow.
3. Build, run, evaluate, and deploy, the summarization app to Azure.

Features

The project comes with:

• Sample model configurations, evaluation prompts, and Prompty assets
• Sample application code for copilot chat and evaluation functions
• Sample azd-template configuration for managing application on Azure

The sample is also a signature application for demonstrating the new capabilities of the Azure AI platform. Expect regular updates to showcase cutting-edge features and best practices for generative AI development.

Architecture Diagram

Getting Started

Prerequisites

Azure Account

IMPORTANT: In order to deploy and run this example, you'll need:

• Azure account. If you're new to Azure, get an Azure account for free and you'll get some free Azure credits to get started. See guide to deploying with the free trial.
• Azure subscription with access enabled for the Azure OpenAI service. You can request access with this form.
  • Ability to deploy these models - gpt-35-turbo, gpt-4, text-embeddings-ada-002
  • We recommend using Sweden Central or East US 2
• Azure account permissions:
  • Your Azure account must have Microsoft.Authorization/roleAssignments/write permissions, such as Role Based Access Control Administrator, User Access Administrator, or Owner. If you don't have subscription-level permissions, you must be granted RBAC for an existing resource group and deploy to that existing group.
  • Your Azure account also needs Microsoft.Resources/deployments/write permissions on the subscription level.

• Ability to provision Azure AI Search (Paid) - Required for Semantic Ranker
  • We recommend using East US 2
• Ability to provision Azure Monitor (Free tier)
• Ability to deploy to Azure Container Apps (Free tier)

AZD

• Install azd
  • Windows: winget install microsoft.azd
  • Linux: curl -fsSL https://aka.ms/install-azd.sh | bash
  • MacOS: brew tap azure/azd && brew install azd

Step 1: Development Environment

The repository is instrumented with a devcontainer.json configuration that can provide you with a pre-built environment that can be launched locally, or in the cloud. You can also elect to do a manual environment setup locally, if desired. Here are the three options in increasing order of complexity and effort on your part.

Pick one!

• Pre-built environment, in cloud with GitHub Codespaces
• Pre-built environment, on device with Docker Desktop
• Manual setup environment, on device with .NET and NuGet installed

The first approach is recommended for minimal user effort in startup and maintenance. The third approach will require you to manually update or maintain your local environment, to reflect any future updates to the repo.

To setup the development environment you can leverage either GitHub Codespaces, a local .NET with NuGet environment (using Anaconda or venv), or a VS Code Dev Container environment (using Docker).

Step 1.1: Pre-Built Environment, in cloud (GitHub Codespaces)

The recommended option!

1. Fork the repo into your personal profile.

2. In your fork, click the green Code button on the repository

3. Select the Codespaces tab and click Create codespace...

   • You can also click this button:

4. This should open a new browser tab with a Codespaces container setup process running.

5. On completion, this will launch a Visual Studio Code editor in the browser, with all relevant dependencies already installed in the running development container beneath.

Congratulations! Your cloud dev environment is ready!

Once you've launched Codespaces you can proceed to step 2.

Step 1.2: Pre-Built Environment, in cloud (GitHub Codespaces)

This option uses the same devcontainer.json configuration, but launches the development container in your local device using Docker Desktop.

To use this approach, you need to have the following tools pre-installed in your local device:

• Visual Studio Code (with Dev Containers Extension)
• Docker Desktop (community or free version is fine)

Make sure your Docker Desktop daemon is running on your local device. After this:

• Fork this repo to your personal profile
• Clone that fork to your local device
• Open the cloned repo using Visual Studio Code
• If your Dev Containers extension is installed correctly, you will be prompted to "re-open the project in a container" - just confirm to launch the container locally.
• Alternatively, you may need to trigger this step manually. See the Dev Containers Extension for more information.

Once your project launches in the local Docker desktop container, you should see the Visual Studio Code editor reflect that connection in the status bar (blue icon, bottom left).

Congratulations! Your local dev environment is ready!

Once you've launched your docker container environment you can proceed to step 2.

Step 1.3: Manual Setup Environment on device

In order to run this sample locally you will need to:

If all of the above are correctly installed you can set up your local developer environment as follows.

1. First, fork the repo, and then clone the code sample locally:

   git clone https://github.com/Azure-Samples/summarization-openai-python-promptflow.git

2. Open the repo in VS Code and navgate to the src directory

   cd 
   code .
   cd ./src/SummarizationAPI

3. Install the .NET SDK

4. Install the newest version of PowerShell

   • Open PowerShell and use winget install --id Microsoft.Powershell --source winget

5. Install the C# Dev Kit in VS Code

   • Open the VS Code Extensions tab
   • Search for "C# Dev Kit"
   • Install the extension

6. Install the Azure CLI for your device OS

2. Create Azure resources

We setup our development ennvironment in the previous step. In this step, we'll provision Azure resources for our project, ready to use for developing our LLM Application.

Authenticate with Azure

Start by connecting your Visual Studio Code environment to your Azure account:

1. Open the terminal in VS Code and use command az login.
2. Complete the authentication flow.

If you are running within a dev container, use these instructions to login instead:

1. Open the terminal in VS Code and use command az login --use-device-code
2. The console message will give you an alphanumeric code
3. Navigate to https://microsoft.com/devicelogin in a new tab
4. Enter the code from step 2 and complete the flow.

In either case, verify that the console shows a message indicating a successful authentication. Set your subscription in your enviroment with:

az account set -s <SUB_ID>

Congratulations! Your VS Code session is now connected to your Azure subscription!

Provision with Azure Developer CLI

For this project, we need to provision multiple Azure resources in a specific order. Before, we achieved this by running the provision.sh script. Now, we'll use the Azure Developer CLI (or azd) instead, and follow the steps below. Visit the azd reference for more details on tool syntax, commands and options.

Install azd

• If you setup your development environment manually, follow these instructions to install azd for your local device OS.
• If you used a pre-built dev container environment (e.g., GitHub Codespaces or Docker Desktop) the tool is pre-installed for you.
• Verify that the tool is installed by typing azd version in a terminal.

Authenticate with Azure

• Start the authentication flow from a terminal:

  azd auth login

• This should activate a Device Code authentication flow as shown below. Just follow the instructions and complete the auth flow till you get the Logged in on Azure message indicating success.

  Start by copying the next code: <code-here>
  Then press enter and continue to log in from your browser...

Provision and Deploy

• Run this unified command to provision all resources. This will take a non-trivial amount of time to complete.

  azd up

• You will be prompted for the subscription you want you use and the region. The bicep parameters declare two location fields: the first one is the primary location for all resources, the second is a location field specifically for where the OpenAI resource should be created.

That's it! You should now be ready to continue the process as before. Note that this is a new process so there may be some issues to iron out. Start by completing the verification steps below and taking any troubleshooting actions identified.

Verify Provisioning

The script should set up a dedicated resource group with the following resources:

• Azure AI services resource
• Azure Container app resource
• Azure Open AI Service resource
• Azure Speech to Text resource

Add endpoints to the .NET Application

The project now needs to have access to Azure, to do this, go to .\src\SummarizationAPI\SummarizationAPI\appsettings.json and change the following variables.

3. Running Locally

First, you will need to navegate in the console to our folder in Summarization, use the following cd .src\SummarizationAPI\.

After this, we need to restore our dotNet packages to build our solution, we can use dotnet restore. To build our .dll from solution, use dotnet build, it will report two dlls being created, the test and the main one.

To execute our solution use dotnet run --project .\SummarizationAPI\SummarizationAPI.csproj. This will run our solution, which will make available our API to test and connect from our data collection located in chat.json.

To the Swagger endpoint, use the application on http://localhost:5282/swagger/

4. Clean up

To clean up all the resources created by this sample:

1. Run azd down
2. When asked if you are sure you want to continue, enter y
3. When asked if you want to permanently delete the resources, enter y

The resource group and all the resources will be deleted.

Costs

You can estimate the cost of this project's architecture with Azure's pricing calculator

• Azure OpenAI - Standard tier, GPT-4, GPT-35-turbo and Ada models. See Pricing
• Azure Monitor - Serverless, Free Tier See Pricing
• Azure Container Apps - Severless, Free Tier See Pricing

Security Guidelines

This template uses Managed Identity or Key Vault to eliminate the need for developers to manage credentials. Applications can use managed identities to obtain Microsoft Entra tokens without having to manage any credentials.

Additionally, we have added a GitHub Action tool that scans the infrastructure-as-code files and generates a report containing any detected issues.

To ensure best security practices in your repo, we recommend anyone creating solutions based on our templates ensure that the Github secret scanning setting is enabled in your repos.

Resources

• Take a look on more .NET AI Samples.
• Learn more .NET AI with Microsoft Learn
• Learn Azure, deploying in GitHub!

Troubleshooting

Have questions or issues to report? Please open a new issue after first verifying that the same question or issue has not already been reported. In the latter case, please add any additional comments you may have, to the existing issue.

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [e-mail] with any additional questions or comments.

Trademarks

This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft's Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party's policies.