Azure Event Hub Consumer Client to Index events with Azure Cognitive Search

This is a sample Event Hub consumer client code to demonstrate the ability to index event data with Azure Cognitive Search by using an Event Hub Consumer Client. This sample uses Python SDK for Event Hub and Azure Cognitive Search.

Pre-work

Python 3.6 or later
Setup and Event Hub Namespace and an Event Hub.
Setup the Azure Cognitive Search service.

Setup the Search Index

Create an index with the following fields and the index schema.

id
title
summary

Setup the Environment

Install Python Packages

Install Azure Event Hub packages

pip install azure-eventhub
pip install azure-eventhub-checkpointstoreblob-aio

Install Azure Cognitive Search packages

pip install azure-search-documents

Setup Variables

Set the Event Hub connection information environment variables.

export EVENT_HUB_CONN_STR="<Your Event Hub Namespace Connection String>"
export EVENT_HUB_NAME="<Your Event Hub Name>"

NOTE: The default consumer group "$Default" is used by the sample.

Setup the Cognitive Search connection information environment variables.

export SEARCH_ENDPOINT="<Your Azure Cognitive Search Endpoint>"
export SEARCH_API_KEY="<Your Search API Key>"
export SEARCH_INDEX_NAME="<Your Search Index Name>"

Run the Sample Code

Run the Receiver

To test that you can receive events from the Event Hub without sending data to Azure Cognitive Search run the "receiver.py"

Run the receiver in a terminal that indexes data to Azure Cognitive Search.

python3 receiver-acs.py

Run the Sender

In a separate terminal window run the Event Hub sender code.

python3 send.py

When the sender runs, there should be an indication on the receiver terminal that data was uploaded successfully to Azure Cognitive Search.

Go to the Azure Cognitive Search service and to the index you created and in the Search Explorer click "Search". You should see the data sent to Event Hub will be indexed in Azure Cognitive Search.

Event Hub Samples

Azure Cognitive Search Samples

Disclaimer: These samples are not production ready code. These samples do not have the apprpriate unit tests and exception handling needed for production code.

fsaleemm / eventhub-acs