Business Context and Aim:
In this POC, we are addressing a scenario where health monitoring devices are sending real-time patients’ health data to the cloud for analysis and storage. The aim is to showcase how Azure services can be utilized to collect, transform, and store this data securely and efficiently.
Let’s go through the detailed steps of implementing Azure Stream Analytics Proof of Concept (POC) for sending patients’ health data from an IoT device to an Azure SQL Database.
Architecture of Real Time Streaming using Azure Stream Analytics:
Step 1: Set up Azure Resources
1. Create an Azure IoT Hub:
The IoT Hub will serve as the central hub for receiving data from IoT devices and managing their communication.
- Go to Azure Portal
- Click on “+ Create a resource”, then search for “IoT Hub” and create a new IoT Hub.
- Configure settings such as name, pricing tier, resource group, region, etc.
- We have created an IOT Hub named “iothubstream1”.
2. Register an IoT Device:
Registering IoT devices allows secure and authorized communication with the IoT Hub.
- Inside the IoT Hub, navigate to “IoT devices”.
- Click on “+ New” to register a new IoT device.
- Provide a device ID and optionally set authentication type.
- We have registered an IoT device named “Patient-data-iot-P1”.
- Note down the “Connection string – primary key” from the device’s settings.
3. Create an Azure SQL Database:
Storing patients’ health data in a structured manner for future analysis.
- Create a new Azure SQL Database.
- Configure settings like server, database name, pricing tier, etc.
- While Creating SQL database we can create a new Server as well as use our existing SQL server.
- We will go with the basic tier so we can reduce costs.
4. Set up Azure Stream Analytics:
Process and transform real-time data as it flows through the system.
- Create a new Azure Stream Analytics job.
- Configure settings such as job name, resource group, location, etc.
- Choose an appropriate timestamp for event ordering.
- Configure input, output, and query as described in the following steps.
- We have created a Stream Analytics job named “StreamAnalytics_01”.
Step 2: Python Code for Generating Patients Health Data
1. Install Required Libraries
Develop a simulation of IoT device data generation.
- Open a command prompt or terminal.
- Run: pip install azure-iot-device.
2. Generate Patients Health Data
Simulate the health data generated by IoT devices.
- Write a Python script to simulate health data generation.
- Use libraries like random or faker to create realistic data.
- Instantiate a DeviceClient using the device connection string.
- Create a JSON payload containing health data (e.g., temperature, heart rate) and use the send_event method to send data.
3. Send Data to IoT Hub
Emulate IoT device communication with the IoT Hub.
- Import necessary modules: from azure.iot.device import IoTHubDeviceClient, Message.
- Create an instance of IoTHubDeviceClient using the device connection string.
- Use the create_from_connection_string method to establish the connection.
- Create a Message object containing your health data in JSON format.
- Use the send_message method to send the data to the IoT Hub.
- Here is the Python code which will generate the Patients Health data and send this data to IoT Hub.
Step 3: Configure Azure Stream Analytics
1. Input Configuration:
Ingest data from the IoT Hub for further processing.
- In your Stream Analytics job, click on “Inputs” and then “+ Add stream input”.
- Choose “IoT Hub” as the source.
- Provide the IoT Hub connection string and consumer group (if required).
2. Output Configuration:
Store transformed data in a structured format for analysis.
- Click on “Outputs” and then “+ Add” to create a new output.
- Choose “SQL Database” as the sink.
- Provide the SQL Database connection string and specify a table where the data will be stored.
- To specify a table first you should create a table in the SQL database.
- Map the output fields to the corresponding columns in the SQL table.
3. SQL Query Configuration:
Process and transform incoming data as per business requirements.
- Click on “Query” to write the SQL query that transforms incoming data.
- Use the SELECT INTO statement to specify the fields to be selected and the output table.
- You can apply filtering, aggregation, and formatting operations as needed.
Step 4: Start the Stream Analytics Job
1. Start the Job:
Begin the data processing pipeline.
- Click on “Start” to begin the Stream Analytics job.
- The job will start processing data from the IoT Hub according to your query logic.
2. Monitor Job Execution:
Ensure the data transformation is happening as expected.
- Monitor the job’s progress in the Stream Analytics portal.
- Check for any errors or issues in the job logs.
- Adjust your query or configurations if needed.
Step 5: Verification
Confirm that data is stored accurately for future analysis.
1. Verify Data in SQL Database:
- Connect to your Azure SQL Database using a tool like Azure Data Studio.
- Run a SELECT query on the table where the data is stored to ensure it’s being populated correctly.
Business Outcome:
Through this POC, we’ve demonstrated the ability to collect health data from IoT devices, process it in real-time using Azure Stream Analytics, and store it securely in an Azure SQL Database. This showcases the power of Azure services in healthcare applications, ensuring data accuracy and security while paving the way for more advanced health monitoring solutions.