Contact Us
2890 Zanker Road, Suite 200, San Jose, CA – 95134
[email protected] (Inquiry)
CALL US 24/7:
+1 510 358 4310

Latest Blogs

  • Break The Connectivity Barriers with Wi-Fi 7
  • Intelligent Health Environment
  • AI-Driven Quality Engineering
  • Complete Guide to Develop a Home Automation App
  • Machine to Machine Communications
  • Safe RTOS

Data Engineering on Unstructured Dataset using AWS for US-Based Home Automation Company

Banner Cloud Data Engineering Home Automation

Client Overview

US based OEM producing HVAC equipment, water heaters and boilers for residential and commercial buildings.

Business Need

  • The client has 120K+ live devices that send 40GB data per day in an unstructured format and it keeps on growing
  • They wanted a scalable, secure and cost-effective solution with flexible architecture and intelligence to analyze such a large data

VOLANSYS Contribution

  • Secure, scalable and flexible architecture having Serverless compute and storage
  • Data collection, Extract-Transform-Load (ETL) and Data pipeline
  • Data catalog and Database management
  • Project based implementation with Infrastructure as Code (IaC)
  • CloudFormation script for Infrastructure management
  • CICD pipeline using GitHub Action
  • Data collection script to transfer MongoDb data to S3
  • Developed ETL script to convert unstructured data to structured format
  • Dashboard development to display and monitor field devices data
  • SSO authentication
  • Analyze and visualize data

Solution Diagram

Benefits Delivered

  • Enabled faster data transformation by dividing day execution to hour execution for time series data
  • Designed pipelines to process data for 1 year that continuously delivered meaningful insights to client

EXT:- 261

Success Stories