The Internet of Things and Its Challenges
The Internet permeates all areas of life. Beyond phone and web activity, modern consumer appliances, home sensors, vehicle engines, medical wearables, and billions of other devices are providing data through the Internet.
Rapidly aggregating and accurately analyzing this data is critical to making people healthy, systems secure, commerce profitable, and machines efficient. The same is true for the increasing volumes of data generated by machine and application logs which may or may not flow through the internet in real-time.
To produce actionable information from all this data, you need secure, efficient, and reliable ways to process and apply that data so it works for you, and not against you. But take a look at the landscape of tools and techniques applied to these problems today. Consider their speed in volume, reliance on big hardware, design complexity, security holes, and cost.
Aggregate on the Edge
Edge computing entails gathering and examining data while it is being generated, as close to its source as possible. In edge analytics, data is prepared and pre-analyzed on the devices, rather than being sent directly and unchanged to a centralized data store for later use.
According to the definition of event stream processing in this Big Data Insights article, such real-time reporting on the edge (before it hits the hub), can speed up time to action and save lives.
"Edge aggregation and/or analytics can decrease latency in the decision-making process. Using data closer to its source can be faster, easier, and more practical than traditional methods, and much of the data produced by IoT devices is not worth storing at all."
- Shawn Rodgers, Quest Software
Storing unnecessary data is costly and complex and introduces security risks. Processing data on the edge means less data needs to be collected, stored, or re-processed en masse later. Also, since edge analytics can occur in real-time, some insights can be produced immediately.
The lightweight, high-speed SortCL program in IRI CoSort is the perfect tool for filtering, PII-masking, aggregating, segmenting, reporting on, and otherwise transforming data on edge devices. The data sets it produces are lean, clean, and compliant, and easily distributed to various platforms for further use or processing, like the IRI Voracity data management platform, which supports CoSort, Hadoop, and multiple analytic and visualization options.
Analyze in the Hub
The IRI Workbench GUI for Voracity, built on Eclipse™, supports data processing and presentation in the same pane of glass. After setting up connections to your data stores and aggregation jobs, wherever they happen to be, you can further process and analyze the data at unprecedented speed using built-in statistical and reporting functions in Voracity, or wrangle (integrate, transform, anonymize, reformat, and hand-off) it for the analytic and visualization tools that you prefer.
IRI Voracity can aggregate device information streaming through MQTT and display it immediately in the free BIRT plug-in in the same Eclipse pane of glass. Voracity can also now also feed prepared data to open source KNIME Analytic Platform nodes in Eclipse, in real-time or batch, for machine learning and other data science projects that improve predictive models and outcomes.
Voracity allows a single process for the capture, basic analysis, and integration of IoT data to both the model development and execution environments, defined and managed from a single platform.
- Dr. Barry Devlin, 9sight Consulting
Learn more about IRI's IoT capabilities in our Introduction to IoT & MQTT and Aggregation on the Edge articles.
