When Is Real Time Reporting Actually Real Time?

01 May When Is Real Time Reporting Actually Real Time?

The use of the term “Real Time” has become all pervasive and encompassing  in the lexicon  of the Big Data world it has lost its meaning. In truth, all Hadoop Platforms ingest structured and unstructured data in real time. However when it come to visualization, query manipulation and access to the resulting analytic, report or dashboard, a parallel real time platform or stand alone version is required to deliver the holy grail of true real time analytics.

 In the absence of a real time platform, traditional batch processes are used to query data sets which can be displayed in near real time or which ever number you are in the processing queue. In addition, real time platforms allow for analysis on the fly, even the ability to change queries and merge data sets from other sources all for presentation all in real time.

 Batch data processing is an efficient way of processing high volumes of data is where a group of transactions is collected over a period of time. Data is collected, entered, processed and then the batch results are produced (Hadoop is focused on batch data processing). Batch processing requires separate programs for input, process and output. An example is payroll and billing systems.

 In contrast, real time data processing involves a continual input, process and output of data. Data must be processed in a small time period (in seconds/ near real time) or instantaneously ( Nano/mill second) Radar systems, customer services and bank ATMs are examples.

 While most organizations use batch data processing, sometimes an organization needs real time data processing. Real time data processing and analytics allows an organization the ability to take immediate action for those times when acting within seconds or minutes is significant. The goal is to obtain the insight required to act prudently at the right time – which increasingly means immediately.

 Complex event processing (CEP) combines data from multiple sources to detect patterns and attempt to identify either opportunities or threats. The goal is to identify significant events and respond fast. Sales leads, orders or customer service calls are examples.

 Operational Intelligence (OI) uses real time data processing and CEP to gain insight into operations by running query analysis against live feeds and event data. OI is near real time analytics over operational data and provides visibility over many data sources. The goal is to obtain near real time insight using continuous analytics to allow the organization to take immediate action. Real time analytics delivers the right offer at the right price at the right time. 5% increase in customer retention can lead to a 25-100% increase in profit.

 Contrast this with operational business intelligence (BI) –  descriptive or historical analysis of operational data. OI real time analysis of operational data has much greater value. Real time decision making is a new competitive advantage. It means you will never hear problem issues first from your customers. If your are familiar with Service Level Agreements (SLA), real time actionable management can reduce SLA mean time to repair from hours to minutes… virtually eliminating operational risk and financial penalties and loss of reputation associated with a missed SLA.