Technology Talk

HOPS, or Heuristic Optimized Processing System technology, is at the heart of the Reveal line of products - an amazing data analysis tool 25 years in the making. The result is one of the fastest data analysis tools ever produced without the scalability limitations of traditional database solutions. The speed, compressibility and sorting capabilities of the HOPS technology is unparalleled in the market today.

The HOPS technology is able to analyze data in near linear fashion, helping our clients gain valuable insight into their data in real-time.

Traditionally, the time-per-row of data being processed by a database increases exponentially as the number of rows increases, creating scalability limitations to how quickly data can be analyzed. Traditional bottlenecks such as I/O and sorting have been addressed within the HOPS database in unique ways, such as parallelism, a near-linear sort technique, and data compression. As a result, HOPS technology allows for near linear data processing, eliminating the challenges with analyzing large data sets.

In a recent test, the HOPS technology was able to perform a full, all-columns scan of 70 million rows in just 13 seconds! This is done through the creation of specific algorithms designed to fine tune the data for processing. The three key algorithms are for compression, sorting and indexing.

The HOPS compression algorithm has the ability to compress and decompress data on-the-fly, eliminating the massive storage requirements of traditional systems. Ranging from an average of 5-to-1 up to 20-to-1, the compression algorithm allows HOPS to securely house more than half a Petabyte of data - the equivalent of 115 paper files for every man, woman and child in the US - in a space smaller than 500 square feet. The compression and decompression is transparent to the users and allows the HOPS database to make the most effective use possible of the valuable hardware.

HOPS has powerful sorting and indexing technology. The HOPS sorting algorithm is as fast as or faster than any sorting product in the industry. It is built into the HOPS database kernel, providing tremendous performance advantages in many aspects of data processing. The HOPS indexing algorithm utilizes the parallel-processing of scanning and sorting to create indexes quickly, which provide high-performance optimization opportunities for database queries.

The efficiency of having data sorted and indexed within the HOPS database and compressed to take less storage space saves you time and money! It also allows data to get in and back out faster, helping you analyze information sooner, which brings bottom line results to your business.

By combining these key processing algorithms together with the many other innovative HOPS capabilities, you have the ability to analyze not just one department's data, but all of the data for the entire company easily and painlessly from a single desktop computer. All of this gives you an ease of doing business that has never been achieved before - unlocking the value of your information so you can finally see the big picture of your business - all at a fraction of the cost of a traditional data analysis solution.

HOPS utilizes many techniques to bring you information faster

The HOPS database, configured for all types of analytic processing, employs a multi-dimensional data model allowing for complex analytical and ad-hoc queries with a rapid execution time.

Seed, a flexible alternative to SQL, encapsulates complex business rules and logic into a series of smaller steps that are easy to create and manipulate. The HOPS ProSeed tool provides a simple point-and-click application that is ideal for the creation of complex queries required by the financial services and health care industries.

Part of the HOPS Seed paradigm is a concept known as the Unified Join, which allows any number of tables to be joined together simply by specifying several attributes for each table. The Unified Join handles all ANSI-defined joins, plus others that are not defined.

Another advantage of HOPS is the ease with which large intermediate files can be handled, thus allowing for the creation of many algorithms that would be difficult to impossible with a traditional database. The speed with which HOPS operates makes it ideal to test data against theories and hypotheses, to do "what-if" scenarios and to tune an algorithm based on the data and patterns that appear.

Super Data Merge (SDM) allows for adds/edits to large amounts of source data to large destination tables. HOPS can insert tens of thousands of rows of new or modified data into a 1+ billion row table in seconds.

.Net Data Provider allows for open access to the database from standard tools.

Fuzzy Logic, is built into the HOPS database, allowing relational reasoning rather than just true/false options, so you can gain more insight from your data. Fuzzy logic is a mathematical logic that allows a range of values from "true" to "false" used in decision-making with imprecise data. It provides logic that "adapts" to the characteristics of the data being analyzed and avoids the problem of hard coded thresholds that may apply today but not tomorrow. An example is relative terms such as "tall" or "short" or other data where clear true/false does not apply. The HOPS database can search for information based on these "fuzzy logic" parameters.

• The HOPS technology has identified more than $200 million in overpayment exposure for the healthcare industry, running algorithms on more than 1.4 billion rows of data.
• HOPS currently does financial reporting to tens of thousands of merchants daily, performing loads and processing on more than 3.6 billion financial records a day.