Coffing Data Warehousing Software

The Nexus Query Chameleon and the Hub

The great master Confucius once said, “It does not matter how slowly you go as long as you do not stop”.  Putting one step forward is an essential key to success and it is how things evolve. Nowhere is this more prevalent than in the computer industry.

For many years, most people queried a single system. Eventually, some people began to query multiple systems, but think about how you perform searches on the Internet.  You don’t connect to a single database or multiple databases.  You get data from thousands of servers.  All that matters is that you make a request and the data is returned in a timely manner.

That is the future of how queries will be processed. And the future is now!

I have hundreds of customers doing this very thing!  Here is another one of my inventions and I call it “The Hub”.  I got the idea from watching how Airlines manage their planes and flights.  This is an essential part of the Nexus Chameleon strategy.

The Nexus allows users to see tables and views from all accessible systems and join data across many different platforms.  For example, in a single query I ran a join with tables from Teradata, Oracle, SQL Server, DB2, and an Excel spreadsheet, and I made the hub my Teradata system.

By making the hub Teradata, this causes the Oracle, SQL Server, DB2, and the Excel spreadsheet to be automatically moved temporarily to Teradata where the join is performed.  Once the report is delivered, the temporary tables are dropped.  As a test, I did it again but I made the Hub SQL Server, and all of the other tables plus the Excel spreadsheet moved to SQL Server to process the join.  The Nexus automatically changes the SQL to T-SQL and the load utilities change from Teradata TPT to Bulk Copy.

Of course, I was able to do this repeatedly by changing the hub to DB2 and then Oracle.  I even changed the Hub to “My PC” where all of the tables were queried separately, including the Excel Spreadsheet and in the background of my PC the join was completed.  Choosing “My PC” for the hub is absolutely the fastest and least expensive resource for tables with less than 1,000,000 rows.  The user gets the same results no matter what system is the hub.  The key point is to process data where it makes the best sense and where it uses the least amount of resources, plus the flexibility is priceless.

Plus, the Nexus always tracks the size and row count of each table so users can process a query on the best system or least busy system.  For example, for any joins where there are less than 1,000,000 rows, it is best to automatically perform the joins with “My PC” as the hub.  This is lightning quick, and it saves resources on the data warehouse and OLTP systems.

This is a key to cloud computing because a cross-system join using tables from 10 different systems can choose to make the hub any cloud system.  If resources are tight at the end of the quarter or year, a company can spin-up a cloud system and process the queries there.  Just like the Internet, it does not matter where or how the data is processed, but that users get the report back in a timely manner and the company saves enormous money by having the flexibility to process the query where they choose.

Confucius was right!  It took us 10 years to be able to achieve this for our Nexus customers, but we moved slowly, methodically, and never stopped.