The consolidation within the BI market continues, this time with the purchase of Applix by Cognos. As Timo Elliott points out, the interesting bit is the Applix TM1 memory-centric OLAP product. For the vast majority of OLAP users (i.e. the millions of Excel Pivot table jockeys) in-memory OLAP is nothing new, but traditionally big-ticket OLAP was disk-centric, be that ROLAP star-schemas or MOLAP products like Essbase. Until recently, in-memory was a non-runner for large datasets as the cost of memory was high and more importantly 32bit machine programs could only map up to 2G (or with a bit of trickery,3G) of the stuff. But with the appearance of cheap 64-bit servers , and even cheaper RAM, the memory-mapping of large datasets is now possible. While the focus of the big guns in BI turns towards 64-bit servers, the cheapness of RAM makes even a 32-bit machine (i.e. the majority of business PCs) useful for in-memory OLAP especially for SMEs or departmental data-marts – 2 or 3 Gigs of MOLAP data is a lot of data.
This is where tools such as the open source PALO MOLAP server come in to play. Although Palo is both 32 and 64 bit enabled, its status as a free open-source tool, its ability to run client-side, and its close integration with Excel, makes it ideal for adoption in those smaller organisations that previously would not have considered OLAP technologies.
But it’s not just MOLAP that can benefit from cheap client-side RAM, loading data into in-memory relational databases such as SQLite‘s :memory: can also be useful, particularly when that memory is shared with Excel. I’ve already experimented with embedding SQLite into Proto; as Proto’s scripting language is VBA I based it on code I had used in the past for integrating SQLite into Excel. I currently use SQLite/Excel as a micro-ETL tool and I’m looking at the possibility of front-ending a SQLite star-schema with Excel, in effect, a micro-BI environment.