SQL Server 2016: Real-Time Operational Analytics
SQL Server 2016 introduces updatable nonclustered columnstore indexes. The complexity of that phrase should not mask the simplicity and importance of its application. Updatable nonclustered columnstore indexes make possible what Microsoft calls “Real-Time Operational Analytics”. It would be hard to overstate the importance of making real-time transactional data available immediately for data mining and advanced […]
What Are Traffic Analysis and Metadata?
In Learning Tree’s System and Network Security Introduction we discuss “traffic analysis,” noting that even if data are encrypted, one can still find out information by looking at who is sending encrypted data to whom. Along that same line, there has been a lot of discussion in the press recently about “metadata” – information about […]
How Big is Big Data?
One of the questions I’m often asked is “How big does my data have to be before I need to start using big data tooling?” There’s no right answer to this—but that’s largely because it’s not the right question. Big Data is actually a pretty unhelpful term. It focuses attention exclusively on the volume of […]
How to Predict Outcomes Using Random Forests and Spark
Random forests are an ensemble, or model of models, machine learning approach. The algorithm builds multiple decision trees, based on different subsets of the features in the data. Outcomes are then predicted by running observations through all the trees and averaging the individual predictions. Think wisdom of crowds. Spark’s machine learning library, MLlib, has support […]
Paying by Numbers—Should Data Scientists Receive Performance Based Compensation?
A recent article suggests that, in the “near future”, data analysts will be compensated based on performance. They will receive commission-based payments, rather like salesmen, rather that being paid purely for their time. This performance will presumably be determined by the impact that the data analyst has on the key goals of the organization, e.g. […]