Tagg: Big Data

Imperative vs functional programming. It’s a debate that goes back to the birth of high-level languages—Fortran vs Lisp. In later years, it was retreaded as object-oriented vs function programming (OOP vs FP)—OOP having become the (massively) dominant software development paradigm. And, I’m a fully paid-up member. I embraced Object Pascal via Delphi 1 on 1995 […]

As interest in big data has exploded, organizations have rushed to grab competitive advantage by deploying analytics pipelines that exploit this newly available resource. Many projects have been set up in a “skunkworks” environment, often by data science teams. While this has accelerated the time to market for new features, it has created a potential […]

SQL Server 2016 introduces updatable nonclustered columnstore indexes. The complexity of that phrase should not mask the simplicity and importance of its application. Updatable nonclustered columnstore indexes make possible what Microsoft calls “Real-Time Operational Analytics”. It would be hard to overstate the importance of making real-time transactional data available immediately for data mining and advanced […]

In Learning Tree’s System and Network Security Introduction we discuss “traffic analysis,” noting that even if data are encrypted, one can still find out information by looking at who is sending encrypted data to whom. Along that same line, there has been a lot of discussion in the press recently about “metadata” – information about […]

The release of Spark 1.5 increased support for using data frames with MLLib—Spark’s machine learning library. MLlib now divides into two packages spark.mllib which contains the original API built on top of RDDs spark.ml which provides a higher-level API built on top of DataFrames for constructing machine learning pipelines While the spark.mllib package will continue […]