The use of Big Data continues to get headlines. At times its use is controversial but regardless, Big Data is here to stay. Here is some data, reported by eWeek, to support that claim. IDC reveals in its 2013 Worldwide Study of HPC End-User Sites that two-thirds of high-performance computing sites are conducting big data analysis as part of their HPC workloads. The research also finds that HPC sites using co-processors and accelerators more than doubled during the past two years.
The demand to handle Biig Data is driving several industries including hardware, as well as analysts and software. The proportion of IT sites using co-processors or accelerators in their HPC systems climbed from 28.2 percent in the 2011 to 76.9 percent in 2013. This is quite significant. Earl Joseph, IDC's program vice president for Technical Computing is quoted, "The most surprising finding of the 2013 study is the substantially increased penetration of co-processors and accelerators at HPC sites around the world, along with the large proportion of sites that are applying big data technologies and methods to their problems."
The report also stated that, “an unexpected finding reveals that 67 percent of the HPC sites perform big data analysis on their HPC systems, with 30 percent of the available computing cycles devoted on average to big data analysis.” This does not seem surprising to me given the thirst for Big Data by industry and governments.
I wrote about the growth of cloud services earlier this week. This applies to Big Data as well. The report also indicates that the HPC sites using cloud computing to handle part of the HPC workloads climbed from 13.8 percent in 2011 to 23.5 percent in 2013. Public and private cloud use was split among the sites in 2013. Hardware vendors are benefiting from this surge. IDC said the study confirmed its supply-side research finding that storage is the fastest-growing technology area at HPC sites.