Abstract

Data analytics is the crucial step to reveal essential values of datasets and complete the value chain of big data. In practice, both the hardware infrastructure and the software stack play a fundamental role in big data analytics (BDA). Unfortunately, it is evident that a disproportionately larger amount of effort is being invested in the hardware infrastructure development over the software stack development. Given our concern about a software crisis brewing in the big data ecosystem, we argue that it is time to further strengthen and expand the role of software in BDA implementations. This special issue is then aimed to create a common ground and a reference point for both researchers and practitioners from multiple disciplines to discuss the rigor, relevance, experience and challenges of software-driven BDA as an emerging domain. We also expect to use this special issue to attract more attention and efforts to tighten communication and collaboration between the software engineering community and the data science community.