Towards Use And Reuse Driven Big Data Management

Xie, Zhiwu; Chen, Yinlin; Griffin, Julie; Walters, Tyler; Tarazaga, Pablo Alberto; Kasarda, Mary

Towards Use And Reuse Driven Big Data Management

dc.contributor.author	Xie, Zhiwu	en
dc.contributor.author	Chen, Yinlin	en
dc.contributor.author	Griffin, Julie	en
dc.contributor.author	Walters, Tyler	en
dc.contributor.author	Tarazaga, Pablo Alberto	en
dc.contributor.author	Kasarda, Mary	en
dc.contributor.department	University Libraries	en
dc.contributor.department	Mechanical Engineering	en
dc.date.accessioned	2015-06-03T09:50:57Z	en
dc.date.available	2015-06-03T09:50:57Z	en
dc.date.issued	2015-06-03	en
dc.description.abstract	We propose a use and reuse driven big data management approach that fuses the data repository and data processing capabilities in a co-located, public cloud. It answers to the urgent data management needs from the growing number of researchers who don’t fit in the big science/small science dichotomy. This approach will allow researchers to more easily use, manage, and collaborate around big data sets, as well as give librarians the opportunity to work alongside the researchers to preserve and curate data while it is still fresh and being actively used. This also provides the technological foundation to foster a sharing culture more aligned with the open source software development paradigm than the lone-wolf, gift-exchanging small science sharing or the top-down, highly structured big science sharing. To materialize this vision, we provide a system architecture consisting of a scalable digital repository system coupled with the co-located cloud storage and cloud computing, as well as a job scheduler and a deployment management system. Motivated by Virginia Tech’s Goodwin Hall instrumentation project, we implemented and evaluated a prototype. The results show not only sufficient capacities for this particular case, but also near perfect linear storage and data processing scalabilities under moderately high workload.	en
dc.description.version	pre-print	en
dc.identifier.doi	https://doi.org/10.1145/2756406.2756924	en
dc.identifier.uri	http://hdl.handle.net/10919/51621	en
dc.language.iso	en_US	en
dc.relation.hasversion	http://dl.acm.org/citation.cfm?id=2756924	en
dc.rights	Creative Commons Attribution-ShareAlike 3.0 United States	en
dc.rights.uri	http://creativecommons.org/licenses/by-sa/3.0/us/	en
dc.subject	Big data	en
dc.subject	Digital library	en
dc.subject	Cloud computing	en
dc.subject	Digital repository	en
dc.subject	Smart infrastructure	en
dc.subject	Sensor data	en
dc.title	Towards Use And Reuse Driven Big Data Management	en
dc.type	Conference proceeding	en
dc.type	Presentation	en