active networks Advertising batch workloads big data cloud computing Clustering algorithms Computational modeling Context-awareness corba dynamic configuration Electronic Commerce Hadoop hdfs MapReduce Middleware Mobile communication monitoring Multimedia distribution operating systems Organizations Pervasive Comput SDN networks Fabric Security security policies SNMP Software storage ubiquitous computing Upper bound verification
2016 |
Campbell, Chris Cai Franck Le Xin Sun Geoffrey Xie Hanii Jamjoom Roy CRONets: Cloud-Routed Overlay Networks (Conference) I36th IEEE International Conference on Distributed Computer Systems (ICDCS 2016), IEEE, Nara, Japan, 2016. (Abstract | BibTeX | Tags: networks overlays cloud routing) @conference{Cai2016, title = {CRONets: Cloud-Routed Overlay Networks}, author = {Chris X. Cai Franck Le Xin Sun Geoffrey Xie Hanii Jamjoom Roy H. Campbell}, year = {2016}, date = {2016-06-27}, booktitle = {I36th IEEE International Conference on Distributed Computer Systems (ICDCS 2016)}, pages = {12}, publisher = {IEEE}, address = {Nara, Japan}, abstract = {Overlay networking and ISP-assisted tunneling are effective solutions to overcome problematic BGP routes and bypass troublesome autonomous systems. Despite their demonstrated effectiveness, overlay support is not broadly available. In this paper, we propose Cloud-Routed Overlay Networks (CRONets), whereby users can readily build their own overlays using nodes from global and well-provisioned cloud providers like IBM Softlayer or Amazon EC2. While previous studies have demonstrated the benefits of overlay networks with the high-speed experimental Internet2 backbone, we are the first to evaluate the improvements in a realistic—cloud—setting. We conduct a large-scale experiment where we observe 6,600 Internet paths. The results show that CRONets improve the throughput for 78% of the default Internet paths with a median improvement factor of 3.26 times, at a tenth of the cost of leasing private lines of comparable performance. We also performed a longitudinal measurement, and demonstrate that the performance gains are consistent over time with only a small number of overlay nodes needed to be deployed. However, given the size and dynamic nature of the Internet routing system (e.g., due to congestion and failures), selecting the proper path is still a challenging problem. To address it, we propose a novel solution based on the newly introduced MPTCP extensions. Our experiments show that MPTCP can achieve the maximum observed throughput across the different overlay paths. Yet, for some instances, MPTCP performs up to 38% lower than expected indicating room to improve the MPTCP congestion control algorithms.}, keywords = {networks overlays cloud routing}, pubstate = {published}, tppubtype = {conference} } Overlay networking and ISP-assisted tunneling are effective solutions to overcome problematic BGP routes and bypass troublesome autonomous systems. Despite their demonstrated effectiveness, overlay support is not broadly available. In this paper, we propose Cloud-Routed Overlay Networks (CRONets), whereby users can readily build their own overlays using nodes from global and well-provisioned cloud providers like IBM Softlayer or Amazon EC2. While previous studies have demonstrated the benefits of overlay networks with the high-speed experimental Internet2 backbone, we are the first to evaluate the improvements in a realistic—cloud—setting. We conduct a large-scale experiment where we observe 6,600 Internet paths. The results show that CRONets improve the throughput for 78% of the default Internet paths with a median improvement factor of 3.26 times, at a tenth of the cost of leasing private lines of comparable performance. We also performed a longitudinal measurement, and demonstrate that the performance gains are consistent over time with only a small number of overlay nodes needed to be deployed. However, given the size and dynamic nature of the Internet routing system (e.g., due to congestion and failures), selecting the proper path is still a challenging problem. To address it, we propose a novel solution based on the newly introduced MPTCP extensions. Our experiments show that MPTCP can achieve the maximum observed throughput across the different overlay paths. Yet, for some instances, MPTCP performs up to 38% lower than expected indicating room to improve the MPTCP congestion control algorithms. |
Campbell, Shadi Noghabi Sriram Subramanian Priyesh Narayanan Sivabalan Narayanan Gopalakrishna Holla Mammad Zadeh Tianwei Li Indranil Gupta Roy Ambry: LinkedIn's Scalable Geo-distributed Object Store (Conference) ACM SIGMOD/PODS Conference, ACM, San Francisco, USA, 2016. (Abstract | BibTeX | Tags: geo-distributed object store) @conference{Shadi2016, title = {Ambry: LinkedIn's Scalable Geo-distributed Object Store}, author = {Shadi A. Noghabi Sriram Subramanian Priyesh Narayanan Sivabalan Narayanan Gopalakrishna Holla Mammad Zadeh Tianwei Li Indranil Gupta Roy H. Campbell}, year = {2016}, date = {2016-06-26}, booktitle = {ACM SIGMOD/PODS Conference}, publisher = {ACM}, address = {San Francisco, USA}, abstract = {The infrastructure of a globe-wide social network has to serve millions of users' massive media objects such as photos, videos, and audio continually. Creation and reading of this data must be supported with low latency, high through-put, multiple geo-distributed datacenters, and in a scalable and load-balanced way. Existing file systems and blob stores face a challenge when serving such large objects. We present Ambry, a new blob store that caters to this demand. Ambry is a production-quality system for storing large immutable data (called blobs). Ambry leverages techniques such as a decentralized design, logical blob grouping, asynchronous replication, rebalancing mechanisms, and OS caching. Ambry has been running in a production environment for 24 months, serving up to 10K requests per second across more than 400 million users. Our experimental evaluation reveals that Ambry off ers high-efficiency (it utilizes up to 88% of the network bandwidth), responsiveness with low latency, and load balancing (improves imbalance of request rates and disk usage among the cluster by 8x-10x).}, keywords = {geo-distributed object store}, pubstate = {published}, tppubtype = {conference} } The infrastructure of a globe-wide social network has to serve millions of users' massive media objects such as photos, videos, and audio continually. Creation and reading of this data must be supported with low latency, high through-put, multiple geo-distributed datacenters, and in a scalable and load-balanced way. Existing file systems and blob stores face a challenge when serving such large objects. We present Ambry, a new blob store that caters to this demand. Ambry is a production-quality system for storing large immutable data (called blobs). Ambry leverages techniques such as a decentralized design, logical blob grouping, asynchronous replication, rebalancing mechanisms, and OS caching. Ambry has been running in a production environment for 24 months, serving up to 10K requests per second across more than 400 million users. Our experimental evaluation reveals that Ambry off ers high-efficiency (it utilizes up to 88% of the network bandwidth), responsiveness with low latency, and load balancing (improves imbalance of request rates and disk usage among the cluster by 8x-10x). |
Campbell, Sayed Hadi Hashemi Faraz Faghri Paul Rausch Roy World of Empowered IoT Users (Conference) The 1st IEEE International Conference on Internet-of-Things Design and Implementation, IEEE, Berlin, Germany, 2016. (Abstract | BibTeX | Tags: IoT Privacy Blockchains) @conference{Hadi20016, title = {World of Empowered IoT Users}, author = {Sayed Hadi Hashemi Faraz Faghri Paul Rausch Roy H. Campbell}, year = {2016}, date = {2016-04-05}, booktitle = {The 1st IEEE International Conference on Internet-of-Things Design and Implementation}, publisher = {IEEE}, address = {Berlin, Germany}, abstract = {In a world deploying an Internet of Things, sensors and actuators are owned, accessed, and activated by a plethora of individuals and organizations. Access to the data produced by this world can both be beneficial and have drawbacks to society. This data potentially represents the activities of millions of individuals and their possessions collected by billions of things and aggregations of this data can be analyzed through the Internet and Clouds. This raises possible privacy, security, moral and ethical challenges whose solutions will require flexible protection mechanisms. How do we acquire and distribute data at the IoT world scale while retaining the rights of individuals and organizations to protect, use, and share their data? Clearly a well-defined mechanism and control needs to regulate access to the data and its aggregations. Our paper describes a user-centric multi-level multiple granularity mechanism to share the data from these devices to people and organizations. Revisiting the fundamental mechanisms in security for providing protection, our solution uses capabilities, access lists, and access rights following well-understood formal notions for reasoning about access. Our contribution is to describe an auditable, transparent, distributed, decentralized, publication-subscription based, robust mechanism and automation of these ideas in the IoT realm that is well-matched to the current generation of clouds. It is based on well-tested principles and practices used in crypto currencies exploiting block chains of transactions. The scheme puts users (including organizational entities) in the center of control over the access to their collections of sensory data. In our paper, we describe a deployment of these ideas for medical health devices, smart cities, and transportation.}, keywords = {IoT Privacy Blockchains}, pubstate = {published}, tppubtype = {conference} } In a world deploying an Internet of Things, sensors and actuators are owned, accessed, and activated by a plethora of individuals and organizations. Access to the data produced by this world can both be beneficial and have drawbacks to society. This data potentially represents the activities of millions of individuals and their possessions collected by billions of things and aggregations of this data can be analyzed through the Internet and Clouds. This raises possible privacy, security, moral and ethical challenges whose solutions will require flexible protection mechanisms. How do we acquire and distribute data at the IoT world scale while retaining the rights of individuals and organizations to protect, use, and share their data? Clearly a well-defined mechanism and control needs to regulate access to the data and its aggregations. Our paper describes a user-centric multi-level multiple granularity mechanism to share the data from these devices to people and organizations. Revisiting the fundamental mechanisms in security for providing protection, our solution uses capabilities, access lists, and access rights following well-understood formal notions for reasoning about access. Our contribution is to describe an auditable, transparent, distributed, decentralized, publication-subscription based, robust mechanism and automation of these ideas in the IoT realm that is well-matched to the current generation of clouds. It is based on well-tested principles and practices used in crypto currencies exploiting block chains of transactions. The scheme puts users (including organizational entities) in the center of control over the access to their collections of sensory data. In our paper, we describe a deployment of these ideas for medical health devices, smart cities, and transportation. |
Campbell, Mayank Pundir Manoj Kumar Luke Leslie Indranil Gupta Roy Supporting On-demand Elasticity in Distributed Graph Processing (Conference) IEEE International Conference on Cloud Engineering (IC2E), 2016. (Abstract | BibTeX | Tags: graph processing elasticity) @conference{Pundir2016b, title = {Supporting On-demand Elasticity in Distributed Graph Processing}, author = {Mayank Pundir Manoj Kumar Luke M. Leslie Indranil Gupta Roy H. Campbell}, year = {2016}, date = {2016-04-05}, booktitle = {IEEE International Conference on Cloud Engineering (IC2E)}, journal = {IEEE International Conference on Cloud Engineering (IC2E)}, abstract = {While distributed graph processing engines have become popular for processing large graphs, these engines are typically configured with a static set of servers in the cluster. In other words, they lack the flexibility to scale-out or scalein the number of servers, when requested to do so by the user. In this paper, we propose the first techniques to make distributed graph processing truly elastic. While supporting ondemand scale-out/in operations, we meet three goals: i) perform scale-out/in without interrupting the graph computation, ii) minimize the background network overhead involved in the scaleout/in, and iii) mitigate stragglers by maintaining load balance across servers. We present and analyze two techniques called Contiguous Vertex Repartitioning (CVR) and Ring-based Vertex Repartitioning (RVR) to address these goals. We implement our techniques in the LFGraph distributed graph processing system, and incorporate several systems optimizations. Experiments performed with multiple graph benchmark applications on a real graph indicate that our techniques perform within 9% and 21% of the optimum for scale-out and scale-in operations, respectively.}, keywords = {graph processing elasticity}, pubstate = {published}, tppubtype = {conference} } While distributed graph processing engines have become popular for processing large graphs, these engines are typically configured with a static set of servers in the cluster. In other words, they lack the flexibility to scale-out or scalein the number of servers, when requested to do so by the user. In this paper, we propose the first techniques to make distributed graph processing truly elastic. While supporting ondemand scale-out/in operations, we meet three goals: i) perform scale-out/in without interrupting the graph computation, ii) minimize the background network overhead involved in the scaleout/in, and iii) mitigate stragglers by maintaining load balance across servers. We present and analyze two techniques called Contiguous Vertex Repartitioning (CVR) and Ring-based Vertex Repartitioning (RVR) to address these goals. We implement our techniques in the LFGraph distributed graph processing system, and incorporate several systems optimizations. Experiments performed with multiple graph benchmark applications on a real graph indicate that our techniques perform within 9% and 21% of the optimum for scale-out and scale-in operations, respectively. |
Le, Chris Cai Shayan Saeed Indranil Gupta Roy Campbell Franck Phurti: Application and Network-Aware Flow Scheduling for Multi-Tenant MapReduce Clusters (Conference) IEEE International Conference on Cloud Engineering (IC2E), Berlin, Germany, 2016. (BibTeX | Tags: network aware flow scheduling) @conference{Cai22016, title = {Phurti: Application and Network-Aware Flow Scheduling for Multi-Tenant MapReduce Clusters}, author = {Chris X. Cai Shayan Saeed Indranil Gupta Roy H. Campbell Franck Le}, year = {2016}, date = {2016-04-05}, booktitle = {IEEE International Conference on Cloud Engineering (IC2E)}, address = {Berlin, Germany}, keywords = {network aware flow scheduling}, pubstate = {published}, tppubtype = {conference} } |
Campbell, Sayed Hadi Hashemi Shadi Noghabi John Bellessa Roy Toward Fabric: A Middleware Implementing High-level Description Languages on a Fabric-like Network (Conference) ANCS '16 Proceedings of the 2016 Symposium on Architectures for Networking and Communications Systems, ACM ACM, New York, USA, 2016, ISBN: 978-1-4503-4183-7. (Abstract | Links | BibTeX | Tags: Software Defined Networks Routing) @conference{Hadi32016, title = {Toward Fabric: A Middleware Implementing High-level Description Languages on a Fabric-like Network}, author = {Sayed Hadi Hashemi Shadi A. Noghabi John Bellessa Roy H. Campbell}, doi = {10.1145/2881025.2889487}, isbn = {978-1-4503-4183-7}, year = {2016}, date = {2016-03-17}, booktitle = {ANCS '16 Proceedings of the 2016 Symposium on Architectures for Networking and Communications Systems}, pages = {117-118}, publisher = {ACM}, address = {New York, USA}, organization = {ACM}, abstract = {Many in the networking community believe that Software-Defined Networking, in which entire networks are managed centrally, has the potential to revolutionize the field. However, SDN faces several challenges that have prevented its wide-spread adoption. Current SDN technologies, such as OpenFlow, provide powerful and flexible APIs, but can be unreasonably complex for implementing nontrivial network control logic. The generality offered by these low-level abstractions impose no structure on the network, requiring programmers to herd switches themselves, with little guidance. Many researchers argue that SDNs must adopt more structured models, such as Fabric, with an intelligent edge and a fast but simple label-switched core. Our work draws heavily from these ideas. To that end, we propose ToF, a middleware architecture for implementing policies and behaviors from high-level network descriptions on top of a Fabric-like network. We have implemented a prototype using a combination of widely used technologies, such as MPLS, and our own proposed technologies. Based on our results, we reach near linear scalability with respect to the number of addresses routed over the network, all while introducing minimal performance overhead and requiring no changes to packet structure.}, keywords = {Software Defined Networks Routing}, pubstate = {published}, tppubtype = {conference} } Many in the networking community believe that Software-Defined Networking, in which entire networks are managed centrally, has the potential to revolutionize the field. However, SDN faces several challenges that have prevented its wide-spread adoption. Current SDN technologies, such as OpenFlow, provide powerful and flexible APIs, but can be unreasonably complex for implementing nontrivial network control logic. The generality offered by these low-level abstractions impose no structure on the network, requiring programmers to herd switches themselves, with little guidance. Many researchers argue that SDNs must adopt more structured models, such as Fabric, with an intelligent edge and a fast but simple label-switched core. Our work draws heavily from these ideas. To that end, we propose ToF, a middleware architecture for implementing policies and behaviors from high-level network descriptions on top of a Fabric-like network. We have implemented a prototype using a combination of widely used technologies, such as MPLS, and our own proposed technologies. Based on our results, we reach near linear scalability with respect to the number of addresses routed over the network, all while introducing minimal performance overhead and requiring no changes to packet structure. |
2015 |
Yao, Fangzhou; Chang, Kevin; Campbell, Roy Ushio: Analyzing News Media and Public Trends in Twitter (Miscellaneous) 2015. @misc{, title = {Ushio: Analyzing News Media and Public Trends in Twitter}, author = {Yao, Fangzhou and Chang, Kevin and Campbell, Roy H}, url = {http://www.staffs.ac.uk/personal/engineering_and_technology/eb26/BDSN-2015/}, year = {2015}, date = {2015-12-01}, keywords = {}, pubstate = {published}, tppubtype = {misc} } |
Liu, Weijie; Bobba, Rakesh; Mohan, Sibin; Campbell, Roy Inter-flow consistency: A novel SDN update abstraction for supporting inter-flow constraints. (Miscellaneous) 2015. (BibTeX | Tags: ) @misc{b, title = {Inter-flow consistency: A novel SDN update abstraction for supporting inter-flow constraints.}, author = {Liu, Weijie and Bobba, Rakesh B. and Mohan, Sibin and Campbell, Roy H.}, year = {2015}, date = {2015-09-01}, pages = {469-478}, publisher = {IEEE}, keywords = {}, pubstate = {published}, tppubtype = {misc} } |
Liu, Weijie; Bobba, Rakesh; Mohan, Sibin; Campbell, Roy Inter-Flow Consistency: Novel SDN Update Abstraction for Supporting Inter-Flow Constraints (Miscellaneous) 2015. @misc{b, title = {Inter-Flow Consistency: Novel SDN Update Abstraction for Supporting Inter-Flow Constraints}, author = {Liu, Weijie and Bobba, Rakesh B. and Mohan, Sibin and Campbell, Roy H.}, url = {http://www.internetsociety.org/doc/inter-flow-consistency-novel-sdn-update-abstraction-supporting-inter-flow-constraints}, year = {2015}, date = {2015-02-01}, publisher = {Internet Society}, keywords = {}, pubstate = {published}, tppubtype = {misc} } |
Stephens, Zachary; Lee, Skylar; Faghri, Faraz; Campbell, Roy; Zhai, Chengxiang; Efron, Miles; Iyer, Ravishankar; Schatz, Michael; Sinha, Saurabh; Robinson, Gene Big Data: Astronomical or Genomical? (Journal Article) PLoS Biol, 13(7): e1002195. , 2015. @article{b, title = {Big Data: Astronomical or Genomical?}, author = {Stephens, Zachary D. and Lee, Skylar Y. and Faghri, Faraz and Campbell, Roy H. and Zhai, Chengxiang and Efron, Miles J. and Iyer, Ravishankar and Schatz, Michael C. and Sinha, Saurabh and Robinson, Gene E.}, doi = {10.1371/journal.pbio.1002195}, year = {2015}, date = {2015-01-01}, journal = {PLoS Biol}, volume = {13(7): e1002195.}, keywords = {}, pubstate = {published}, tppubtype = {article} } |
Pundir, Mayank; Leslie, Luke; Gupta, Indranil; Campbell, Roy Zorro: zero-cost reactive failure recovery in distributed graph processing (Inproceeding) SoCC'15 Proceedings of the Sixth ACM Symposium on Cloud Computing, pp. 195-208, ACM New York, NY, USA ©2015, 2015, ISBN: 978-1-4503-3651-2. @inproceedings{b, title = {Zorro: zero-cost reactive failure recovery in distributed graph processing}, author = {Pundir, Mayank and Leslie, Luke M. and Gupta, Indranil and Campbell, Roy H.}, doi = {10.1145/2806777.2806934}, isbn = {978-1-4503-3651-2}, year = {2015}, date = {2015-01-01}, booktitle = {SoCC'15 Proceedings of the Sixth ACM Symposium on Cloud Computing}, pages = {195-208}, publisher = {ACM New York, NY, USA ©2015}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } |
Peng, Boyang; Hosseini, Mohammad; Hong, Zhihao; Farivar, Reza; Campbell, Roy R-Storm: Resource-Aware Scheduling in Storm (Miscellaneous) 2015. (BibTeX | Tags: ) @misc{b, title = {R-Storm: Resource-Aware Scheduling in Storm}, author = {Peng, Boyang and Hosseini, Mohammad and Hong, Zhihao and Farivar, Reza and Campbell, Roy}, year = {2015}, date = {2015-01-01}, pages = {149-161}, keywords = {}, pubstate = {published}, tppubtype = {misc} } |
Palmer, Imani; Wood, Elaine; Nagy, Stefan; Garcia, Gabriela; Bashir, Masooda; Campbell, Roy Digital Forensics Education: A Multidisciplinary Curriculum Model (Book Chapter) James, Joshua; Breitinger, Frank (Ed.): Digital Forensics and Cyber Crime: 7th International Conference, ICDF2C 2015, 157 , pp. 3-15, Springer International Publishing, Seoul, South Korea, 2015, ISBN: 978-3-319-25512-5. @inbook{b, title = {Digital Forensics Education: A Multidisciplinary Curriculum Model}, author = {Palmer, Imani and Wood, Elaine and Nagy, Stefan and Garcia, Gabriela and Masooda Bashir and Campbell, Roy}, editor = {James, Joshua I. and Breitinger, Frank}, doi = {10.1007/978-3-319-25512-5_1}, isbn = {978-3-319-25512-5}, year = {2015}, date = {2015-01-01}, booktitle = {Digital Forensics and Cyber Crime: 7th International Conference, ICDF2C 2015}, volume = {157}, pages = {3-15}, publisher = {Springer International Publishing}, address = {Seoul, South Korea}, series = {Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering}, keywords = {}, pubstate = {published}, tppubtype = {inbook} } |
Cai, Chris; Le, Franck; Sun, Xin; Xie, Geoffrey BYOO: Build Your Own Overlay (Technical Report) 2015. @techreport{byoo, title = {BYOO: Build Your Own Overlay}, author = {Chris Cai and Franck Le and Xin Sun and Geoffrey Xie}, url = {http://srg.cs.illinois.edu/wp-content/uploads/2016/04/techReport.pdf}, year = {2015}, date = {2015-01-01}, keywords = {}, pubstate = {published}, tppubtype = {techreport} } |
2014 |
Verma, Abhishek; Cherkasova, Ludmila; Campbell, Roy Profiling and Evaluating Hardware Choices for MapReduce Environments:<br/>an Application-Aware Approach (Conference) Proc. of the 32nd Intl. Symposium on Computer Performance, Modeling, Measurements, and Evaluation, (IFIP WG 7.3 Performncetextquoteright2014), Turin, Italy, 2014. (Abstract | BibTeX | Tags: benchmarking, MapReduce, performance modeling) @conference{175, title = {Profiling and Evaluating Hardware Choices for MapReduce Environments: an Application-Aware Approach}, author = {Abhishek Verma and Ludmila Cherkasova and Roy H Campbell}, year = {2014}, date = {2014-10-01}, booktitle = {Proc. of the 32nd Intl. Symposium on Computer Performance, Modeling, Measurements, and Evaluation, (IFIP WG 7.3 Performncetextquoteright2014)}, address = {Turin, Italy}, abstract = {The core business of many companies depends on the timely analysis of large quantities of new data. MapReduce clusters that routinely process petabytes of data represent a new entity in the evolving landscape of clouds and data centers. During the lifetime of a datacenter, old hardware needs to be eventually replaced by new hardware. The hardware selection process needs to be driven by performance objectives of the existing production workloads. In this work, we present a general framework, called Ariel, that automates system administratorstextquoteright eorts for evaluating different hardware choices and predicting completion times of MapReduce applications for their migration to a Hadoop cluster based on the new hardware. The proposed framework consists of two key components: i) a set of microbenchmarks to profile the MapReduce processing pipeline on a given platform, and ii) a regression-based model that establishes a performance relationship between the source and target platforms. Benchmarking and model derivation can be done using a small test cluster based on new hardware. However, the designed model can be used for predicting the jobstextquoteright completion time on a large Hadoop cluster and be applied for its sizing to achieve desirable service level objectives (SLOs). We validate the effectiveness of the proposed approach using a set of twelve realistic MapReduce applications and three different hardware platforms. The evaluation study justifes our design choices and shows that the derived model accurately predicts performance of the test applications. The predicted completion times of eleven applications (out of twelve) are within 10% of the measured completion times on the target platforms.}, keywords = {benchmarking, MapReduce, performance modeling}, pubstate = {published}, tppubtype = {conference} } The core business of many companies depends on the timely analysis of large quantities of new data.<br/>MapReduce clusters that routinely process petabytes of data represent a new entity in the evolving landscape<br/>of clouds and data centers. During the lifetime of a datacenter, old hardware needs to be eventually replaced<br/>by new hardware. The hardware selection process needs to be driven by performance objectives of the<br/>existing production workloads. In this work, we present a general framework, called Ariel, that automates<br/>system administratorstextquoteright eorts for evaluating different hardware choices and predicting completion times of<br/>MapReduce applications for their migration to a Hadoop cluster based on the new hardware. The proposed<br/>framework consists of two key components: i) a set of microbenchmarks to profile the MapReduce processing<br/>pipeline on a given platform, and ii) a regression-based model that establishes a performance relationship<br/>between the source and target platforms. Benchmarking and model derivation can be done using a small<br/>test cluster based on new hardware. However, the designed model can be used for predicting the jobstextquoteright<br/>completion time on a large Hadoop cluster and be applied for its sizing to achieve desirable service level<br/>objectives (SLOs). We validate the effectiveness of the proposed approach using a set of twelve realistic<br/>MapReduce applications and three different hardware platforms. The evaluation study justifes our design<br/>choices and shows that the derived model accurately predicts performance of the test applications. The<br/>predicted completion times of eleven applications (out of twelve) are within 10% of the measured completion<br/>times on the target platforms. |
Yao, Fangzhou; Spraybery, Read; Campbell, Roy CryptVMI: a Flexible and Encrypted Virtual Machine Introspection in the Cloud (Conference) Proceedings of the Second International Workshop on Security in Cloud Computing, ACM New York ACM New York, Kyoto, Japan, 2014. @conference{172, title = {CryptVMI: a Flexible and Encrypted Virtual Machine Introspection in the Cloud}, author = {Fangzhou Yao and Spraybery, Read T. and Roy H Campbell}, year = {2014}, date = {2014-06-01}, booktitle = {Proceedings of the Second International Workshop on Security in Cloud Computing}, pages = {11-18}, publisher = {ACM New York}, address = {Kyoto, Japan}, organization = {ACM New York}, abstract = {Virtualization has demonstrated its importance in both public and private cloud computing solutions. In such environments, multiple virtual instances run on the same physical machine concurrently. Thus, the isolation in the system is not guaranteed by the physical infrastructure anymore. Reliance on logical isolation makes a system vulnerable to attacks. Thus, Virtual Machine Introspection techniques become essential,since they simplify the process to acquire evidence for further analysis in this complex system. However, Virtual Machine Introspection tools for the cloud are usually written specifically for a single system and do not provide a standard interface to work with other security monitoring systems. Moreover, this technique breaks down the borders of the segregation between multiple tenants, which should be avoided in a public cloud computing environment. In this paper, we focus on building a flexible and encrypted Virtual Machine Introspection system, CryptVMI, to address the above concerns. Our approach maintains a client application on the user end to send queries to the cloud, as well as parse the results returned in a standard form. We also have a handler that cooperates with an introspection application in the cloud infrastructure to process queries and return encrypted results. This work shows our design and implementation of this system, and the benchmark results prove that it does not incur much performance overhead.}, keywords = {}, pubstate = {published}, tppubtype = {conference} } Virtualization has demonstrated its importance in both public and private cloud computing solutions. In such environments, multiple virtual instances run on the same physical machine concurrently. Thus, the isolation in the system is not guaranteed by the physical infrastructure anymore. Reliance on logical isolation makes a system vulnerable to attacks. Thus, Virtual Machine Introspection techniques become essential,since they simplify the process to acquire evidence for further analysis in this complex system. However, Virtual Machine Introspection tools for the cloud are usually written specifically for a single system and do not provide a standard interface to work with other security monitoring systems. Moreover, this technique breaks down the borders of the segregation between multiple tenants, which should be avoided in a public cloud computing environment. In this paper, we focus on building a flexible and encrypted Virtual Machine Introspection system, CryptVMI, to address the above concerns. Our approach maintains a client application on the user end to send queries to the cloud, as well as parse the results returned in a standard form. We also have a handler that cooperates with an introspection application in the cloud infrastructure to process queries and return encrypted results. This work shows our design and implementation of this system, and the benchmark results prove that it does not incur much performance overhead. |
Huang, Jingwei; Nicol, David; Campbell, Roy Denial-of-Service Threat to Hadoop/YARN Clusters with Multi-Tenancy (Conference) 2014 IEEE Second international Congress on Big Data (BigData Congress 2014), IEEE IEEE, Anchorage,AK,USA, 2014. (BibTeX | Tags: ) @conference{176, title = {Denial-of-Service Threat to Hadoop/YARN Clusters with Multi-Tenancy}, author = {Jingwei Huang and David M. Nicol and Roy H. Campbell}, year = {2014}, date = {2014-06-01}, booktitle = {2014 IEEE Second international Congress on Big Data (BigData Congress 2014)}, publisher = {IEEE}, address = {Anchorage,AK,USA}, organization = {IEEE}, keywords = {}, pubstate = {published}, tppubtype = {conference} } |
Abad, Cristina; Lu, Yi; Campbell, Roy; Roberts, Nathan A Model-Based Namespace Metadata Benchmark for HDFS (Conference) International Conference on Autonomic Computing, USENIX USENIX, Philidelphia, 2014. (BibTeX | Tags: ) @conference{177, title = {A Model-Based Namespace Metadata Benchmark for HDFS}, author = {Cristina L. Abad and Yi Lu and Roy H. Campbell and Nathan Roberts}, year = {2014}, date = {2014-06-01}, booktitle = {International Conference on Autonomic Computing}, publisher = {USENIX}, address = {Philidelphia}, organization = {USENIX}, keywords = {}, pubstate = {published}, tppubtype = {conference} } |
Abad, Cristina Big Data Storage Workload Characterization, Modeling and Synthetic Generation (Masters Thesis) University of Illinois at Urbana-Champaign, 2014. (BibTeX | Tags: ) @mastersthesis{589, title = {Big Data Storage Workload Characterization, Modeling and Synthetic Generation}, author = {Cristina Abad}, year = {2014}, date = {2014-05-01}, volume = {Ph.D.}, address = {Urbana, Illinois}, school = {University of Illinois at Urbana-Champaign}, keywords = {}, pubstate = {published}, tppubtype = {mastersthesis} } |
Schmidt, Maxie A Computer Algebra Package for Polynomial Sequence Recognition (Masters Thesis) University of Illinois, 2014. (BibTeX | Tags: ) @mastersthesis{586, title = {A Computer Algebra Package for Polynomial Sequence Recognition}, author = {Maxie Schmidt}, year = {2014}, date = {2014-05-01}, volume = {MS}, address = {Urbana, Illinois}, school = {University of Illinois}, keywords = {}, pubstate = {published}, tppubtype = {mastersthesis} } |
Lang, Anthony A New Portable Digital Forensics Curriculum (Masters Thesis) University of Illinois, 2014. (BibTeX | Tags: ) @mastersthesis{587, title = {A New Portable Digital Forensics Curriculum}, author = {Anthony Lang}, year = {2014}, date = {2014-05-01}, volume = {MS}, address = {Urbana, Illinois}, school = {University of Illinois}, keywords = {}, pubstate = {published}, tppubtype = {mastersthesis} } |
Chu, Jonathan The Triple Pot and Techniques in Distributed System Call Intrusion Detection (Masters Thesis) University of Illinois, 2014. (BibTeX | Tags: ) @mastersthesis{588, title = {The Triple Pot and Techniques in Distributed System Call Intrusion Detection}, author = {Jonathan Chu}, year = {2014}, date = {2014-05-01}, volume = {MS}, address = {Urbana, Illinois}, school = {University of Illinois}, keywords = {}, pubstate = {published}, tppubtype = {mastersthesis} } |
Yao, Fangzhou; Campbell, Roy CouchFS: A High-Performance File System for Large Data Sets (Conference) IEEE Proceedings of the 3rd International Congress on Big Data 2014, Anchorage, Alaska, 2014. (BibTeX | Tags: ) @conference{174, title = {CouchFS: A High-Performance File System for Large Data Sets}, author = {Fangzhou Yao and Roy H. Campbell}, year = {2014}, date = {2014-01-01}, booktitle = {IEEE Proceedings of the 3rd International Congress on Big Data 2014}, address = {Anchorage, Alaska}, keywords = {}, pubstate = {published}, tppubtype = {conference} } |
Yao, Fangzhou; Campbell, Roy CryptVMI: Encrypted Virtual Machine Introspection in the Cloud (Conference) 2014 IEEE Seventh International Conference on Cloud Computing (Cloud 2014), IEEE IEEE, Anchorage, Alaska, 2014. (BibTeX | Tags: ) @conference{173, title = {CryptVMI: Encrypted Virtual Machine Introspection in the Cloud}, author = {Fangzhou Yao and Roy H Campbell}, year = {2014}, date = {2014-01-01}, booktitle = {2014 IEEE Seventh International Conference on Cloud Computing (Cloud 2014)}, publisher = {IEEE}, address = {Anchorage, Alaska}, organization = {IEEE}, keywords = {}, pubstate = {published}, tppubtype = {conference} } |
Lamps, Jereme WinWizard: Expanding Xen with a LibVMI Intrusion Detection Tool (Proceeding) 2014. (BibTeX | Tags: ) @proceedings{804, title = {WinWizard: Expanding Xen with a LibVMI Intrusion Detection Tool}, author = {Jereme Lamps}, editor = {Imani Palmer}, year = {2014}, date = {2014-01-01}, journal = {IEEE Sixth International Conference on Cloud Computing}, keywords = {}, pubstate = {published}, tppubtype = {proceedings} } |
2013 |
Malik, Muhammad Salman; Berthier, Robin; Bobba, Rekesh; Campbell, Roy; Sanders, William Formal Design of Communication Checkers for ICCP using UPPAAL (Conference) IEEE International Conference on Smart Grid Communications (SmartGridComm), Vancouver, Canada, 2013. (BibTeX | Tags: ) @conference{184, title = {Formal Design of Communication Checkers for ICCP using UPPAAL}, author = {Muhammad Salman Malik and Robin Berthier and Rekesh B. Bobba and Roy H. Campbell and William H. Sanders}, year = {2013}, date = {2013-10-01}, booktitle = {IEEE International Conference on Smart Grid Communications (SmartGridComm)}, pages = {486-491}, address = {Vancouver, Canada}, keywords = {}, pubstate = {published}, tppubtype = {conference} } |
Abad, Cristina; Yuan, Mindi; Cai, Chris; Lu, Yi; Roberts, Nathan; Campbell, Roy Generating request streams on Big Data using clustered renewal processes (Journal Article) Performance EvaluationPerformance EvaluationPerformance Evaluation, 70 (10), pp. 704-719, 2013, ISBN: 01665316, (Sp. Iss. SI<br/>238MX<br/>Times Cited:0<br/>Cited References Count:29). (Abstract | BibTeX | Tags: big data, hdfs, popularity, storage, temporal locality, workload generation) @article{188, title = {Generating request streams on Big Data using clustered renewal processes}, author = {Abad, Cristina L. and Yuan, Mindi and Cai, Chris X. and Lu, Yi and Roberts, Nathan and Campbell, Roy H.}, isbn = {01665316}, year = {2013}, date = {2013-10-01}, journal = {Performance EvaluationPerformance EvaluationPerformance Evaluation}, volume = {70}, number = {10}, pages = {704-719}, abstract = {The performance evaluation of large file systems, such as storage and media streaming, motivates scalable generation of representative traces. We focus on two key characteristics of traces, popularity and temporal locality. The common practice of using a system-wide distribution obscures per-object behavior, which is important for system evaluation. We propose a model based on delayed renewal processes which, by sampling interarrival times for each object, accurately reproduces popularity and temporal locality for the trace. A lightweight version reduces the dimension of the model with statistical clustering. It is workload-agnostic and object type-aware, suitable for testing emerging workloads and textquoterightwhat-iftextquoteright scenarios. We implemented a synthetic trace generator and validated it using: (1) a Big Data storage (HDFS) workload from Yahoo!, (2) a trace from a feature animation company, and (3) a streaming media workload. Two case studies in caching and replicated distributed storage systems show that our traces produce application-level results similar to the real workload. The trace generator is fast and readily scales to a system of 4.3 million Files. It outperforms existing models in terms of accurately reproducing the characteristics of the real trace. (C) 2013 Elsevier B.V. All rights reserved.}, note = {Sp. Iss. SI<br/>238MX<br/>Times Cited:0<br/>Cited References Count:29}, keywords = {big data, hdfs, popularity, storage, temporal locality, workload generation}, pubstate = {published}, tppubtype = {article} } The performance evaluation of large file systems, such as storage and media streaming, motivates scalable generation of representative traces. We focus on two key characteristics of traces, popularity and temporal locality. The common practice of using a system-wide distribution obscures per-object behavior, which is important for system evaluation. We propose a model based on delayed renewal processes which, by sampling interarrival times for each object, accurately reproduces popularity and temporal locality for the trace. A lightweight version reduces the dimension of the model with statistical clustering. It is workload-agnostic and object type-aware, suitable for testing emerging workloads and textquoterightwhat-iftextquoteright scenarios. We implemented a synthetic trace generator and validated it using: (1) a Big Data storage (HDFS) workload from Yahoo!, (2) a trace from a feature animation company, and (3) a streaming media workload. Two case studies in caching and replicated distributed storage systems show that our traces produce application-level results similar to the real workload. The trace generator is fast and readily scales to a system of 4.3 million Files. It outperforms existing models in terms of accurately reproducing the characteristics of the real trace. (C) 2013 Elsevier B.V. All rights reserved. |
Montanari, Mirko Limiting Information Exposure in Multi-Domain Monitoring Systems (Masters Thesis) University of Illinois at Urbana-Champaign, 2013. (BibTeX | Tags: ) @mastersthesis{590, title = {Limiting Information Exposure in Multi-Domain Monitoring Systems}, author = {Mirko Montanari}, year = {2013}, date = {2013-08-01}, volume = {Ph.D.}, address = {Urbana, Illinois}, school = {University of Illinois at Urbana-Champaign}, keywords = {}, pubstate = {published}, tppubtype = {mastersthesis} } |
textasciidieresis an, Gyorgy; Bobba, Rakesh; Gross, George; Campbell, Roy Cloud Computing for the Power Grid: From Service Composition to Assured Clouds (Conference) USENIX Workshop on Hot Topics in Cloud Computing, USENIX USENIX, San Jose, CA, 2013. (BibTeX | Tags: ) @conference{187, title = {Cloud Computing for the Power Grid: From Service Composition to Assured Clouds}, author = {Gyorgy D textasciidieresis an and Rakesh B. Bobba and George Gross and Roy H. Campbell}, year = {2013}, date = {2013-06-01}, booktitle = {USENIX Workshop on Hot Topics in Cloud Computing}, publisher = {USENIX}, address = {San Jose, CA}, organization = {USENIX}, keywords = {}, pubstate = {published}, tppubtype = {conference} } |
Marsan, Robert Android Behind the Scenes: Revealing Hidden Malware with Andromeda (Masters Thesis) University of Illinois, 2013. (BibTeX | Tags: ) @mastersthesis{591, title = {Android Behind the Scenes: Revealing Hidden Malware with Andromeda}, author = {Robert Marsan}, year = {2013}, date = {2013-05-01}, volume = {MS}, address = {Urbana, Illinois}, school = {University of Illinois}, keywords = {}, pubstate = {published}, tppubtype = {mastersthesis} } |
Kharbanda, Harshit Software Systems for Power and Energy Conservation (Masters Thesis) University of Illinois, 2013. (BibTeX | Tags: ) @mastersthesis{592, title = {Software Systems for Power and Energy Conservation}, author = {Harshit Kharbanda}, year = {2013}, date = {2013-05-01}, volume = {MS}, address = {Urbana, Illinois}, school = {University of Illinois}, keywords = {}, pubstate = {published}, tppubtype = {mastersthesis} } |
Huh, Jun Ho; Montanari, Mirko; Dagit, Derek; Bobba, Rakesh; Kim, Dongwook; Choi, Yoonjoo; Campbell, Roy Assessing software integrity of virtual appliances through software whitelists (Conference) NDSS, 2013. (BibTeX | Tags: ) @conference{186, title = {Assessing software integrity of virtual appliances through software whitelists}, author = {Huh, Jun Ho and Montanari, Mirko and Dagit, Derek and Bobba, Rakesh B. and Kim, Dongwook and Choi, Yoonjoo and Campbell, Roy H.}, year = {2013}, date = {2013-01-01}, booktitle = {NDSS}, keywords = {}, pubstate = {published}, tppubtype = {conference} } |
Montanari, Mirko; Chan, Ellick; Larson, Kevin; Yoo, Wucherl; Campbell, Roy Distributed security policy conformance (Conference) Computers & Security, 33 , 2013, ISBN: 01674048. (BibTeX | Tags: ) @conference{182, title = {Distributed security policy conformance}, author = {Montanari, Mirko and Chan, Ellick and Larson, Kevin and Yoo, Wucherl and Campbell, Roy H.}, isbn = {01674048}, year = {2013}, date = {2013-01-01}, booktitle = {Computers & Security}, volume = {33}, pages = {28-40}, keywords = {}, pubstate = {published}, tppubtype = {conference} } |
Montanari, Mirko; Huh, Jun Ho; Bobba, Rakesh; Campbell, Roy Limiting Data Exposure in Monitoring Multi-domain Policy Conformance (Conference) TRUST, IEEE IEEE, Imperial College, London, 2013. (BibTeX | Tags: ) @conference{181, title = {Limiting Data Exposure in Monitoring Multi-domain Policy Conformance}, author = {Montanari, Mirko and Huh, Jun Ho and Bobba, Rakesh B. and Campbell, Roy H.}, year = {2013}, date = {2013-01-01}, booktitle = {TRUST}, pages = {65-82}, publisher = {IEEE}, address = {Imperial College, London}, organization = {IEEE}, keywords = {}, pubstate = {published}, tppubtype = {conference} } |
Huh, Jun Ho; Montanari, Mirko; Dagit, Derek; Bobba, Rakesh; Kim, Dong Wook; Choi, Yoonjoo; Campbell, Roy An empirical study on the software integrity of virtual appliances (Conference) Proceedings of the 8th ACM SIGSAC symposium on Information, computer and communications security - ASIA CCS textquoteright13, ACM Press ACM Press, New York, New York, USA, 2013, ISBN: 9781450317672. (BibTeX | Tags: ) @conference{185, title = {An empirical study on the software integrity of virtual appliances}, author = {Huh, Jun Ho and Montanari, Mirko and Dagit, Derek and Bobba, Rakesh B. and Kim, Dong Wook and Choi, Yoonjoo and Campbell, Roy}, isbn = {9781450317672}, year = {2013}, date = {2013-00-01}, booktitle = {Proceedings of the 8th ACM SIGSAC symposium on Information, computer and communications security - ASIA CCS textquoteright13}, pages = {231-231}, publisher = {ACM Press}, address = {New York, New York, USA}, organization = {ACM Press}, keywords = {}, pubstate = {published}, tppubtype = {conference} } |
Verma, Abhishek; Cherkasova, Ludmila; Campbell, Roy Orchestrating an Ensemble of MapReduce Jobs for Minimizing Their Makespan (Journal Article) IEEE Transactions on Dependable and Secure ComputingIEEE Transactions on Dependable and Secure ComputingIEEE Transactions on Dependable and Secure Computing, 10 (5), pp. 314-327, 2013, ISBN: 1545-5971, (217UE<br/>Times Cited:0<br/>Cited References Count:33). (Abstract | BibTeX | Tags: Algorithm design and analysis, batch workloads, Business, Clustering algorithms, Computational modeling, Hadoop, MapReduce, minimized makespan, optimized schedule, Production, Schedules, Upper bound) @article{179, title = {Orchestrating an Ensemble of MapReduce Jobs for Minimizing Their Makespan}, author = {Verma, Abhishek and Cherkasova, Ludmila and Campbell, Roy H.}, isbn = {1545-5971}, year = {2013}, date = {2013-00-01}, journal = {IEEE Transactions on Dependable and Secure ComputingIEEE Transactions on Dependable and Secure ComputingIEEE Transactions on Dependable and Secure Computing}, volume = {10}, number = {5}, pages = {314-327}, publisher = {IEEE}, abstract = {Cloud computing offers an attractive option for businesses to rent a suitable size MapReduce cluster, consume resources as a service, and pay only for resources that were consumed. A key challenge in such environments is to increase the utilization of MapReduce clusters to minimize their cost. One way of achieving this goal is to optimize the execution of Mapreduce jobs on the cluster. For a set of production jobs that are executed periodically on new data, we can perform an offline analysis for evaluating performance benefits of different optimization techniques. In this work, we consider a subset of production workloads that consists of MapReduce jobs with no dependencies. We observe that the order in which these jobs are executed can have a significant impact on their overall completion time and the cluster resource utilization. Our goal is to automate the design of a job schedule that minimizes the completion time (makespan) of such a set of MapReduce jobs. We introduce a simple abstraction where each MapReduce job is represented as a pair of map and reduce stage durations. This representation enables us to apply the classic Johnson algorithm that was designed for building an optimal two-stage job schedule. We evaluate the performance benefits of the constructed schedule through an extensive set of simulations over a variety of realistic workloads. The results are workload and cluster-size dependent, but it is typical to achieve up to 10-25 percent of makespan improvements by simply processing the jobs in the right order. However, in some cases, the simplified abstraction assumed by Johnsontextquoterights algorithm may lead to a suboptimal job schedule. We design a novel heuristic, called BalancedPools, that significantly improves Johnsontextquoterights schedule results (up to 15-38 percent), exactly in the situations when it produces suboptimal makespan. Overall, we observe up to 50 percent in the makespan improvements with the new BalancedPools algorithm. The results of our simulation study are validated through experiments on a 66-node Hadoop cluster.}, note = {217UE<br/>Times Cited:0<br/>Cited References Count:33}, keywords = {Algorithm design and analysis, batch workloads, Business, Clustering algorithms, Computational modeling, Hadoop, MapReduce, minimized makespan, optimized schedule, Production, Schedules, Upper bound}, pubstate = {published}, tppubtype = {article} } Cloud computing offers an attractive option for businesses to rent a suitable size MapReduce cluster, consume resources as a service, and pay only for resources that were consumed. A key challenge in such environments is to increase the utilization of MapReduce clusters to minimize their cost. One way of achieving this goal is to optimize the execution of Mapreduce jobs on the cluster. For a set of production jobs that are executed periodically on new data, we can perform an offline analysis for evaluating performance benefits of different optimization techniques. In this work, we consider a subset of production workloads that consists of MapReduce jobs with no dependencies. We observe that the order in which these jobs are executed can have a significant impact on their overall completion time and the cluster resource utilization. Our goal is to automate the design of a job schedule that minimizes the completion time (makespan) of such a set of MapReduce jobs. We introduce a simple abstraction where each MapReduce job is represented as a pair of map and reduce stage durations. This representation enables us to apply the classic Johnson algorithm that was designed for building an optimal two-stage job schedule. We evaluate the performance benefits of the constructed schedule through an extensive set of simulations over a variety of realistic workloads. The results are workload and cluster-size dependent, but it is typical to achieve up to 10-25 percent of makespan improvements by simply processing the jobs in the right order. However, in some cases, the simplified abstraction assumed by Johnsontextquoterights algorithm may lead to a suboptimal job schedule. We design a novel heuristic, called BalancedPools, that significantly improves Johnsontextquoterights schedule results (up to 15-38 percent), exactly in the situations when it produces suboptimal makespan. Overall, we observe up to 50 percent in the makespan improvements with the new BalancedPools algorithm. The results of our simulation study are validated through experiments on a 66-node Hadoop cluster. |
Tedesco, Jon; Dudko, Roman; Sharma, Abhishek; Farivar, Reza; Campbell, Roy Theius: A Streaming Visualization Suite for Hadoop Clusters* (Conference) IC2E: International Conference on Cloud Engineering, IEEE IEEE, Redwood City, CA, 2013, ISBN: 978-0-7695-4945-3. (Abstract | BibTeX | Tags: cloud computing, cloud computing cluster, cluster computing, data visualisation, failure detection, failure detection system, failure prediction, fault tolerant computing, Hadoop, Hadoop cluster, monitoring, monitoring system, parallel processing, streaming visualization suite, Theius suite, visualization, visualization tool, Web-based visualization suite) @conference{180, title = {Theius: A Streaming Visualization Suite for Hadoop Clusters*}, author = {Tedesco, Jon and Dudko, Roman and Sharma, Abhishek and Farivar, Reza and Campbell, Roy}, isbn = {978-0-7695-4945-3}, year = {2013}, date = {2013-00-01}, booktitle = {IC2E: International Conference on Cloud Engineering}, pages = {177-182}, publisher = {IEEE}, address = {Redwood City, CA}, organization = {IEEE}, abstract = {As cloud computing clusters continue to grow, maintaining the health of these clusters becomes increasingly challenging. Recent work has studied how we can efficiently monitor the status of machines in these clusters and how we can detect problems or predict them before they occur, yet little work has focused on addressing the bottleneck between when these failures occur and when they are fixed: system administrators. As monitoring and failure detection systems mature, we are able to extract tremendous amounts of information about the status of the system in real time. However, this amount of data is difficult to understand for human beings, especially those inexperienced with the particular cluster. In this paper, we introduce a web-based visualization suite called Theius to allow system administrators to quickly understand the state of the cloud system as a whole. We outline the key features of this visualization tool, and show that it is more intuitive and easy to use than Ganglia, a state-of-the art visualization tool for clusters. Likewise, we demonstrate that our tool can scale, presenting a use case with our visualization showing a 5000 node cluster. Although our tool is implemented for Hadoop clusters, our contribution is general to any cloud computing system.}, keywords = {cloud computing, cloud computing cluster, cluster computing, data visualisation, failure detection, failure detection system, failure prediction, fault tolerant computing, Hadoop, Hadoop cluster, monitoring, monitoring system, parallel processing, streaming visualization suite, Theius suite, visualization, visualization tool, Web-based visualization suite}, pubstate = {published}, tppubtype = {conference} } As cloud computing clusters continue to grow, maintaining the health of these clusters becomes increasingly challenging. Recent work has studied how we can efficiently monitor the status of machines in these clusters and how we can detect problems or predict them before they occur, yet little work has focused on addressing the bottleneck between when these failures occur and when they are fixed: system administrators. As monitoring and failure detection systems mature, we are able to extract tremendous amounts of information about the status of the system in real time. However, this amount of data is difficult to understand for human beings, especially those inexperienced with the particular cluster. In this paper, we introduce a web-based visualization suite called Theius to allow system administrators to quickly understand the state of the cloud system as a whole. We outline the key features of this visualization tool, and show that it is more intuitive and easy to use than Ganglia, a state-of-the art visualization tool for clusters. Likewise, we demonstrate that our tool can scale, presenting a use case with our visualization showing a 5000 node cluster. Although our tool is implemented for Hadoop clusters, our contribution is general to any cloud computing system. |
Malik, Muhammad Salman; Montanari, Mirko; Huh, Jun Ho; Bobba, Rakesh; Campbell, Roy Towards SDN enabled network control delegation in clouds (Conference) 43rd Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN), IEEE IEEE, Budapest, Hungary, 2013, ISBN: 978-1-4673-6472-0. (Abstract | BibTeX | Tags: Cloud, Control Delegation, monitoring, Network topology, Protocols, SDN, Security, Switches, Topology) @conference{183, title = {Towards SDN enabled network control delegation in clouds}, author = {Malik, Muhammad Salman and Montanari, Mirko and Huh, Jun Ho and Bobba, Rakesh B. and Campbell, Roy H.}, isbn = {978-1-4673-6472-0}, year = {2013}, date = {2013-00-01}, booktitle = {43rd Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN)}, pages = {1-6}, publisher = {IEEE}, address = {Budapest, Hungary}, organization = {IEEE}, abstract = {In todaytextquoterights IaaS clouds users only get a logical view of the underlying network and have limited control. Delegating more control to end users would be beneficial but would also raise security concerns for the provider. Emerging Software Defined Networking (SDN) technologies have the capabilities to facilitate delegation of network controls and provide some level of network abstractions to end users. However, any delegation solution should try to balance the level of controls delegated to end users with the security constraints of the provider. In this paper, we propose a SDN-based framework to facilitate delegation of some network controls to end users, providing the means to monitor and configure their own slices of the underlying networks. Using two instantiations of this framework, we illustrate the tradeoffs between security and the level of network abstractions provided to end users.}, keywords = {Cloud, Control Delegation, monitoring, Network topology, Protocols, SDN, Security, Switches, Topology}, pubstate = {published}, tppubtype = {conference} } In todaytextquoterights IaaS clouds users only get a logical view of the underlying network and have limited control. Delegating more control to end users would be beneficial but would also raise security concerns for the provider. Emerging Software Defined Networking (SDN) technologies have the capabilities to facilitate delegation of network controls and provide some level of network abstractions to end users. However, any delegation solution should try to balance the level of controls delegated to end users with the security constraints of the provider. In this paper, we propose a SDN-based framework to facilitate delegation of some network controls to end users, providing the means to monitor and configure their own slices of the underlying networks. Using two instantiations of this framework, we illustrate the tradeoffs between security and the level of network abstractions provided to end users. |
2012 |
Farivar, Reza Adaptation of the MapReduce Programming Framework to Compute-Intensive Data Analytics Kernels (Masters Thesis) University of Illinois at Urbana-Champaign, 2012. (BibTeX | Tags: ) @mastersthesis{596, title = {Adaptation of the MapReduce Programming Framework to Compute-Intensive Data Analytics Kernels}, author = {Reza Farivar}, year = {2012}, date = {2012-12-01}, volume = {Ph.D.}, pages = {123}, address = {Urbana, Illinois}, school = {University of Illinois at Urbana-Champaign}, keywords = {}, pubstate = {published}, tppubtype = {mastersthesis} } |
Yoo, Wucherl Automated Performance Characterization of Applications Using Hardware Monitoring Events (Masters Thesis) University of Illinois at Urbana-Champaign, 2012. (BibTeX | Tags: ) @mastersthesis{593, title = {Automated Performance Characterization of Applications Using Hardware Monitoring Events}, author = {Wucherl Yoo}, year = {2012}, date = {2012-12-01}, volume = {Ph.D.}, address = {Urbana, Illinois}, school = {University of Illinois at Urbana-Champaign}, keywords = {}, pubstate = {published}, tppubtype = {mastersthesis} } |
Verma, Abhishek Performance Modeling Framework for Slo-Driven Map-reduce Environments (Masters Thesis) University of Illinois at Urbana-Champaign, 2012. (BibTeX | Tags: ) @mastersthesis{594, title = {Performance Modeling Framework for Slo-Driven Map-reduce Environments}, author = {Abhishek Verma}, year = {2012}, date = {2012-12-01}, volume = {Ph.D.}, address = {Urbana, Illinois}, school = {University of Illinois at Urbana-Champaign}, keywords = {}, pubstate = {published}, tppubtype = {mastersthesis} } |
Shi, Shu A Low Latency Remote Rendering System for Interactive Mobile Graphics (Masters Thesis) University of Illinois at Urbana-Champaign, 2012. (BibTeX | Tags: ) @mastersthesis{595, title = {A Low Latency Remote Rendering System for Interactive Mobile Graphics}, author = {Shu Shi}, year = {2012}, date = {2012-05-01}, volume = {Ph.D.}, address = {Urbana, Illinois}, school = {University of Illinois at Urbana-Champaign}, keywords = {}, pubstate = {published}, tppubtype = {mastersthesis} } |
Bakhouya, Mohamed; Campbell, Roy; Coronato, Antonio; de Pietro, Giuseppe; Ranganathan, Anand Introduction to special section on formal methods in pervasive computing (Journal Article) ACM Transactions on Autonomous and Adaptive SystemsACM Transactions on Autonomous and Adaptive SystemsACM Transactions on Autonomous and Adaptive Systems, 7 (1), pp. 1-9, 2012, ISBN: 1556-4665, (951AH<br/>Times Cited:0<br/>Cited References Count:36). (Abstract | BibTeX | Tags: design, formal specification and verification, methodologies and tools, mobile ambients, temporal logic, ubiquitous and pervasive safety-critical applications, verification) @article{200, title = {Introduction to special section on formal methods in pervasive computing}, author = {Bakhouya, Mohamed and Campbell, Roy and Coronato, Antonio and Pietro, Giuseppe de and Ranganathan, Anand}, isbn = {1556-4665}, year = {2012}, date = {2012-04-01}, journal = {ACM Transactions on Autonomous and Adaptive SystemsACM Transactions on Autonomous and Adaptive SystemsACM Transactions on Autonomous and Adaptive Systems}, volume = {7}, number = {1}, pages = {1-9}, abstract = {Ubiquitous and pervasive applications may present critical requirements from the point of view of functional correctness, reliability, availability, security, and safety. Unlike traditional safety-critical applications, the behavior of ubiquitous and pervasive applications is affected by the movements and location of users and resources. In this article, we first present emerging formal methods for the description of both entities and their behavior in pervasive computing environments; then, we introduce this special issue. Despite many previous works that have focused on modeling the entities, relatively few have concentrated on modeling or verifying behaviors; and almost none has dealt with combining techniques proposed in these two aspects. The articles accepted in this special issue cover some of the topics aforementioned and constitute a representative sample of the latest development of formal methods in pervasive computing environments.}, note = {951AH<br/>Times Cited:0<br/>Cited References Count:36}, keywords = {design, formal specification and verification, methodologies and tools, mobile ambients, temporal logic, ubiquitous and pervasive safety-critical applications, verification}, pubstate = {published}, tppubtype = {article} } Ubiquitous and pervasive applications may present critical requirements from the point of view of functional correctness, reliability, availability, security, and safety. Unlike traditional safety-critical applications, the behavior of ubiquitous and pervasive applications is affected by the movements and location of users and resources. In this article, we first present emerging formal methods for the description of both entities and their behavior in pervasive computing environments; then, we introduce this special issue. Despite many previous works that have focused on modeling the entities, relatively few have concentrated on modeling or verifying behaviors; and almost none has dealt with combining techniques proposed in these two aspects. The articles accepted in this special issue cover some of the topics aforementioned and constitute a representative sample of the latest development of formal methods in pervasive computing environments. |
Yoo, Wucherl; Larson, Kevin; Baugh, Lee; Kim, Sangkyum; Campbell, Roy ADP: automated diagnosis of performance pathologies using hardware events (Conference) SIGMETRICS, 2012. (BibTeX | Tags: ) @conference{189, title = {ADP: automated diagnosis of performance pathologies using hardware events}, author = {Yoo, Wucherl and Larson, Kevin and Baugh, Lee and Kim, Sangkyum and Campbell, Roy H.}, year = {2012}, date = {2012-01-01}, booktitle = {SIGMETRICS}, pages = {283-294}, keywords = {}, pubstate = {published}, tppubtype = {conference} } |
Montanari, Mirko; Campbell, Roy Confidentiality of event data in policy-based monitoring (Conference) IEEE/IFIP International Conference on Dependable Systems and Networks (DSN 2012), IEEE IEEE, 2012, ISBN: 978-1-4673-1625-5. (Abstract | BibTeX | Tags: confidentiality, distributed systems, monitoring, policy compliance, Security) @conference{196, title = {Confidentiality of event data in policy-based monitoring}, author = {Montanari, Mirko and Campbell, Roy H.}, isbn = {978-1-4673-1625-5}, year = {2012}, date = {2012-01-01}, booktitle = {IEEE/IFIP International Conference on Dependable Systems and Networks (DSN 2012)}, pages = {1-12}, publisher = {IEEE}, organization = {IEEE}, abstract = {Monitoring systems observe important information that could be a valuable resource to malicious users: attackers can use the knowledge of topology information, application logs, or configuration data to target attacks and make them hard to detect. The increasing need for correlating information across distributed systems to better detect potential attacks and to meet regulatory requirements can potentially exacerbate the problem if the monitoring is centralized. A single zero-day vulnerability would permit an attacker to access all information. This paper introduces a novel algorithm for performing policy-based security monitoring. We use policies to distribute information across several hosts, so that any host compromise has limited impact on the confidentiality of the data about the overall system. Experiments show that our solution spreads information uniformly across distributed monitoring hosts and forces attackers to perform multiple actions to acquire important data.}, keywords = {confidentiality, distributed systems, monitoring, policy compliance, Security}, pubstate = {published}, tppubtype = {conference} } Monitoring systems observe important information that could be a valuable resource to malicious users: attackers can use the knowledge of topology information, application logs, or configuration data to target attacks and make them hard to detect. The increasing need for correlating information across distributed systems to better detect potential attacks and to meet regulatory requirements can potentially exacerbate the problem if the monitoring is centralized. A single zero-day vulnerability would permit an attacker to access all information. This paper introduces a novel algorithm for performing policy-based security monitoring. We use policies to distribute information across several hosts, so that any host compromise has limited impact on the confidentiality of the data about the overall system. Experiments show that our solution spreads information uniformly across distributed monitoring hosts and forces attackers to perform multiple actions to acquire important data. |
Verma, Abhishek; Cherkasova, Ludmila; Kumar, Vijay; Campbell, Roy Deadline-based workload management for MapReduce environments: Pieces of the performance puzzle. (Conference) IEEE Network Operations and Management Symposium (NOMS), 2012. (BibTeX | Tags: ) @conference{190, title = {Deadline-based workload management for MapReduce environments: Pieces of the performance puzzle.}, author = {Verma, Abhishek and Cherkasova, Ludmila and Kumar, Vijay S. and Roy H. Campbell}, year = {2012}, date = {2012-01-01}, booktitle = {IEEE Network Operations and Management Symposium (NOMS)}, pages = {900-905}, keywords = {}, pubstate = {published}, tppubtype = {conference} } |
Montanari, Mirko; Huh, Jun Ho; Dagit, Derek; Bobba, Rakesh; Campbell, Roy Evidence of log integrity in policy-based security monitoring (Conference) IEEE/IFIP International Conference on Dependable Systems and Networks Workshops (DSN 2012), IEEE IEEE, 2012, ISBN: 978-1-4673-2266-9. (Abstract | BibTeX | Tags: attackers, auditing, auditing agency, centralized monitoring system, cloud computing, cloud security monitoring API, cloud services, cloud-based framework, Computer architecture, computer crime, computerised monitoring, Credit cards, CSM API, data integrity, data loggers, data storage, expensive noncompliance fines, IDS, log data generation, log integrity, malicious employees, monitoring, network operations, Organizations, policy-based security monitoring, regulatory authorities, resource compliance verification, Security, security managers, SNMP, Software, software agents, Standards organizations) @conference{194, title = {Evidence of log integrity in policy-based security monitoring}, author = {Montanari, Mirko and Huh, Jun Ho and Dagit, Derek and Bobba, Rakesh B. and Campbell, Roy H.}, isbn = {978-1-4673-2266-9}, year = {2012}, date = {2012-01-01}, booktitle = {IEEE/IFIP International Conference on Dependable Systems and Networks Workshops (DSN 2012)}, pages = {1-6}, publisher = {IEEE}, organization = {IEEE}, abstract = {Monitoring systems are commonly used by many organizations to collect information about their system and network operations. Typically, SNMP, IDS, or software agents generate log data and store them in a centralized monitoring system for analysis. However, malicious employees, attackers, or even organizations themselves can modify such data to hide malicious activities or to avoid expensive non-compliance fines. This paper proposes a cloud-based framework for verifying the trustworthiness of the logs based on a small amount of evidence data. A simple Cloud Security Monitoring (CSM) API, made available on the cloud services, allows organizations operating on the cloud to collect additional textquotedblleftevidencetextquotedblright about their systems. Such evidence is used to verify system compliance against the policies set by security managers or regulatory authorities. We present a strategy for randomly auditing and verifying resource compliance, and propose an architecture that allows the organizations to prove compliance to an external auditing agency.}, keywords = {attackers, auditing, auditing agency, centralized monitoring system, cloud computing, cloud security monitoring API, cloud services, cloud-based framework, Computer architecture, computer crime, computerised monitoring, Credit cards, CSM API, data integrity, data loggers, data storage, expensive noncompliance fines, IDS, log data generation, log integrity, malicious employees, monitoring, network operations, Organizations, policy-based security monitoring, regulatory authorities, resource compliance verification, Security, security managers, SNMP, Software, software agents, Standards organizations}, pubstate = {published}, tppubtype = {conference} } Monitoring systems are commonly used by many organizations to collect information about their system and network operations. Typically, SNMP, IDS, or software agents generate log data and store them in a centralized monitoring system for analysis. However, malicious employees, attackers, or even organizations themselves can modify such data to hide malicious activities or to avoid expensive non-compliance fines. This paper proposes a cloud-based framework for verifying the trustworthiness of the logs based on a small amount of evidence data. A simple Cloud Security Monitoring (CSM) API, made available on the cloud services, allows organizations operating on the cloud to collect additional textquotedblleftevidencetextquotedblright about their systems. Such evidence is used to verify system compliance against the policies set by security managers or regulatory authorities. We present a strategy for randomly auditing and verifying resource compliance, and propose an architecture that allows the organizations to prove compliance to an external auditing agency. |
Tan, Yu Shyang; Lee, Bu-Sung; He, Bingsheng; Campbell, Roy A Map-Reduce Based Framework for Heterogeneous Processing Element Cluster Environments (Conference) 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (ccgrid 2012), IEEE IEEE, 2012, ISBN: 978-1-4673-1395-7. (Abstract | BibTeX | Tags: GPGPU, Heterogeneous resource framework, MapReduce) @conference{192, title = {A Map-Reduce Based Framework for Heterogeneous Processing Element Cluster Environments}, author = {Tan, Yu Shyang and Lee, Bu-Sung and He, Bingsheng and Campbell, Roy H.}, isbn = {978-1-4673-1395-7}, year = {2012}, date = {2012-01-01}, booktitle = {12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (ccgrid 2012)}, pages = {57-64}, publisher = {IEEE}, organization = {IEEE}, abstract = {In this paper, we present our design of a Processing Element (PE) Aware MapReduce base framework, Pamar. Pamar is designed for supporting distributed computing on clusters where node PE configurations are asymmetric on different nodes. Pamartextquoterights main goal is to allow users to seamlessly utilize different kinds of processing elements (e.g., CPUs or GPUs) collaboratively for large scale data processing. To show proof of concept, we have incorporated our designs into the Hadoop framework and tested it on cluster environments having asymmetric node PE configurations. We demonstrate Pamartextquoterights ability to identify PEs available on each node and match-make user jobs with nodes, base on job PE requirements. Pamar allows users to easily parallelize applications across large datasets and at the same time utilizes different PEs for processing different classes of functions efficiently. The experiments show improvement in job queue completion time with Pamar over clusters with asymmetric nodes as compared to clusters with symmetric nodes.}, keywords = {GPGPU, Heterogeneous resource framework, MapReduce}, pubstate = {published}, tppubtype = {conference} } In this paper, we present our design of a Processing Element (PE) Aware MapReduce base framework, Pamar. Pamar is designed for supporting distributed computing on clusters where node PE configurations are asymmetric on different nodes. Pamartextquoterights main goal is to allow users to seamlessly utilize different kinds of processing elements (e.g., CPUs or GPUs) collaboratively for large scale data processing. To show proof of concept, we have incorporated our designs into the Hadoop framework and tested it on cluster environments having asymmetric node PE configurations. We demonstrate Pamartextquoterights ability to identify PEs available on each node and match-make user jobs with nodes, base on job PE requirements. Pamar allows users to easily parallelize applications across large datasets and at the same time utilizes different PEs for processing different classes of functions efficiently. The experiments show improvement in job queue completion time with Pamar over clusters with asymmetric nodes as compared to clusters with symmetric nodes. |
Abad, Cristina; Luu, Huong; Roberts, Nathan; Lee, Kihwal; Lu, Yi; Campbell, Roy Metadata Traces and Workload Models for Evaluating Big Storage Systems (Conference) 2012 IEEE Fifth International Conference on Utility and Cloud Computing, IEEE IEEE, 2012, ISBN: 978-1-4673-4432-6. (Abstract | BibTeX | Tags: big data, hdfs, MDS, metadata, storage) @conference{202, title = {Metadata Traces and Workload Models for Evaluating Big Storage Systems}, author = {Abad, Cristina L. and Luu, Huong and Roberts, Nathan and Lee, Kihwal and Lu, Yi and Campbell, Roy H.}, isbn = {978-1-4673-4432-6}, year = {2012}, date = {2012-01-01}, booktitle = {2012 IEEE Fifth International Conference on Utility and Cloud Computing}, pages = {125-132}, publisher = {IEEE}, organization = {IEEE}, abstract = {Efficient namespace metadata management is increasingly important as next-generation file systems are designed for peta and exascales. New schemes have been proposed, however, their evaluation has been insufficient due to a lack of appropriate namespace metadata traces. Specifically, no Big Data storage system metadata trace is publicly available and existing ones are a poor replacement. We studied publicly available traces and one Big Data trace from Yahoo! and note some of the differences and their implications to metadata management studies. We discuss the insufficiency of existing evaluation approaches and present a first step towards a statistical metadata workload model that can capture the relevant characteristics of a workload and is suitable for synthetic workload generation. We describe Mimesis, a synthetic workload generator, and evaluate its usefulness through a case study in a least recently used metadata cache for the Hadoop Distributed File System. Simulation results show that the traces generated by Mimesis mimic the original workload and can be used in place of the real trace providing accurate results.}, keywords = {big data, hdfs, MDS, metadata, storage}, pubstate = {published}, tppubtype = {conference} } Efficient namespace metadata management is increasingly important as next-generation file systems are designed for peta and exascales. New schemes have been proposed, however, their evaluation has been insufficient due to a lack of appropriate namespace metadata traces. Specifically, no Big Data storage system metadata trace is publicly available and existing ones are a poor replacement. We studied publicly available traces and one Big Data trace from Yahoo! and note some of the differences and their implications to metadata management studies. We discuss the insufficiency of existing evaluation approaches and present a first step towards a statistical metadata workload model that can capture the relevant characteristics of a workload and is suitable for synthetic workload generation. We describe Mimesis, a synthetic workload generator, and evaluate its usefulness through a case study in a least recently used metadata cache for the Hadoop Distributed File System. Simulation results show that the traces generated by Mimesis mimic the original workload and can be used in place of the real trace providing accurate results. |
Montanari, Mirko; Cook, Lucas; Campbell, Roy Multi-organization Policy-Based Monitoring (Conference) 2012 IEEE International Symposium on Policies for Distributed Systems and Networks, IEEE IEEE, 2012, ISBN: 978-1-4673-1993-5. (Abstract | BibTeX | Tags: cloud computing, Cognition, compliance, Knowledge based systems, monitoring, multi-domain, multi-organization, Organizations, policy, Security, Software, Strontium) @conference{195, title = {Multi-organization Policy-Based Monitoring}, author = {Montanari, Mirko and Cook, Lucas T. and Campbell, Roy H.}, isbn = {978-1-4673-1993-5}, year = {2012}, date = {2012-01-01}, booktitle = {2012 IEEE International Symposium on Policies for Distributed Systems and Networks}, pages = {70-77}, publisher = {IEEE}, organization = {IEEE}, abstract = {The monitoring of modern large scale infrastructure systems often relies on complex event processing (CEP) rules to detect security and performance problems. For example, the continuous monitoring of compliance to regulatory requirements such as PCI-DSS and NERC CIP requires analyzing events to identify if specific conditions over the configurations of devices occur. In multi-organization systems, detecting these problems often requires integrating events generated by different organizations. As events provide information about the infrastructuretextquoteright internal structure, organizations are interested in reducing the amount of information shared with external entities. This paper analyses the problem of detecting policy violations in network infrastructure systems managed by two organizations (e.g., a cloud user and a cloud provider). We focus on CEP monitoring systems and we introduce two protocols for selecting the events to share between the two organizations to ensure the detection of all possible policy violations. Our experimental evaluation shows that reciprocal information sharing between the two organizations significantly reduces the amount of information to transfer. In our SNMP monitoring test case, we obtain a 80% reduction in the information shared by any single organization.}, keywords = {cloud computing, Cognition, compliance, Knowledge based systems, monitoring, multi-domain, multi-organization, Organizations, policy, Security, Software, Strontium}, pubstate = {published}, tppubtype = {conference} } The monitoring of modern large scale infrastructure systems often relies on complex event processing (CEP) rules to detect security and performance problems. For example, the continuous monitoring of compliance to regulatory requirements such as PCI-DSS and NERC CIP requires analyzing events to identify if specific conditions over the configurations of devices occur. In multi-organization systems, detecting these problems often requires integrating events generated by different organizations. As events provide information about the infrastructuretextquoteright internal structure, organizations are interested in reducing the amount of information shared with external entities. This paper analyses the problem of detecting policy violations in network infrastructure systems managed by two organizations (e.g., a cloud user and a cloud provider). We focus on CEP monitoring systems and we introduce two protocols for selecting the events to share between the two organizations to ensure the detection of all possible policy violations. Our experimental evaluation shows that reciprocal information sharing between the two organizations significantly reduces the amount of information to transfer. In our SNMP monitoring test case, we obtain a 80% reduction in the information shared by any single organization. |