Dick Epema - Data Science
Date: - Location: Eurecom
Title: Dynamic Resource Provisioning for Application Frameworks in Datacenters Speaker: Dick Epema Affiliations: Distributed Systems Group, Delft University of Technology, the Netherlands System Architecture and Networking Group, Eindhoven University of Technology, the Netherlands Abstract: In the past, we have designed and analyzed by means of simulations and real experiments with our KOALA scheduler a wide range of scheduling policies for such application types as parallel applications, bags-of-tasks, and workflows. With increasing cluster sizes, users requiring data and failure isolation, and applications requiring the installation of application frameworks such as MapReduce and Spark, we are currently investigating the dynamic, balanced allocation of resources to such frameworks, using both simulations and experimentation through KOALA. In this talk, I will explain the mechanisms and policies we have designed for this purpose, and I will present performance results. In addition, I will give an overview of other recent and current related research in the DS group in Delft (e.g., on workflow and portfolio scheduling). Bio: Dick H.J. Epema holds an MSc and a PhD in mathematics from Leiden University, and an MSc in computer science from Delft University of Technology. Currently, he is a professor of Computer Science at Delft University of Technology, and a part-time professor of Decentralized Distributed Systems at Eindhoven University of Technology, both in the Netherlands. His research interests are in resource management and scheduling in clusters and clouds, and in video distribution and reputation mechanisms in peer-to-peer systems and online social networks. His research is centered around the KOALA multicluster scheduler, which has been deployed on the Dutch DAS system, and the BitTorrent-based peer-to-peer system Tribler, which has been downloaded 1,500,000+ times. He was general co-chair of the Euro-Par 2009 and the IEEE P2P 2010 conferences, and general chair of HPDC'12 and CCGrid'13.
