Date : Nov. 25, 2019, 2:30 p.m. - ZHANG Chao - Salle du conseil
Optimization of User-defined Aggregate Functions: Parallelization and Sharing
Applications of aggregations for information summary have great meanings in various fields. System built-in aggregations are not sufficient to cover the needs of new applications in the age of analytics. UDAFs (user-defined aggregate functions) are becoming a type of fundamental operators in advanced data analytics. The UDAF mechanism provided by most of the modern systems suffers however from at least two severe drawbacks: defining UDAFs requires hardcoding the routine that computes the aggregation function, and the semantics of UDAFs is totally or partially unknown to the query processor which hampers the optimization possibilities. This thesis presents SUDAF (Sharing User-Defined Aggregate Functions), a declarative framework that allows users to formulate UDAFs as mathematical expressions and use them in SQL statements. SUDAF comes equipped with the ability to generate efficient parallel implementation from users' UDAFs automatically and supports dynamic caching and reusing of partial aggregates. Our experiments show that the proposed sharing technique can lead from one to two orders of magnitude improvement in query execution times.
Keywords: Database management system -> Distributed Query processing, Query optimization, Query rewriting, User-defined aggregate functions, MapReduce, Apache Spark
A five-minute demo video of the SUDAF prototype can be accessed at https://www.youtube.com/watch?v=nH1S6PU6AK0.
Mme. Angela Bonifati, Professor, LIRIS, Université Claude Bernard Lyon 1, Assessor;
Mme. Myoung-Ah KANG, Associate Professor, LIMOS, Université Clermont Auvergne, Assessor;
M. Farouk Toumani, Professor, LIMOS, Université Clermont-Auvergne, Thesis director;
M. Emmanuel Gangler, Research Director, LPC, Université Clermont-Auvergne, Thesis director;
M. Reza Akbarinia, Research Director, INRIA Sophia Antipolis Mediterranee, Université de Montpellier, Reviewer;
M. Dimitris Kotzinos, Professor, ETIS Lab, Université de Cergy-Pontoise, Reviewer.