Retrieve, Merge, Predict: Augmenting Tables with Data Lakes

Cappuzzo, Riccardo; Varoquaux, Gael; Coelho, Aimee; Papotti, Paolo
Submitted to VLDB 2024, 50th International Conference on Very Large Databases, 25-29 August 2024, Guangzhou, China / Submitted to ArXiV, 9 February 2024

We present an in-depth analysis of data discovery in data lakes, focusing on table augmentation for given machine learning tasks. We analyze alternative methods used in the three main steps: retrieving joinable tables, merging information, and predicting with the resultant table. As data lakes, the paper uses YADL (Yet Another Data Lake) -- a novel dataset we developed as a tool for benchmarking this data discovery task -- and Open Data US, a well-referenced real data lake. Through systematic exploration on both lakes, our study outlines the importance of accurately retrieving join candidates and the efficiency of simple merging methods. We report new insights on the benefits of existing solutions and on their limitations, aiming at guiding future research in this space.

 

HAL
Type:
Conférence
City:
Guangzhou
Date:
2024-02-09
Department:
Data Science
Eurecom Ref:
7617
Copyright:
© ACM, 2024. This is the author's version of the work. It is posted here by permission of ACM for your personal use. Not for redistribution. The definitive version was published in Submitted to VLDB 2024, 50th International Conference on Very Large Databases, 25-29 August 2024, Guangzhou, China / Submitted to ArXiV, 9 February 2024
See also:

PERMALINK : https://www.eurecom.fr/publication/7617