Skip to main content

Showing 1–1 of 1 results for author: Katsogiannis-Meimarakis, G

Searching in archive cs. Search in all archives.
.
  1. arXiv:2306.04743  [pdf, other

    cs.DB cs.AI cs.CL

    ScienceBenchmark: A Complex Real-World Benchmark for Evaluating Natural Language to SQL Systems

    Authors: Yi Zhang, Jan Deriu, George Katsogiannis-Meimarakis, Catherine Kosten, Georgia Koutrika, Kurt Stockinger

    Abstract: Natural Language to SQL systems (NL-to-SQL) have recently shown a significant increase in accuracy for natural language to SQL query translation. This improvement is due to the emergence of transformer-based language models, and the popularity of the Spider benchmark - the de-facto standard for evaluating NL-to-SQL systems. The top NL-to-SQL systems reach accuracies of up to 85\%. However, Spider… ▽ More

    Submitted 5 December, 2023; v1 submitted 7 June, 2023; originally announced June 2023.

    Comments: 12 pages, 2 figures, 5 tables

    ACM Class: H.2.4; I.2.7

    Journal ref: PVLDB Volume 17, 2023-2024