-
VisAnatomy: An SVG Chart Corpus with Fine-Grained Semantic Labels
Authors:
Chen Chen,
Hannah K. Bako,
Peihong Yu,
John Hooker,
Jeffrey Joyal,
Simon C. Wang,
Samuel Kim,
Jessica Wu,
Aoxue Ding,
Lara Sandeep,
Alex Chen,
Chayanika Sinha,
Zhicheng Liu
Abstract:
Chart corpora, which comprise data visualizations and their semantic labels, are crucial for advancing visualization research. However, the labels in most existing corpora are high-level (e.g., chart types), hindering their utility for broader applications in the era of AI. In this paper, we contribute VisAnatomy, a chart corpus containing 942 real-world SVG charts produced by over 50 tools, encom…
▽ More
Chart corpora, which comprise data visualizations and their semantic labels, are crucial for advancing visualization research. However, the labels in most existing corpora are high-level (e.g., chart types), hindering their utility for broader applications in the era of AI. In this paper, we contribute VisAnatomy, a chart corpus containing 942 real-world SVG charts produced by over 50 tools, encompassing 40 chart types and featuring structural and stylistic design variations. The underlying data tables are also included if available. Each chart is augmented with multi-level fine-grained labels on its semantic components, including each graphical element's type, role, and position, hierarchical groupings of elements, group layouts, and visual encodings. In total, VisAnatomy provides labels for more than 383k graphical elements. We demonstrate the richness of the semantic labels by comparing VisAnatomy with existing corpora. We illustrate its usefulness through four applications: shape recognition for SVG elements, chart semantic decomposition, chart type classification, and content navigation for accessibility. Finally, we discuss our plan to improve VisAnatomy and research opportunities VisAnatomy presents.
△ Less
Submitted 2 April, 2025; v1 submitted 16 October, 2024;
originally announced October 2024.
-
Evaluating the Semantic Profiling Abilities of LLMs for Natural Language Utterances in Data Visualization
Authors:
Hannah K. Bako,
Arshnoor Bhutani,
Xinyi Liu,
Kwesi A. Cobbina,
Zhicheng Liu
Abstract:
Automatically generating data visualizations in response to human utterances on datasets necessitates a deep semantic understanding of the data utterance, including implicit and explicit references to data attributes, visualization tasks, and necessary data preparation steps. Natural Language Interfaces (NLIs) for data visualization have explored ways to infer such information, yet challenges pers…
▽ More
Automatically generating data visualizations in response to human utterances on datasets necessitates a deep semantic understanding of the data utterance, including implicit and explicit references to data attributes, visualization tasks, and necessary data preparation steps. Natural Language Interfaces (NLIs) for data visualization have explored ways to infer such information, yet challenges persist due to inherent uncertainty in human speech. Recent advances in Large Language Models (LLMs) provide an avenue to address these challenges, but their ability to extract the relevant semantic information remains unexplored. In this study, we evaluate four publicly available LLMs (GPT-4, Gemini-Pro, Llama3, and Mixtral), investigating their ability to comprehend utterances even in the presence of uncertainty and identify the relevant data context and visual tasks. Our findings reveal that LLMs are sensitive to uncertainties in utterances. Despite this sensitivity, they are able to extract the relevant data context. However, LLMs struggle with inferring visualization tasks. Based on these results, we highlight future research directions on using LLMs for visualization generation.
△ Less
Submitted 9 July, 2024; v1 submitted 8 July, 2024;
originally announced July 2024.
-
How do Observable Users Decompose D3 Code? A Qualitative Study
Authors:
Melissa Lin,
Heer Patel,
Medina Lamkin,
Hannah Bako,
Leilani Battle
Abstract:
Many toolkit developers seek to streamline the visualization programming process for their users through structured support such as prescribed templates and example galleries. However, few projects examine how users organize their own visualization programs, how their coding choices may deviate from the intents of toolkit developers, and how these differences may impact visualization prototyping a…
▽ More
Many toolkit developers seek to streamline the visualization programming process for their users through structured support such as prescribed templates and example galleries. However, few projects examine how users organize their own visualization programs, how their coding choices may deviate from the intents of toolkit developers, and how these differences may impact visualization prototyping and design. Further, is it possible to infer users' reasoning indirectly through their code, even when users copy code from other sources? Understanding these patterns can reveal opportunities to align toolkit design with actual user behavior, improving usability and supporting more flexible workflows. We explore this question through a qualitative analysis of 715 D3 programs on Observable. We identify three levels of program organization based on how users decompose their code into smaller blocks: Program-, Chart-, and Component-Level code decomposition, with a strong preference for Component-Level reasoning. In a series of interviews, we corroborate that these levels reflect how Observable users reason about visualization programs. We compare common user-made components with those theorized in the Grammar of Graphics to assess overlap in user and toolkit developer reasoning. We find that, while the Grammar of Graphics covers basic visualizations well, it falls short in describing complex visualization types, especially those with animation, interaction, and parameterization components. Our findings highlight how user practices differ from formal grammars and suggest opportunities for rethinking visualization toolkit support, including augmenting learning tools and AI assistants to better reflect real-world coding strategies.
△ Less
Submitted 24 May, 2025; v1 submitted 23 May, 2024;
originally announced May 2024.
-
Streamlining Visualization Authoring in D3 Through User-Driven Templates
Authors:
Hannah Bako,
Alisha Varma,
Anuoluwapo Faboro,
Mahreen Haider,
Favour Nerrise,
Bissaka Kenah,
Leilani Battle
Abstract:
D3 is arguably the most popular tool for implementing web based visualizations. Yet D3 has a steep learning curve that may hinder its adoption and continued use. To simplify the process of programming D3 visualizations, we must first understand the space of implementation practices that D3 users engage in. We present a qualitative analysis of 2500 D3 visualizations and their corresponding implemen…
▽ More
D3 is arguably the most popular tool for implementing web based visualizations. Yet D3 has a steep learning curve that may hinder its adoption and continued use. To simplify the process of programming D3 visualizations, we must first understand the space of implementation practices that D3 users engage in. We present a qualitative analysis of 2500 D3 visualizations and their corresponding implementations. We find that 5 visualization types (Bar Charts, Geomaps, Line Charts, Scatterplots, and Force Directed Graphs) account for 80% of D3 visualizations found in our corpus. While implementation styles vary slightly across designs, the underlying code structure for all visualization types remains the same; presenting an opportunity for code reuse. Using our corpus of D3 examples, we synthesize reusable code templates for eight popular D3 visualization types and share them in our open source repository. Based on our results, we discuss design considerations for leveraging users' implementation patterns to reduce visualization design effort through design templates and auto-generated code recommendations.
△ Less
Submitted 13 July, 2022;
originally announced July 2022.
-
User-Driven Support for Visualization Prototyping in D3
Authors:
Hannah K. Bako,
Alisha Varma,
Anuoluwapo Faboro,
Mahreen Haider,
Favour Nerrise,
Bissaka Kenah,
John P. Dickerson,
Leilani Battle
Abstract:
Templates have emerged as an effective approach to simplifying the visualization design and programming process. For example, they enable users to quickly generate multiple visualization designs even when using complex toolkits like D3. However, these templates are often treated as rigid artifacts that respond poorly to changes made outside of the template's established parameters, limiting user c…
▽ More
Templates have emerged as an effective approach to simplifying the visualization design and programming process. For example, they enable users to quickly generate multiple visualization designs even when using complex toolkits like D3. However, these templates are often treated as rigid artifacts that respond poorly to changes made outside of the template's established parameters, limiting user creativity. Preserving the user's creative flow requires a more dynamic approach to template-based visualization design, where tools can respond gracefully to users' edits when they modify templates in unexpected ways. In this paper, we leverage the structural similarities revealed by templates to design resilient support features for prototyping D3 visualizations: recommendations to suggest complementary interactions for a user's D3 program; and code augmentation to implement recommended interactions with a single click, even when users deviate from pre-defined templates. We demonstrate the utility of these features in Mirny, a d design-focused prototyping environment for D3. In a user study with 20 D3 users, we find that these automated features enable participants to prototype their design ideas with significantly fewer programming iterations. We also characterize key modification strategies used by participants to customize D3 templates. Informed by our findings and participants' feedback, we discuss the key implications of the use of templates for interleaving visualization programming and design.
△ Less
Submitted 21 February, 2023; v1 submitted 6 December, 2021;
originally announced December 2021.
