-
Indexing Analytics to Instances: How Integrating a Dashboard can Support Design Education
Authors:
Ajit Jain,
Andruid Kerne,
Nic Lupfer,
Gabriel Britain,
Aaron Perrine,
Yoonsuck Choe,
John Keyser,
Ruihong Huang,
Jinsil Seo,
Annie Sungkajun,
Robert Lightfoot,
Timothy McGuire
Abstract:
We investigate how to use AI-based analytics to support design education. The analytics at hand measure multiscale design, that is, students' use of space and scale to visually and conceptually organize their design work. With the goal of making the analytics intelligible to instructors, we developed a research artifact integrating a design analytics dashboard with design instances, and the design…
▽ More
We investigate how to use AI-based analytics to support design education. The analytics at hand measure multiscale design, that is, students' use of space and scale to visually and conceptually organize their design work. With the goal of making the analytics intelligible to instructors, we developed a research artifact integrating a design analytics dashboard with design instances, and the design environment that students use to create them. We theorize about how Suchman's notion of mutual intelligibility requires contextualized investigation of AI in order to develop findings about how analytics work for people. We studied the research artifact in 5 situated course contexts, in 3 departments. A total of 236 students used the multiscale design environment. The 9 instructors who taught those students experienced the analytics via the new research artifact.
We derive findings from a qualitative analysis of interviews with instructors regarding their experiences. Instructors reflected on how the analytics and their presentation in the dashboard have the potential to affect design education. We develop research implications addressing: (1) how indexing design analytics in the dashboard to actual design work instances helps design instructors reflect on what they mean and, more broadly, is a technique for how AI-based design analytics can support instructors' assessment and feedback experiences in situated course contexts; and (2) how multiscale design analytics, in particular, have the potential to support design education. By indexing, we mean linking which provides context, here connecting the numbers of the analytics with visually annotated design work instances.
△ Less
Submitted 8 April, 2024;
originally announced April 2024.
-
Limitations of P-Values and $R^2$ for Stepwise Regression Building: A Fairness Demonstration in Health Policy Risk Adjustment
Authors:
Sherri Rose,
Thomas G. McGuire
Abstract:
Stepwise regression building procedures are commonly used applied statistical tools, despite their well-known drawbacks. While many of their limitations have been widely discussed in the literature, other aspects of the use of individual statistical fit measures, especially in high-dimensional stepwise regression settings, have not. Giving primacy to individual fit, as is done with p-values and…
▽ More
Stepwise regression building procedures are commonly used applied statistical tools, despite their well-known drawbacks. While many of their limitations have been widely discussed in the literature, other aspects of the use of individual statistical fit measures, especially in high-dimensional stepwise regression settings, have not. Giving primacy to individual fit, as is done with p-values and $R^2$, when group fit may be the larger concern, can lead to misguided decision making. One of the most consequential uses of stepwise regression is in health care, where these tools allocate hundreds of billions of dollars to health plans enrolling individuals with different predicted health care costs. The main goal of this "risk adjustment" system is to convey incentives to health plans such that they provide health care services fairly, a component of which is not to discriminate in access or care for persons or groups likely to be expensive. We address some specific limitations of p-values and $R^2$ for high-dimensional stepwise regression in this policy problem through an illustrated example by additionally considering a group-level fairness metric.
△ Less
Submitted 6 August, 2018; v1 submitted 14 March, 2018;
originally announced March 2018.
-
Investigating the Evolvability of Web Page Load Time
Authors:
Brendan Cody-Kenny,
Umberto Manganiello,
John Farrelly,
Adrian Ronayne,
Eoghan Considine,
Thomas McGuire,
Michael O'Neill
Abstract:
Client-side Javascript execution environments (browsers) allow anonymous functions and event-based programming concepts such as callbacks. We investigate whether a mutate-and-test approach can be used to optimise web page load time in these environments. First, we characterise a web page load issue in a benchmark web page and derive performance metrics from page load event traces. We parse Javascr…
▽ More
Client-side Javascript execution environments (browsers) allow anonymous functions and event-based programming concepts such as callbacks. We investigate whether a mutate-and-test approach can be used to optimise web page load time in these environments. First, we characterise a web page load issue in a benchmark web page and derive performance metrics from page load event traces. We parse Javascript source code to an AST and make changes to method calls which appear in a web page load event trace. We present an operator based solely on code deletion and evaluate an existing "community-contributed" performance optimising code transform. By exploring Javascript code changes and exploiting combinations of non-destructive changes, we can optimise page load time by 41% in our benchmark web page.
△ Less
Submitted 22 February, 2018;
originally announced March 2018.
-
A Search for Improved Performance in Regular Expressions
Authors:
Brendan Cody-Kenny,
Michael Fenton,
Adrian Ronayne,
Eoghan Considine,
Thomas McGuire,
Michael O'Neill
Abstract:
The primary aim of automated performance improvement is to reduce the running time of programs while maintaining (or improving on) functionality. In this paper, Genetic Programming is used to find performance improvements in regular expressions for an array of target programs, representing the first application of automated software improvement for run-time performance in the Regular Expression la…
▽ More
The primary aim of automated performance improvement is to reduce the running time of programs while maintaining (or improving on) functionality. In this paper, Genetic Programming is used to find performance improvements in regular expressions for an array of target programs, representing the first application of automated software improvement for run-time performance in the Regular Expression language. This particular problem is interesting as there may be many possible alternative regular expressions which perform the same task while exhibiting subtle differences in performance. A benchmark suite of candidate regular expressions is proposed for improvement. We show that the application of Genetic Programming techniques can result in performance improvements in all cases.
As we start evolution from a known good regular expression, diversity is critical in escaping the local optima of the seed expression. In order to understand diversity during evolution we compare an initial population consisting of only seed programs with a population initialised using a combination of a single seed individual with individuals generated using PI Grow and Ramped-half-and-half initialisation mechanisms.
△ Less
Submitted 13 April, 2017;
originally announced April 2017.
-
Cloud Penetration Testing
Authors:
Ralph LaBarge,
Thomas McGuire
Abstract:
This paper presents the results of a series of penetration tests performed on the OpenStack Essex Cloud Management Software. Several different types of penetration tests were performed including network protocol and command line fuzzing, session hijacking and credential theft. Using these techniques exploitable vulnerabilities were discovered that could enable an attacker to gain access to restric…
▽ More
This paper presents the results of a series of penetration tests performed on the OpenStack Essex Cloud Management Software. Several different types of penetration tests were performed including network protocol and command line fuzzing, session hijacking and credential theft. Using these techniques exploitable vulnerabilities were discovered that could enable an attacker to gain access to restricted information contained on the OpenStack server, or to gain full administrative privileges on the server. Key recommendations to address these vulnerabilities are to use a secure protocol, such as HTTPS, for communications between a cloud user and the OpenStack Horizon Dashboard, to encrypt all files that store user or administrative login credentials, and to correct a software bug found in the OpenStack Cinder typedelete command.
△ Less
Submitted 9 January, 2013;
originally announced January 2013.
