-
The Formal Semantics and Implementation of a Domain-Specific Language for Mixed-Initiative Dialogs
Authors:
Zachary S. Rowland,
Saverio Perugini
Abstract:
Human-computer dialog plays a prominent role in interactions conducted at kiosks (e.g., withdrawing money from an atm or filling your car with gas), on smartphones (e.g., installing and configuring apps), and on the web (e.g., booking a flight). Some human-computer dialogs involve an exchange of system-initiated and user-initiated actions. These dialogs are called *mixed-initiative dialogs* and so…
▽ More
Human-computer dialog plays a prominent role in interactions conducted at kiosks (e.g., withdrawing money from an atm or filling your car with gas), on smartphones (e.g., installing and configuring apps), and on the web (e.g., booking a flight). Some human-computer dialogs involve an exchange of system-initiated and user-initiated actions. These dialogs are called *mixed-initiative dialogs* and sometimes also involve the pursuit of multiple interleaved sub-dialogs, which are woven together in a manner akin to coroutines. However, existing dialog-authoring languages have difficulty expressing these dialogs concisely. In this work, we improve the expressiveness of a dialog-authoring language we call *dialog specification language* (dsl), which is based on the programming concepts of functional application, partial function application, currying, and partial evaluation, by augmenting it with additional abstractions to support concise specification of task-based, mixed-initiative dialogs that resemble concurrently executing coroutines. We also formalize the semantics of dsl -- the process of simplifying and staging such dialogs specified in the language. We demonstrate that dialog specifications are compressed by to a higher degree when written in dsl using the new abstractions. We also operationalize the formal semantics of dsl in a Haskell functional programming implementation. The Haskell implementation of the simplification/staging rules provides a proof of concept that the formal semantics are sufficient to implement a dialog system specified with the language. We evaluate dsl from practical (i.e., case study), conceptual (i.e., comparisons to similar systems such as VoiceXML and State Chart XML), and theoretical perspectives. The practical applicability of the new language abstractions introduced in this work is demonstrated in a case study by using it to model portions of an online food ordering system that can be concurrently staged. Our results indicate that dsl enables concise representation of dialogs composed of multiple concurrent sub-dialogs and improves the compression of dialog expressions reported in prior research. We anticipate that the extension of our language and the formalization of the semantics can facilitate concise specification and smooth implementation of task-based, mixed-initiative, human-computer dialog systems across various domains such as atms and interactive, voice-response systems.
△ Less
Submitted 27 February, 2025;
originally announced February 2025.
-
An Interactive, Graphical CPU Scheduling Simulator for Teaching Operating Systems
Authors:
Joshua W. Buck,
Saverio Perugini
Abstract:
We present a graphical simulation tool for visually and interactively exploring the processing of various events handled by an operating system when running a program. Our graphical simulator is available for use on the web and locally by both instructors and students for purposes of pedagogy. Instructors can use it for live demonstrations of course concepts in class, while students can use it out…
▽ More
We present a graphical simulation tool for visually and interactively exploring the processing of various events handled by an operating system when running a program. Our graphical simulator is available for use on the web and locally by both instructors and students for purposes of pedagogy. Instructors can use it for live demonstrations of course concepts in class, while students can use it outside of class to explore the concepts. The graphical simulation tool is implemented using the React library for the fancy ui elements of the Node.js framework and is available as a single page web application at https://cpudemo.azurewebsites.net. Assigning the development of the underling text-based simulation engine, on which the graphical simulator runs, to students as a course project is also an effective approach to teach students the concepts. The goals of this paper are to showcase the demonstrative capabilities of the tool for instruction, share student experiences in developing the engine underlying the simulation, and to inspire its use by other educators.
△ Less
Submitted 1 November, 2019; v1 submitted 12 December, 2018;
originally announced December 2018.
-
Staging Human-computer Dialogs: An Application of the Futamura Projections
Authors:
Brandon M. Williams,
Saverio Perugini
Abstract:
We demonstrate an application of the Futamura Projections to human-computer interaction, and particularly to staging human-computer dialogs. Specifically, by providing staging analogs to the classical Futamura Projections, we demonstrate that the Futamura Projections can be applied to the staging of human-computer dialogs in addition to the execution of programs.
We demonstrate an application of the Futamura Projections to human-computer interaction, and particularly to staging human-computer dialogs. Specifically, by providing staging analogs to the classical Futamura Projections, we demonstrate that the Futamura Projections can be applied to the staging of human-computer dialogs in addition to the execution of programs.
△ Less
Submitted 13 November, 2018;
originally announced November 2018.
-
Parareal Algorithm Implementation and Simulation in Julia
Authors:
Tyler M. Masthay,
Saverio Perugini
Abstract:
We present a full implementation of the parareal algorithm---an integration technique to solve differential equations in parallel---in the Julia programming language for a fully general, first-order, initial-value problem. We provide a brief overview of Julia---a concurrent programming language for scientific computing. Our implementation of the parareal algorithm accepts both coarse and fine inte…
▽ More
We present a full implementation of the parareal algorithm---an integration technique to solve differential equations in parallel---in the Julia programming language for a fully general, first-order, initial-value problem. We provide a brief overview of Julia---a concurrent programming language for scientific computing. Our implementation of the parareal algorithm accepts both coarse and fine integrators as functional arguments. We use Euler's method and another Runge-Kutta integration technique as the integrators in our experiments. We also present a simulation of the algorithm for purposes of pedagogy and as a tool for investigating the performance of the algorithm.
△ Less
Submitted 13 December, 2018; v1 submitted 26 June, 2017;
originally announced June 2017.
-
Revisiting the Futamura Projections: A Diagrammatic Approach
Authors:
Brandon M. Williams,
Saverio Perugini
Abstract:
The advent of language implementation tools such as PyPy and Truffle/Graal have reinvigorated and broadened interest in topics related to automatic compiler generation and optimization. Given this broader interest, we revisit the Futamura Projections using a novel diagram scheme. Through these diagrams we emphasize the recurring patterns in the Futamura Projections while addressing their complexit…
▽ More
The advent of language implementation tools such as PyPy and Truffle/Graal have reinvigorated and broadened interest in topics related to automatic compiler generation and optimization. Given this broader interest, we revisit the Futamura Projections using a novel diagram scheme. Through these diagrams we emphasize the recurring patterns in the Futamura Projections while addressing their complexity and abstract nature. We anticipate that this approach will improve the accessibility of the Futamura Projections and help foster analysis of those new tools through the lens of partial evaluation.
△ Less
Submitted 20 March, 2018; v1 submitted 29 November, 2016;
originally announced November 2016.
-
An Immersive Telepresence System using RGB-D Sensors and Head Mounted Display
Authors:
Xinzhong Lu,
Ju Shen,
Saverio Perugini,
Jianjun Yang
Abstract:
We present a tele-immersive system that enables people to interact with each other in a virtual world using body gestures in addition to verbal communication. Beyond the obvious applications, including general online conversations and gaming, we hypothesize that our proposed system would be particularly beneficial to education by offering rich visual contents and interactivity. One distinct featur…
▽ More
We present a tele-immersive system that enables people to interact with each other in a virtual world using body gestures in addition to verbal communication. Beyond the obvious applications, including general online conversations and gaming, we hypothesize that our proposed system would be particularly beneficial to education by offering rich visual contents and interactivity. One distinct feature is the integration of egocentric pose recognition that allows participants to use their gestures to demonstrate and manipulate virtual objects simultaneously. This functionality enables the instructor to ef- fectively and efficiently explain and illustrate complex concepts or sophisticated problems in an intuitive manner. The highly interactive and flexible environment can capture and sustain more student attention than the traditional classroom setting and, thus, delivers a compelling experience to the students. Our main focus here is to investigate possible solutions for the system design and implementation and devise strategies for fast, efficient computation suitable for visual data processing and network transmission. We describe the technique and experiments in details and provide quantitative performance results, demonstrating our system can be run comfortably and reliably for different application scenarios. Our preliminary results are promising and demonstrate the potential for more compelling directions in cyberlearning.
△ Less
Submitted 20 November, 2015;
originally announced November 2015.
-
Specifying and Staging Mixed-Initiative Dialogs with Program Generation and Transformation
Authors:
Saverio Perugini
Abstract:
Specifying and implementing flexible human-computer dialogs, such as those used in kiosks and smart phone apps, is challenging because of the numerous and varied directions in which each user might steer a dialog. The objective of this research is to improve dialog specification and implementation. To do so we enriched a notation based on concepts from programming languages, especially partial eva…
▽ More
Specifying and implementing flexible human-computer dialogs, such as those used in kiosks and smart phone apps, is challenging because of the numerous and varied directions in which each user might steer a dialog. The objective of this research is to improve dialog specification and implementation. To do so we enriched a notation based on concepts from programming languages, especially partial evaluation, for specifying a variety of unsolicited reporting, mixed-initiative dialogs in a concise representation that serves as a design for dialog implementation. We also built a dialog mining system that extracts a specification in this notation from requirements. To demonstrate that such a specification provides a design for dialog implementation, we built a system that automatically generates an implementation of the dialog, called a stager, from it. These two components constitute a dialog modeling toolkit that automates dialog specification and implementation. These results provide a proof of concept and demonstrate the study of dialog specification and implementation from a programming languages perspective. The ubiquity of dialogs in domains such as travel, education, and health care combined with the demand for smart phone apps provide a landscape for further investigation of these results.
△ Less
Submitted 21 December, 2015; v1 submitted 1 August, 2011;
originally announced August 2011.
-
Automatically Generating Interfaces for Personalized Interaction with Digital Libraries
Authors:
Saverio Perugini,
Naren Ramakrishnan,
Edward A. Fox
Abstract:
We present an approach to automatically generate interfaces supporting personalized interaction with digital libraries; these interfaces augment the user-DL dialog by empowering the user to (optionally) supply out-of-turn information during an interaction, flatten or restructure the dialog, and enquire about dialog options. Interfaces generated using this approach for CITIDEL are described.
We present an approach to automatically generate interfaces supporting personalized interaction with digital libraries; these interfaces augment the user-DL dialog by empowering the user to (optionally) supply out-of-turn information during an interaction, flatten or restructure the dialog, and enquire about dialog options. Interfaces generated using this approach for CITIDEL are described.
△ Less
Submitted 11 February, 2004;
originally announced February 2004.
-
Taking the Initiative with Extempore: Exploring Out-of-Turn Interactions with Websites
Authors:
Saverio Perugini,
Mary E. Pinney,
Naren Ramakrishnan,
Manuel A. Perez-Quinones,
Mary Beth Rosson
Abstract:
We present the first study to explore the use of out-of-turn interaction in websites. Out-of-turn interaction is a technique which empowers the user to supply unsolicited information while browsing. This approach helps flexibly bridge any mental mismatch between the user and the website, in a manner fundamentally different from faceted browsing and site-specific search tools. We built a user int…
▽ More
We present the first study to explore the use of out-of-turn interaction in websites. Out-of-turn interaction is a technique which empowers the user to supply unsolicited information while browsing. This approach helps flexibly bridge any mental mismatch between the user and the website, in a manner fundamentally different from faceted browsing and site-specific search tools. We built a user interface (Extempore) which accepts out-of-turn input via voice or text; and employed it in a US congressional website, to determine if users utilize out-of-turn interaction for information-finding tasks, and their rationale for doing so. The results indicate that users are adept at discerning when out-of-turn interaction is necessary in a particular task, and actively interleaved it with browsing. However, users found cascading information across information-finding subtasks challenging. Therefore, this work not only improves our understanding of out-of-turn interaction, but also suggests further opportunities to enrich browsing experiences for users.
△ Less
Submitted 8 December, 2003;
originally announced December 2003.
-
Staging Transformations for Multimodal Web Interaction Management
Authors:
Michael Narayan,
Chris Williams,
Saverio Perugini,
Naren Ramakrishnan
Abstract:
Multimodal interfaces are becoming increasingly ubiquitous with the advent of mobile devices, accessibility considerations, and novel software technologies that combine diverse interaction media. In addition to improving access and delivery capabilities, such interfaces enable flexible and personalized dialogs with websites, much like a conversation between humans. In this paper, we present a so…
▽ More
Multimodal interfaces are becoming increasingly ubiquitous with the advent of mobile devices, accessibility considerations, and novel software technologies that combine diverse interaction media. In addition to improving access and delivery capabilities, such interfaces enable flexible and personalized dialogs with websites, much like a conversation between humans. In this paper, we present a software framework for multimodal web interaction management that supports mixed-initiative dialogs between users and websites. A mixed-initiative dialog is one where the user and the website take turns changing the flow of interaction. The framework supports the functional specification and realization of such dialogs using staging transformations -- a theory for representing and reasoning about dialogs based on partial input. It supports multiple interaction interfaces, and offers sessioning, caching, and co-ordination functions through the use of an interaction manager. Two case studies are presented to illustrate the promise of this approach.
△ Less
Submitted 20 November, 2003;
originally announced November 2003.
-
A Connection-Centric Survey of Recommender Systems Research
Authors:
Saverio Perugini,
Marcos Andre Goncalves,
Edward A. Fox
Abstract:
Recommender systems attempt to reduce information overload and retain customers by selecting a subset of items from a universal set based on user preferences. While research in recommender systems grew out of information retrieval and filtering, the topic has steadily advanced into a legitimate and challenging research area of its own. Recommender systems have traditionally been studied from a c…
▽ More
Recommender systems attempt to reduce information overload and retain customers by selecting a subset of items from a universal set based on user preferences. While research in recommender systems grew out of information retrieval and filtering, the topic has steadily advanced into a legitimate and challenging research area of its own. Recommender systems have traditionally been studied from a content-based filtering vs. collaborative design perspective. Recommendations, however, are not delivered within a vacuum, but rather cast within an informal community of users and social context. Therefore, ultimately all recommender systems make connections among people and thus should be surveyed from such a perspective. This viewpoint is under-emphasized in the recommender systems literature. We therefore take a connection-oriented viewpoint toward recommender systems research. We posit that recommendation has an inherently social element and is ultimately intended to connect people either directly as a result of explicit user modeling or indirectly through the discovery of relationships implicit in extant data. Thus, recommender systems are characterized by how they model users to bring people together: explicitly or implicitly. Finally, user modeling and the connection-centric viewpoint raise broadening and social issues--such as evaluation, targeting, and privacy and trust--which we also briefly address.
△ Less
Submitted 29 July, 2003; v1 submitted 22 May, 2002;
originally announced May 2002.
-
The Partial Evaluation Approach to Information Personalization
Authors:
Naren Ramakrishnan,
Saverio Perugini
Abstract:
Information personalization refers to the automatic adjustment of information content, structure, and presentation tailored to an individual user. By reducing information overload and customizing information access, personalization systems have emerged as an important segment of the Internet economy. This paper presents a systematic modeling methodology - PIPE (`Personalization is Partial Evalua…
▽ More
Information personalization refers to the automatic adjustment of information content, structure, and presentation tailored to an individual user. By reducing information overload and customizing information access, personalization systems have emerged as an important segment of the Internet economy. This paper presents a systematic modeling methodology - PIPE (`Personalization is Partial Evaluation') - for personalization. Personalization systems are designed and implemented in PIPE by modeling an information-seeking interaction in a programmatic representation. The representation supports the description of information-seeking activities as partial information and their subsequent realization by partial evaluation, a technique for specializing programs. We describe the modeling methodology at a conceptual level and outline representational choices. We present two application case studies that use PIPE for personalizing web sites and describe how PIPE suggests a novel evaluation criterion for information system designs. Finally, we mention several fundamental implications of adopting the PIPE model for personalization and when it is (and is not) applicable.
△ Less
Submitted 7 August, 2001;
originally announced August 2001.
