-
Aligning Crowd-sourced Human Feedback for Reinforcement Learning on Code Generation by Large Language Models
Authors:
Man Fai Wong,
Chee Wei Tan
Abstract:
This paper studies how AI-assisted programming and large language models (LLM) improve software developers' ability via AI tools (LLM agents) like Github Copilot and Amazon CodeWhisperer, while integrating human feedback to enhance reinforcement learning (RLHF) with crowd-sourced computation to enhance text-to-code generation. Additionally, we demonstrate that our Bayesian optimization framework s…
▽ More
This paper studies how AI-assisted programming and large language models (LLM) improve software developers' ability via AI tools (LLM agents) like Github Copilot and Amazon CodeWhisperer, while integrating human feedback to enhance reinforcement learning (RLHF) with crowd-sourced computation to enhance text-to-code generation. Additionally, we demonstrate that our Bayesian optimization framework supports AI alignment in code generation by distributing the feedback collection burden, highlighting the value of collecting human feedback of good quality. Our empirical evaluations demonstrate the efficacy of this approach, showcasing how LLM agents can be effectively trained for improved text-to-code generation. Our Bayesian optimization framework can be designed for general domain-specific languages, promoting the alignment of large language model capabilities with human feedback in AI-assisted programming for code generation.
△ Less
Submitted 19 March, 2025;
originally announced March 2025.
-
Copilot for Xcode: Exploring AI-Assisted Programming by Prompting Cloud-based Large Language Models
Authors:
Chee Wei Tan,
Shangxin Guo,
Man Fai Wong,
Ching Nam Hang
Abstract:
This paper presents an AI-assisted programming tool called Copilot for Xcode for program composition and design to support human software developers. By seamlessly integrating cloud-based Large Language Models (LLM) with Apple's local development environment, Xcode, this tool enhances productivity and unleashes creativity for software development in Apple software ecosystem (e.g., iOS apps, macOS)…
▽ More
This paper presents an AI-assisted programming tool called Copilot for Xcode for program composition and design to support human software developers. By seamlessly integrating cloud-based Large Language Models (LLM) with Apple's local development environment, Xcode, this tool enhances productivity and unleashes creativity for software development in Apple software ecosystem (e.g., iOS apps, macOS). Leveraging advanced natural language processing (NLP) techniques, Copilot for Xcode effectively processes source code tokens and patterns within code repositories, enabling features such as code generation, autocompletion, documentation, and error detection. Software developers can also query and make "small" decisions for program composition, some of which can be made simultaneously, and this is facilitated through prompt engineering in a chat interface of Copilot for Xcode. Finally, we present simple case studies as evidence of the effectiveness of utilizing NLP in Xcode to prompt popular LLM services like OpenAI ChatGPT for program composition and design.
△ Less
Submitted 8 July, 2023;
originally announced July 2023.
-
Natural Language Generation and Understanding of Big Code for AI-Assisted Programming: A Review
Authors:
Man Fai Wong,
Shangxin Guo,
Ching Nam Hang,
Siu Wai Ho,
Chee Wei Tan
Abstract:
This paper provides a comprehensive review of the literature concerning the utilization of Natural Language Processing (NLP) techniques, with a particular focus on transformer-based large language models (LLMs) trained using Big Code, within the domain of AI-assisted programming tasks. LLMs, augmented with software naturalness, have played a crucial role in facilitating AI-assisted programming app…
▽ More
This paper provides a comprehensive review of the literature concerning the utilization of Natural Language Processing (NLP) techniques, with a particular focus on transformer-based large language models (LLMs) trained using Big Code, within the domain of AI-assisted programming tasks. LLMs, augmented with software naturalness, have played a crucial role in facilitating AI-assisted programming applications, including code generation, code completion, code translation, code refinement, code summarization, defect detection, and clone detection. Notable examples of such applications include the GitHub Copilot powered by OpenAI's Codex and DeepMind AlphaCode. This paper presents an overview of the major LLMs and their applications in downstream tasks related to AI-assisted programming. Furthermore, it explores the challenges and opportunities associated with incorporating NLP techniques with software naturalness in these applications, with a discussion on extending AI-assisted programming capabilities to Apple's Xcode for mobile software development. This paper also presents the challenges of and opportunities for incorporating NLP techniques with software naturalness, empowering developers with advanced coding assistance and streamlining the software development process.
△ Less
Submitted 4 July, 2023;
originally announced July 2023.
-
EuclidNet: Deep Visual Reasoning for Constructible Problems in Geometry
Authors:
Man Fai Wong,
Xintong Qi,
Chee Wei Tan
Abstract:
In this paper, we present a deep learning-based framework for solving geometric construction problems through visual reasoning, which is useful for automated geometry theorem proving. Constructible problems in geometry often ask for the sequence of straightedge-and-compass constructions to construct a given goal given some initial setup. Our EuclidNet framework leverages the neural network archite…
▽ More
In this paper, we present a deep learning-based framework for solving geometric construction problems through visual reasoning, which is useful for automated geometry theorem proving. Constructible problems in geometry often ask for the sequence of straightedge-and-compass constructions to construct a given goal given some initial setup. Our EuclidNet framework leverages the neural network architecture Mask R-CNN to extract the visual features from the initial setup and goal configuration with extra points of intersection, and then generate possible construction steps as intermediary data models that are used as feedback in the training process for further refinement of the construction step sequence. This process is repeated recursively until either a solution is found, in which case we backtrack the path for a step-by-step construction guide, or the problem is identified as unsolvable. Our EuclidNet framework is validated on complex Japanese Sangaku geometry problems, demonstrating its capacity to leverage backtracking for deep visual reasoning of challenging problems.
△ Less
Submitted 27 December, 2022;
originally announced January 2023.
-
Adaptive Reduced Rank Regression
Authors:
Qiong Wu,
Felix Ming Fai Wong,
Zhenming Liu,
Yanhua Li,
Varun Kanade
Abstract:
We study the low rank regression problem $\my = M\mx + ε$, where $\mx$ and $\my$ are $d_1$ and $d_2$ dimensional vectors respectively. We consider the extreme high-dimensional setting where the number of observations $n$ is less than $d_1 + d_2$. Existing algorithms are designed for settings where $n$ is typically as large as $\Rank(M)(d_1+d_2)$. This work provides an efficient algorithm which onl…
▽ More
We study the low rank regression problem $\my = M\mx + ε$, where $\mx$ and $\my$ are $d_1$ and $d_2$ dimensional vectors respectively. We consider the extreme high-dimensional setting where the number of observations $n$ is less than $d_1 + d_2$. Existing algorithms are designed for settings where $n$ is typically as large as $\Rank(M)(d_1+d_2)$. This work provides an efficient algorithm which only involves two SVD, and establishes statistical guarantees on its performance. The algorithm decouples the problem by first estimating the precision matrix of the features, and then solving the matrix denoising problem. To complement the upper bound, we introduce new techniques for establishing lower bounds on the performance of any algorithm for this problem. Our preliminary experiments confirm that our algorithm often out-performs existing baselines, and is always at least competitive.
△ Less
Submitted 23 October, 2020; v1 submitted 27 May, 2019;
originally announced May 2019.
-
Stock Market Prediction from WSJ: Text Mining via Sparse Matrix Factorization
Authors:
Felix Ming Fai Wong,
Zhenming Liu,
Mung Chiang
Abstract:
We revisit the problem of predicting directional movements of stock prices based on news articles: here our algorithm uses daily articles from The Wall Street Journal to predict the closing stock prices on the same day. We propose a unified latent space model to characterize the "co-movements" between stock prices and news articles. Unlike many existing approaches, our new model is able to simulta…
▽ More
We revisit the problem of predicting directional movements of stock prices based on news articles: here our algorithm uses daily articles from The Wall Street Journal to predict the closing stock prices on the same day. We propose a unified latent space model to characterize the "co-movements" between stock prices and news articles. Unlike many existing approaches, our new model is able to simultaneously leverage the correlations: (a) among stock prices, (b) among news articles, and (c) between stock prices and news articles. Thus, our model is able to make daily predictions on more than 500 stocks (most of which are not even mentioned in any news article) while having low complexity. We carry out extensive backtesting on trading strategies based on our algorithm. The result shows that our model has substantially better accuracy rate (55.7%) compared to many widely used algorithms. The return (56%) and Sharpe ratio due to a trading strategy based on our model are also much higher than baseline indices.
△ Less
Submitted 27 June, 2014;
originally announced June 2014.
-
Mind Your Own Bandwidth: An Edge Solution to Peak-hour Broadband Congestion
Authors:
Felix Ming Fai Wong,
Carlee Joe-Wong,
Sangtae Ha,
Zhenming Liu,
Mung Chiang
Abstract:
Motivated by recent increases in network traffic, we propose a decentralized network edge-based solution to peak-hour broadband congestion that incentivizes users to moderate their bandwidth demands to their actual needs. Our solution is centered on smart home gateways that allocate bandwidth in a two-level hierarchy: first, a gateway purchases guaranteed bandwidth from the Internet Service Provid…
▽ More
Motivated by recent increases in network traffic, we propose a decentralized network edge-based solution to peak-hour broadband congestion that incentivizes users to moderate their bandwidth demands to their actual needs. Our solution is centered on smart home gateways that allocate bandwidth in a two-level hierarchy: first, a gateway purchases guaranteed bandwidth from the Internet Service Provider (ISP) with virtual credits. It then self-limits its bandwidth usage and distributes the bandwidth among its apps and devices according to their relative priorities. To this end, we design a credit allocation and redistribution mechanism for the first level, and implement our gateways on commodity wireless routers for the second level. We demonstrate our system's effectiveness and practicality with theoretical analysis, simulations and experiments on real traffic. Compared to a baseline equal sharing algorithm, our solution significantly improves users' overall satisfaction and yields a fair allocation of bandwidth across users.
△ Less
Submitted 30 December, 2013;
originally announced December 2013.
-
Learning about social learning in MOOCs: From statistical analysis to generative model
Authors:
Christopher G. Brinton,
Mung Chiang,
Shaili Jain,
Henry Lam,
Zhenming Liu,
Felix Ming Fai Wong
Abstract:
We study user behavior in the courses offered by a major Massive Online Open Course (MOOC) provider during the summer of 2013. Since social learning is a key element of scalable education in MOOCs and is done via online discussion forums, our main focus is in understanding forum activities. Two salient features of MOOC forum activities drive our research: 1. High decline rate: for all courses stud…
▽ More
We study user behavior in the courses offered by a major Massive Online Open Course (MOOC) provider during the summer of 2013. Since social learning is a key element of scalable education in MOOCs and is done via online discussion forums, our main focus is in understanding forum activities. Two salient features of MOOC forum activities drive our research: 1. High decline rate: for all courses studied, the volume of discussions in the forum declines continuously throughout the duration of the course. 2. High-volume, noisy discussions: at least 30% of the courses produce new discussion threads at rates that are infeasible for students or teaching staff to read through. Furthermore, a substantial portion of the discussions are not directly course-related.
We investigate factors that correlate with the decline of activity in the online discussion forums and find effective strategies to classify threads and rank their relevance. Specifically, we use linear regression models to analyze the time series of the count data for the forum activities and make a number of observations, e.g., the teaching staff's active participation in the discussion increases the discussion volume but does not slow down the decline rate. We then propose a unified generative model for the discussion threads, which allows us both to choose efficient thread classifiers and design an effective algorithm for ranking thread relevance. Our ranking algorithm is further compared against two baseline algorithms, using human evaluation from Amazon Mechanical Turk.
The authors on this paper are listed in alphabetical order. For media and press coverage, please refer to us collectively, as "researchers from the EDGE Lab at Princeton University, together with collaborators at Boston University and Microsoft Corporation."
△ Less
Submitted 19 December, 2013; v1 submitted 7 December, 2013;
originally announced December 2013.
-
Why Watching Movie Tweets Won't Tell the Whole Story?
Authors:
Felix Ming Fai Wong,
Soumya Sen,
Mung Chiang
Abstract:
Data from Online Social Networks (OSNs) are providing analysts with an unprecedented access to public opinion on elections, news, movies etc. However, caution must be taken to determine whether and how much of the opinion extracted from OSN user data is indeed reflective of the opinion of the larger online population. In this work we study this issue in the context of movie reviews on Twitter and…
▽ More
Data from Online Social Networks (OSNs) are providing analysts with an unprecedented access to public opinion on elections, news, movies etc. However, caution must be taken to determine whether and how much of the opinion extracted from OSN user data is indeed reflective of the opinion of the larger online population. In this work we study this issue in the context of movie reviews on Twitter and compare the opinion of Twitter users with that of the online population of IMDb and Rotten Tomatoes. We introduce new metrics to show that the Twitter users can be characteristically different from general users, both in their rating and their relative preference for Oscar-nominated and non-nominated movies. Additionally, we investigate whether such data can truly predict a movie's box-office success.
△ Less
Submitted 20 March, 2012;
originally announced March 2012.
