-
Transformer meets wcDTW to improve real-time battery bids: A new approach to scenario selection
Authors:
Sujal Bhavsar,
Vera Zaychik Moffitt,
Justin Appleby
Abstract:
Stochastic battery bidding in real-time energy markets is a nuanced process, with its efficacy depending on the accuracy of forecasts and the representative scenarios chosen for optimization. In this paper, we introduce a pioneering methodology that amalgamates Transformer-based forecasting with weighted constrained Dynamic Time Warping (wcDTW) to refine scenario selection. Our approach harnesses…
▽ More
Stochastic battery bidding in real-time energy markets is a nuanced process, with its efficacy depending on the accuracy of forecasts and the representative scenarios chosen for optimization. In this paper, we introduce a pioneering methodology that amalgamates Transformer-based forecasting with weighted constrained Dynamic Time Warping (wcDTW) to refine scenario selection. Our approach harnesses the predictive capabilities of Transformers to foresee Energy prices, while wcDTW ensures the selection of pertinent historical scenarios by maintaining the coherence between multiple uncertain products. Through extensive simulations in the PJM market for July 2023, our method exhibited a 10% increase in revenue compared to the conventional method, highlighting its potential to revolutionize battery bidding strategies in real-time markets.
△ Less
Submitted 2 April, 2024;
originally announced April 2024.
-
Zooming in on NYC taxi data with Portal
Authors:
Julia Stoyanovich,
Matthew Gilbride,
Vera Zaychik Moffitt
Abstract:
In this paper we develop a methodology for analyzing transportation data at different levels of temporal and geographic granularity, and apply our methodology to the TLC Trip Record Dataset, made publicly available by the NYC Taxi & Limousine Commission. This data is naturally represented by a set of trajectories, annotated with time and with additional information such as passenger count and cost…
▽ More
In this paper we develop a methodology for analyzing transportation data at different levels of temporal and geographic granularity, and apply our methodology to the TLC Trip Record Dataset, made publicly available by the NYC Taxi & Limousine Commission. This data is naturally represented by a set of trajectories, annotated with time and with additional information such as passenger count and cost. We analyze TLC data to identify hotspots, which point to lack of convenient public transportation options, and popular routes, which motivate ride-sharing solutions or addition of a bus route.
Our methodology is based on using a system called Portal, which implements efficient representations and principled analysis methods for evolving graphs. Portal is implemented on top of Apache Spark, a popular distributed data processing system, is inter-operable with other Spark libraries like SparkSQL, and supports sophisticated kinds of analysis of evolving graphs efficiently. Portal is currently under development in the Data, Responsibly Lab at Drexel. We plan to release Portal in the open source in Fall 2017.
△ Less
Submitted 18 September, 2017;
originally announced September 2017.
-
Querying Evolving Graphs with Portal
Authors:
Vera Zaychik Moffitt,
Julia Stoyanovich
Abstract:
Graphs are used to represent a plethora of phenomena, from the Web and social networks, to biological pathways, to semantic knowledge bases. Arguably the most interesting and important questions one can ask about graphs have to do with their evolution. Which Web pages are showing an increasing popularity trend? How does influence propagate in social networks? How does knowledge evolve?
This pape…
▽ More
Graphs are used to represent a plethora of phenomena, from the Web and social networks, to biological pathways, to semantic knowledge bases. Arguably the most interesting and important questions one can ask about graphs have to do with their evolution. Which Web pages are showing an increasing popularity trend? How does influence propagate in social networks? How does knowledge evolve?
This paper proposes a logical model of an evolving graph called a TGraph, which captures evolution of graph topology and of its vertex and edge attributes. We present a compositional temporal graph algebra TGA, and show a reduction of TGA to temporal relational algebra with graph-specific primitives. We formally study the properties of TGA, and also show that it is sufficient to concisely express a wide range of common use cases. We describe an implementation of our model and algebra in Portal, built on top of Apache Spark / GraphX. We conduct extensive experiments on real datasets, and show that Portal scales.
△ Less
Submitted 12 December, 2016; v1 submitted 1 February, 2016;
originally announced February 2016.
-
Introducing Access Control in Webdamlog
Authors:
Serge Abiteboul,
Émilien Antoine,
Gerome Miklau,
Julia Stoyanovich,
Vera Zaychik Moffitt
Abstract:
We survey recent work on the specification of an access control mechanism in a collaborative environment. The work is presented in the context of the WebdamLog language, an extension of datalog to a distributed context. We discuss a fine-grained access control mechanism for intentional data based on provenance as well as a control mechanism for delegation, i.e., for deploying rules at remote peers…
▽ More
We survey recent work on the specification of an access control mechanism in a collaborative environment. The work is presented in the context of the WebdamLog language, an extension of datalog to a distributed context. We discuss a fine-grained access control mechanism for intentional data based on provenance as well as a control mechanism for delegation, i.e., for deploying rules at remote peers.
△ Less
Submitted 31 July, 2013;
originally announced July 2013.
