Skip to main content

Showing 1–9 of 9 results for author: Siu, C

Searching in archive stat. Search in all archives.
.
  1. arXiv:2308.14004  [pdf, ps, other

    stat.ML cs.LG

    Online GentleAdaBoost -- Technical Report

    Authors: Chapman Siu

    Abstract: We study the online variant of GentleAdaboost, where we combine a weak learner to a strong learner in an online fashion. We provide an approach to extend the batch approach to an online approach with theoretical justifications through application of line search. Finally we compare our online boosting approach with other online approaches across a variety of benchmark datasets.

    Submitted 9 September, 2023; v1 submitted 27 August, 2023; originally announced August 2023.

  2. arXiv:2204.07821  [pdf, other

    math.ST cs.CG math.AT stat.ML

    Detection of Small Holes by the Scale-Invariant Robust Density-Aware Distance (RDAD) Filtration

    Authors: Chunyin Siu, Gennady Samorodnitsky, Christina Lee Yu, Andrey Yao

    Abstract: A novel topological-data-analytical (TDA) method is proposed to distinguish, from noise, small holes surrounded by high-density regions of a probability density function. The proposed method is robust against additive noise and outliers. Traditional TDA tools, like those based on the distance filtration, often struggle to distinguish small features from noise, because both have short persistences.… ▽ More

    Submitted 30 March, 2024; v1 submitted 16 April, 2022; originally announced April 2022.

    Comments: 39 pages, 38 figs, J Appl. and Comput. Topology (2024). GitHub: [github.com/c-siu/RDAD]. Published version: [rdcu.be/dCXLa]. Diff of v2/3: added publication info, NO post-submission improvements (Cor2-3 rephrased and proven, setup of Sec4.1 explained, complexity computed in Sec6.1, Thm5 simplified, comparison with DTM in Sec1,8, streamlining), so no change in pdf. Diff of v1/2: more thms, more discussion on conformality, fewer egs

    MSC Class: 62R40; 55N31; 52R40; 68T09

  3. arXiv:2109.09038  [pdf, other

    cs.LG cs.MA stat.ML

    Regularize! Don't Mix: Multi-Agent Reinforcement Learning without Explicit Centralized Structures

    Authors: Chapman Siu, Jason Traish, Richard Yi Da Xu

    Abstract: We propose using regularization for Multi-Agent Reinforcement Learning rather than learning explicit cooperative structures called {\em Multi-Agent Regularized Q-learning} (MARQ). Many MARL approaches leverage centralized structures in order to exploit global state information or removing communication constraints when the agents act in a decentralized manner. Instead of learning redundant structu… ▽ More

    Submitted 18 September, 2021; originally announced September 2021.

  4. arXiv:2109.09037  [pdf, other

    cs.LG stat.ML

    Dual Behavior Regularized Reinforcement Learning

    Authors: Chapman Siu, Jason Traish, Richard Yi Da Xu

    Abstract: Reinforcement learning has been shown to perform a range of complex tasks through interaction with an environment or collected leveraging experience. However, many of these approaches presume optimal or near optimal experiences or the presence of a consistent environment. In this work we propose dual, advantage-based behavior policy based on counterfactual regret minimization. We demonstrate the f… ▽ More

    Submitted 18 September, 2021; originally announced September 2021.

  5. arXiv:2109.09034  [pdf, other

    cs.LG cs.MA stat.ML

    Greedy UnMixing for Q-Learning in Multi-Agent Reinforcement Learning

    Authors: Chapman Siu, Jason Traish, Richard Yi Da Xu

    Abstract: This paper introduces Greedy UnMix (GUM) for cooperative multi-agent reinforcement learning (MARL). Greedy UnMix aims to avoid scenarios where MARL methods fail due to overestimation of values as part of the large joint state-action space. It aims to address this through a conservative Q-learning approach through restricting the state-marginal in the dataset to avoid unobserved joint state action… ▽ More

    Submitted 18 September, 2021; originally announced September 2021.

  6. arXiv:1909.11790  [pdf, other

    stat.ML cs.LG

    Residual Networks Behave Like Boosting Algorithms

    Authors: Chapman Siu

    Abstract: We show that Residual Networks (ResNet) is equivalent to boosting feature representation, without any modification to the underlying ResNet training algorithm. A regret bound based on Online Gradient Boosting theory is proved and suggests that ResNet could achieve Online Gradient Boosting regret bounds through neural network architectural changes with the addition of a shrinkage parameter in the i… ▽ More

    Submitted 25 September, 2019; originally announced September 2019.

    Comments: This work was supported by and completed whilst author was at Suncorp Group Limited. to appear 2019 IEEE International Conference on Data Science and Advance Analytics (DSAA)

  7. TreeGrad: Transferring Tree Ensembles to Neural Networks

    Authors: Chapman Siu

    Abstract: Gradient Boosting Decision Tree (GBDT) are popular machine learning algorithms with implementations such as LightGBM and in popular machine learning toolkits like Scikit-Learn. Many implementations can only produce trees in an offline manner and in a greedy manner. We explore ways to convert existing GBDT implementations to known neural network architectures with minimal performance loss in order… ▽ More

    Submitted 9 December, 2019; v1 submitted 24 April, 2019; originally announced April 2019.

    Comments: Technical Report on Implementation of Deep Neural Decision Forests Algorithm. To accompany implementation here: https://github.com/chappers/TreeGrad. Update: Please cite as: Siu, C. (2019). "Transferring Tree Ensembles to Neural Networks". International Conference on Neural Information Processing. Springer, 2019. arXiv admin note: text overlap with arXiv:1909.11790

  8. Automatic Induction of Neural Network Decision Tree Algorithms

    Authors: Chapman Siu

    Abstract: This work presents an approach to automatically induction for non-greedy decision trees constructed from neural network architecture. This construction can be used to transfer weights when growing or pruning a decision tree, allowing non-greedy decision tree algorithms to automatically learn and adapt to the ideal architecture. In this work, we examine the underpinning ideas within ensemble modell… ▽ More

    Submitted 24 April, 2019; v1 submitted 26 November, 2018; originally announced November 2018.

    Comments: This is a pre-print of a contribution "Chapman Siu, Automatic Induction of Neural Network Decision Tree Algorithms." To appear in Computing Conference 2019 Proceedings. Advances in Intelligent Systems and Computing. Implementation: https://github.com/chappers/automatic-induction-neural-decision-tree

  9. arXiv:1806.04308  [pdf, ps, other

    stat.ML cs.LG

    Diverse Online Feature Selection

    Authors: Chapman Siu, Richard Yi Da Xu

    Abstract: Online feature selection has been an active research area in recent years. We propose a novel diverse online feature selection method based on Determinantal Point Processes (DPP). Our model aims to provide diverse features which can be composed in either a supervised or unsupervised framework. The framework aims to promote diversity based on the kernel produced on a feature level, through at most… ▽ More

    Submitted 24 April, 2019; v1 submitted 11 June, 2018; originally announced June 2018.