Showing 1–10 of 10 results for author: Carmi, A

Humanity's Last Exam

Authors: Long Phan, Alice Gatti, Ziwen Han, Nathaniel Li, Josephina Hu, Hugh Zhang, Chen Bo Calvin Zhang, Mohamed Shaaban, John Ling, Sean Shi, Michael Choi, Anish Agrawal, Arnav Chopra, Adam Khoja, Ryan Kim, Richard Ren, Jason Hausenloy, Oliver Zhang, Mantas Mazeika, Dmitry Dodonov, Tung Nguyen, Jaeho Lee, Daron Anderson, Mikhail Doroshenko, Alun Cennyth Stokes , et al. (1084 additional authors not shown)

Abstract: Benchmarks are important tools for tracking the rapid advancements in large language model (LLM) capabilities. However, benchmarks are not keeping pace in difficulty: LLMs now achieve over 90\% accuracy on popular benchmarks like MMLU, limiting informed measurement of state-of-the-art LLM capabilities. In response, we introduce Humanity's Last Exam (HLE), a multi-modal benchmark at the frontier of… ▽ More Benchmarks are important tools for tracking the rapid advancements in large language model (LLM) capabilities. However, benchmarks are not keeping pace in difficulty: LLMs now achieve over 90\% accuracy on popular benchmarks like MMLU, limiting informed measurement of state-of-the-art LLM capabilities. In response, we introduce Humanity's Last Exam (HLE), a multi-modal benchmark at the frontier of human knowledge, designed to be the final closed-ended academic benchmark of its kind with broad subject coverage. HLE consists of 2,500 questions across dozens of subjects, including mathematics, humanities, and the natural sciences. HLE is developed globally by subject-matter experts and consists of multiple-choice and short-answer questions suitable for automated grading. Each question has a known solution that is unambiguous and easily verifiable, but cannot be quickly answered via internet retrieval. State-of-the-art LLMs demonstrate low accuracy and calibration on HLE, highlighting a significant gap between current LLM capabilities and the expert human frontier on closed-ended academic questions. To inform research and policymaking upon a clear understanding of model capabilities, we publicly release HLE at https://lastexam.ai. △ Less

Submitted 19 April, 2025; v1 submitted 24 January, 2025; originally announced January 2025.

Comments: 29 pages, 6 figures

Secure communication using low dimensional topological elements

Authors: Manuel F. Ferrer-Garcia, Avishy Carmi, Alessio D'Errico, Hugo Larocque, Eliahu Cohen, Ebrahim Karimi

Abstract: Low-dimensional topological objects, such as knots and braids, have become prevalent in multiple areas of physics, such as fluid dynamics, optics, and quantum information processing. Such objects also now play a role in cryptography, where a framed knot can store encoded information using its braid representation for communications purposes. The greater resilience of low-dimensional topological el… ▽ More Low-dimensional topological objects, such as knots and braids, have become prevalent in multiple areas of physics, such as fluid dynamics, optics, and quantum information processing. Such objects also now play a role in cryptography, where a framed knot can store encoded information using its braid representation for communications purposes. The greater resilience of low-dimensional topological elements under deformations allows them to be employed as a reliable framework for information exchange. Here, we introduce a challenge-response protocol as an application of this construction for authentication. We provide illustrative examples of both procedures showing how framed links and braids may help to enhance secure communication. △ Less

Submitted 7 December, 2022; originally announced December 2022.

Comments: 5 pages, 4 figures

Computing by nowhere increasing complexity

Authors: Bar Y. Peled, Vikas K. Mishra, Avishy Y. Carmi

Abstract: A cellular automaton is presented whose governing rule is that the Kolmogorov complexity of a cell's neighborhood may not increase when the cell's present value is substituted for its future value. Using an approximation of this two-dimensional Kolmogorov complexity the underlying automaton is shown to be capable of simulating logic circuits. It is also shown to capture trianry logic described by… ▽ More A cellular automaton is presented whose governing rule is that the Kolmogorov complexity of a cell's neighborhood may not increase when the cell's present value is substituted for its future value. Using an approximation of this two-dimensional Kolmogorov complexity the underlying automaton is shown to be capable of simulating logic circuits. It is also shown to capture trianry logic described by a quandle, a non-associative algebraic structure. A similar automaton whose rule permits at times the increase of a cell's neighborhood complexity is shown to produce animated entities which can be used as information carriers akin to gliders in Conway's game of life. △ Less

Submitted 4 October, 2017; originally announced October 2017.

On Approximating Univariate NP-Hard Integrals

Authors: Ohad Asor, Avishy Carmi

Abstract: Approximating a definite integral of product of cosines to within an accuracy of n binary digits where the integrand depends on input integers x[k] given in binary radix, is equivalent to counting the number of equal-sum partitions of the integers and is thus a #P problem. Similarly, integrating this function from zero to infinity and deciding whether the result is either zero or infinity is an NP… ▽ More Approximating a definite integral of product of cosines to within an accuracy of n binary digits where the integrand depends on input integers x[k] given in binary radix, is equivalent to counting the number of equal-sum partitions of the integers and is thus a #P problem. Similarly, integrating this function from zero to infinity and deciding whether the result is either zero or infinity is an NP-Complete problem. Efficient numerical integration methods such as the double exponential formula and the sinc approximation have been around since the mid 70's. Noting the hardness of approximating the integral we argue that the proven rates of convergence of such methods cannot possibly be correct since they give rise to an anomalous result as P=#P. △ Less

Submitted 5 January, 2016; v1 submitted 26 December, 2015; originally announced December 2015.

Comments: Need to show more evidence to the claims

Tales told by coloured tangles

Authors: Daniel Moskovich, Avishy Y. Carmi

Abstract: Tangle machines are a topologically inspired diagrammatic formalism to describe information flow in networks. This paper begins with an expository account of tangle machines motivated by the problem of describing `covariance intersection' fusion of Gaussian estimators in networks. It then gives two examples in which tangle machines tell stories of adiabatic quantum computations, and discusses lear… ▽ More Tangle machines are a topologically inspired diagrammatic formalism to describe information flow in networks. This paper begins with an expository account of tangle machines motivated by the problem of describing `covariance intersection' fusion of Gaussian estimators in networks. It then gives two examples in which tangle machines tell stories of adiabatic quantum computations, and discusses learning tangle machines from data. △ Less

Submitted 20 December, 2015; v1 submitted 16 November, 2015; originally announced November 2015.

Comments: 29 pages, 28 figures. Revised to be more self-contained

MSC Class: 94A15; 81P68; 57M99 ACM Class: H.1.1; F.0

Journal ref: Int. J. Unconv. Comput. 12(1) 71-105 (2016)

Low-Dimensional Topology of Information Fusion

Authors: Avishy Y. Carmi, Daniel Moskovich

Abstract: We provide an axiomatic characterization of information fusion, on the basis of which we define an information fusion network. Our construction is reminiscent of tangle diagrams in low dimensional topology. Information fusion networks come equipped with a natural notion of equivalence. Equivalent networks `contain the same information', but differ locally. When fusing streams of information, an in… ▽ More We provide an axiomatic characterization of information fusion, on the basis of which we define an information fusion network. Our construction is reminiscent of tangle diagrams in low dimensional topology. Information fusion networks come equipped with a natural notion of equivalence. Equivalent networks `contain the same information', but differ locally. When fusing streams of information, an information fusion network may adaptively optimize itself inside its equivalence class. This provides a fault tolerance mechanism for such networks. △ Less

Submitted 18 September, 2014; originally announced September 2014.

Comments: 8 pages. Conference proceedings version. Will be superceded by a journal version

MSC Class: 94A15; 57M25 ACM Class: H.1.1

Journal ref: Proceedings of BICT '14, 8th International Conference on Bio-inspired Information and Communications Technologies, Boston, MA, USA, Dec. 01 - 03, 2014, pp. 251-258

Computing with Coloured Tangles

Authors: Avishy Y. Carmi, Daniel Moskovich

Abstract: We suggest a diagrammatic model of computation based on an axiom of distributivity. A diagram of a decorated coloured tangle, similar to those that appear in low dimensional topology, plays the role of a circuit diagram. Equivalent diagrams represent bisimilar computations. We prove that our model of computation is Turing complete, and that with bounded resources it can moreover decide any languag… ▽ More We suggest a diagrammatic model of computation based on an axiom of distributivity. A diagram of a decorated coloured tangle, similar to those that appear in low dimensional topology, plays the role of a circuit diagram. Equivalent diagrams represent bisimilar computations. We prove that our model of computation is Turing complete, and that with bounded resources it can moreover decide any language in complexity class IP, sometimes with better performance parameters than corresponding classical protocols. △ Less

Submitted 21 July, 2015; v1 submitted 12 August, 2014; originally announced August 2014.

Comments: 36 pages,; Introduction entirely rewritten, Section 4.3 added

MSC Class: 68Q15; 57M99 ACM Class: F.2.2

Journal ref: Symmetry 2015, 7(3), 1289-1332

Tangle Machines II: Invariants

Authors: Avishy Y. Carmi, Daniel Moskovich

Abstract: The preceding paper constructed tangle machines as diagrammatic models, and illustrated their utility with a number of examples. The information content of a tangle machine is contained in characteristic quantities associated to equivalence classes of tangle machines, which are called invariants. This paper constructs invariants of tangle machines. Chief among these are the prime factorizations of… ▽ More The preceding paper constructed tangle machines as diagrammatic models, and illustrated their utility with a number of examples. The information content of a tangle machine is contained in characteristic quantities associated to equivalence classes of tangle machines, which are called invariants. This paper constructs invariants of tangle machines. Chief among these are the prime factorizations of a machine, which are essentially unique. This is proven using low dimensional topology, through representing a colour-suppressed machine as a diagram for a network of jointly embedded spheres and intervals in 4-space. The complexity of a tangle machine is defined as its number of prime factors. △ Less

Submitted 10 April, 2014; originally announced April 2014.

Comments: 26 pages, 30 figures

MSC Class: 94A15; 81P68; 57M99 ACM Class: H.1.1; F.0

Tangle Machines

Authors: Avishy Y. Carmi, Daniel Moskovich

Abstract: Tangle machines are topologically inspired diagrammatic models. Their novel feature is their natural notion of equivalence. Equivalent tangle machines may differ locally, but globally they are considered to share the same information content. The goal of tangle machine equivalence is to provide a context-independent method to select, from among many ways to perform a task, the `best' way to perfor… ▽ More Tangle machines are topologically inspired diagrammatic models. Their novel feature is their natural notion of equivalence. Equivalent tangle machines may differ locally, but globally they are considered to share the same information content. The goal of tangle machine equivalence is to provide a context-independent method to select, from among many ways to perform a task, the `best' way to perform the task. The concept of equivalent tangle machines is illustrated through examples in which they represent recursive computations, networks of adiabatic quantum computations, and networks of distributed information processing. △ Less

Submitted 3 March, 2015; v1 submitted 10 April, 2014; originally announced April 2014.

Comments: 29 pages, 37 figures. Major revision. Introduction rewritten. Definitions simplified, with the Gauss diagram definition pushed to an appendix. Previous appendix removed

MSC Class: 94A15; 81P68; 57M99 ACM Class: H.1.1; F.0

Journal ref: Proc. R. Soc. A 2015 471 20150111

Convex Feasibility Methods for Compressed Sensing

Authors: Avishy Carmi, Pini Gurfil

Abstract: We present a computationally-efficient method for recovering sparse signals from a series of noisy observations, known as the problem of compressed sensing (CS). CS theory requires solving a convex constrained minimization problem. We propose to transform this optimization problem into a convex feasibility problem (CFP), and solve it using subgradient projection methods, which are iterative, fast,… ▽ More We present a computationally-efficient method for recovering sparse signals from a series of noisy observations, known as the problem of compressed sensing (CS). CS theory requires solving a convex constrained minimization problem. We propose to transform this optimization problem into a convex feasibility problem (CFP), and solve it using subgradient projection methods, which are iterative, fast, robust and convergent schemes for solving CFPs. As opposed to some of the recently-introduced CS algorithms, such as Bayesian CS and gradient projections for sparse reconstruction, which become inefficient as the problem dimension and sparseness degree increase, the newly-proposed methods exhibit a marked robustness with respect to these factors. This renders the subgradient projection methods highly viable for large-scale compressible scenarios. △ Less

Submitted 19 June, 2010; v1 submitted 25 January, 2010; originally announced January 2010.