Showing 1–43 of 43 results for author: Wolff, J

Energy-Efficient QoS-Aware Scheduling for S-NUCA Many-Cores

Authors: Sudam M. Wasala, Jurre Wolff, Yixian Shen, Anuj Pathania, Clemens Grelck, Andy D. Pimentel

Abstract: Optimizing performance and energy efficiency in many-core processors, especially within Non-Uniform Cache Access (NUCA) architectures, remains a critical challenge. The performance heterogeneity inherent in S-NUCA systems complicates task scheduling due to varying cache access latencies across cores. This paper introduces a novel QoS management policy to maintain application execution within prede… ▽ More Optimizing performance and energy efficiency in many-core processors, especially within Non-Uniform Cache Access (NUCA) architectures, remains a critical challenge. The performance heterogeneity inherent in S-NUCA systems complicates task scheduling due to varying cache access latencies across cores. This paper introduces a novel QoS management policy to maintain application execution within predefined Quality of Service (QoS) targets, measured using the Application Heartbeats framework. QoS metrics like Heartbeats ensure predictable application performance in dynamic computing environments. The proposed policy dynamically controls QoS by orchestrating task migrations within the S-NUCA many-core system and adjusting the clock frequency of cores. After satisfying the QoS objectives, the policy optimizes energy efficiency, reducing overall system energy consumption without compromising performance constraints. Our work leverages the state-of-the-art multi-/many-core simulator {\em HotSniper}. We have extended it with two key components: an integrated heartbeat framework for precise, application-specific performance monitoring, and our QoS management policy that maintains application QoS requirements while minimizing the system's energy consumption. Experimental evaluations demonstrate that our approach effectively maintains desired QoS levels and achieves 18.7\% energy savings compared to state-of-the-art scheduling methods. △ Less

Submitted 29 May, 2025; originally announced May 2025.

A Fully Generative Motivational Interviewing Counsellor Chatbot for Moving Smokers Towards the Decision to Quit

Authors: Zafarullah Mahmood, Soliman Ali, Jiading Zhu, Mohamed Abdelwahab, Michelle Yu Collins, Sihan Chen, Yi Cheng Zhao, Jodi Wolff, Osnat Melamed, Nadia Minian, Marta Maslej, Carolynne Cooper, Matt Ratto, Peter Selby, Jonathan Rose

Abstract: The conversational capabilities of Large Language Models (LLMs) suggest that they may be able to perform as automated talk therapists. It is crucial to know if these systems would be effective and adhere to known standards. We present a counsellor chatbot that focuses on motivating tobacco smokers to quit smoking. It uses a state-of-the-art LLM and a widely applied therapeutic approach called Moti… ▽ More The conversational capabilities of Large Language Models (LLMs) suggest that they may be able to perform as automated talk therapists. It is crucial to know if these systems would be effective and adhere to known standards. We present a counsellor chatbot that focuses on motivating tobacco smokers to quit smoking. It uses a state-of-the-art LLM and a widely applied therapeutic approach called Motivational Interviewing (MI), and was evolved in collaboration with clinician-scientists with expertise in MI. We also describe and validate an automated assessment of both the chatbot's adherence to MI and client responses. The chatbot was tested on 106 participants, and their confidence that they could succeed in quitting smoking was measured before the conversation and one week later. Participants' confidence increased by an average of 1.7 on a 0-10 scale. The automated assessment of the chatbot showed adherence to MI standards in 98% of utterances, higher than human counsellors. The chatbot scored well on a participant-reported metric of perceived empathy but lower than typical human counsellors. Furthermore, participants' language indicated a good level of motivation to change, a key goal in MI. These results suggest that the automation of talk therapy with a modern LLM has promise. △ Less

Submitted 1 June, 2025; v1 submitted 22 May, 2025; originally announced May 2025.

Comments: To be published in the Findings of the 63rd Annual Meeting of the Association for Computational Linguistics (ACL), Vienna, Austria, 2025

Mixture-of-experts VAEs can disregard variation in surjective multimodal data

Authors: Jannik Wolff, Tassilo Klein, Moin Nabi, Rahul G. Krishnan, Shinichi Nakajima

Abstract: Machine learning systems are often deployed in domains that entail data from multiple modalities, for example, phenotypic and genotypic characteristics describe patients in healthcare. Previous works have developed multimodal variational autoencoders (VAEs) that generate several modalities. We consider subjective data, where single datapoints from one modality (such as class labels) describe multi… ▽ More Machine learning systems are often deployed in domains that entail data from multiple modalities, for example, phenotypic and genotypic characteristics describe patients in healthcare. Previous works have developed multimodal variational autoencoders (VAEs) that generate several modalities. We consider subjective data, where single datapoints from one modality (such as class labels) describe multiple datapoints from another modality (such as images). We theoretically and empirically demonstrate that multimodal VAEs with a mixture of experts posterior can struggle to capture variability in such surjective data. △ Less

Submitted 11 April, 2022; originally announced April 2022.

Comments: Accepted at the NeurIPS 2021 workshop on Bayesian Deep Learning

Piercing the Veil: Designs to Support Information Literacy on Social Platforms

Authors: Jan Wolff

Abstract: In this position paper we approach problems concerning critical digital and information literacy with ideas to provide more digestible explanations of abstract concepts through interface design. In particular, we focus on social media platforms where we see the possibility of counteracting the spread of misinformation by providing users with more proficiency through our approaches. We argue that t… ▽ More In this position paper we approach problems concerning critical digital and information literacy with ideas to provide more digestible explanations of abstract concepts through interface design. In particular, we focus on social media platforms where we see the possibility of counteracting the spread of misinformation by providing users with more proficiency through our approaches. We argue that the omnipresent trend to abstract away and hide information from users via UI/UX design opposes their ability to self-learn. This leads us to propose a different framework in which we unify elegant and simple interfaces with nudges that promote a look behind the curtain. Such designs serve to foster a deeper understanding of employed technologies and aim to increase the critical assessment of content encountered on social platforms. Furthermore, we consider users with an intermediary skill level to be largely ignored in current approaches, as they are given no tools to broaden their knowledge without consultation of expert material. The resulting stagnation is exemplified by the tactics of misinformation campaigns, which exploit the ensuing lack of information literacy and critical thinking. We propose an approach to design that sufficiently emancipates users in both aspects by promoting a look behind the abstraction of UI/UX so that an autonomous learning process is given the chance to occur. Furthermore, we name ideas for future research within this area that take our considerations into account. △ Less

Submitted 3 June, 2021; originally announced June 2021.

Comments: Originally submitted to and presented at CHI'21 Workshop on Technologies to Support Critical Thinking in an Age of Misinformation

Learning Graph-Based Priors for Generalized Zero-Shot Learning

Authors: Colin Samplawski, Jannik Wolff, Tassilo Klein, Moin Nabi

Abstract: The task of zero-shot learning (ZSL) requires correctly predicting the label of samples from classes which were unseen at training time. This is achieved by leveraging side information about class labels, such as label attributes or word embeddings. Recently, attention has shifted to the more realistic task of generalized ZSL (GZSL) where test sets consist of seen and unseen samples. Recent approa… ▽ More The task of zero-shot learning (ZSL) requires correctly predicting the label of samples from classes which were unseen at training time. This is achieved by leveraging side information about class labels, such as label attributes or word embeddings. Recently, attention has shifted to the more realistic task of generalized ZSL (GZSL) where test sets consist of seen and unseen samples. Recent approaches to GZSL have shown the value of generative models, which are used to generate samples from unseen classes. In this work, we incorporate an additional source of side information in the form of a relation graph over labels. We leverage this graph in order to learn a set of prior distributions, which encourage an aligned variational autoencoder (VAE) model to learn embeddings which respect the graph structure. Using this approach we are able to achieve improved performance on the CUB and SUN benchmarks over a strong baseline. △ Less

Submitted 21 October, 2020; originally announced October 2020.

Comments: Presented at AAAI 2020 Workshop on Deep Learning on Graphs: Methodologies and Applications (DLGMA'20)

Problems in AI research and how the SP System may help to solve them

Authors: J Gerard Wolff

Abstract: This paper describes problems in AI research and how the SP System (described in an appendix) may help to solve them. Most of the problems are described by leading researchers in AI in interviews with science writer Martin Ford, and reported by him in his book {\em Architects of Intelligence}. These problems are: the need to bridge the divide between symbolic and non-symbolic kinds of knowledge an… ▽ More This paper describes problems in AI research and how the SP System (described in an appendix) may help to solve them. Most of the problems are described by leading researchers in AI in interviews with science writer Martin Ford, and reported by him in his book {\em Architects of Intelligence}. These problems are: the need to bridge the divide between symbolic and non-symbolic kinds of knowledge and processing; the tendency of deep neural networks (DNNs) to make large and unexpected errors in recognition; the need to strengthen the representation and processing of natural languages; the challenges of unsupervised learning; the need for a coherent account of generalisation; how to learn usable knowledge from a single exposure; how to achieve transfer learning; how to increase the efficiency of AI processing; the need for transparency in AI structures and processes; how to achieve varieties of probabilistic reasoning; the need for more emphasis on top-down strategies; how to minimise the risk of accidents with self-driving vehicles; the need for strong compositionality in AI knowledge; the challenges of commonsense reasoning and commonsense knowledge; establishing the importance of information compression in AI research; establishing the importance of a biological perspective in AI research; establishing whether knowledge in the brain is represented in `distributed' or `localist' form; how to bypassing the limited scope for adaptation in deep neural networks; the need to develop `broad AI'; and how to eliminate the problem of catastrophic forgetting. △ Less

Submitted 28 February, 2021; v1 submitted 2 September, 2020; originally announced September 2020.

Transparency and granularity in the SP Theory of Intelligence and its realisation in the SP Computer Model

Authors: J Gerard Wolff

Abstract: This chapter describes how the SP System, meaning the SP Theory of Intelligence, and its realisation as the SP Computer Model, may promote transparency and granularity in AI, and some other areas of application. The chapter describes how transparency in the workings and output of the SP Computer Model may be achieved via three routes: 1) the program provides a very full audit trail for such proces… ▽ More This chapter describes how the SP System, meaning the SP Theory of Intelligence, and its realisation as the SP Computer Model, may promote transparency and granularity in AI, and some other areas of application. The chapter describes how transparency in the workings and output of the SP Computer Model may be achieved via three routes: 1) the program provides a very full audit trail for such processes as recognition, reasoning, analysis of language, and so on. There is also an explicit audit trail for the unsupervised learning of new knowledge; 2) knowledge from the system is likely to be granular and easy for people to understand; and 3) there are seven principles for the organisation of knowledge which are central in the workings of the SP System and also very familiar to people (eg chunking-with-codes, part-whole hierarchies, and class-inclusion hierarchies), and that kind of familiarity in the way knowledge is structured by the system, is likely to be important in the interpretability, explainability, and transparency of that knowledge. Examples from the SP Computer Model are shown throughout the chapter. △ Less

Submitted 9 May, 2021; v1 submitted 7 September, 2020; originally announced September 2020.

Comments: Published in the book {\em Interpretable Artificial Intelligence: A Perspective of Granular Computing}, Witold Pedrycz and Shyi-Ming Chen (editors), Springer: Heidelberg, 2021, ISBN 978-3-030-64948-7, DOI: 10.1007/978-3-030-64949-4

Low-Shot Learning from Imaginary 3D Model

Authors: Frederik Pahde, Mihai Puscas, Jannik Wolff, Tassilo Klein, Nicu Sebe, Moin Nabi

Abstract: Since the advent of deep learning, neural networks have demonstrated remarkable results in many visual recognition tasks, constantly pushing the limits. However, the state-of-the-art approaches are largely unsuitable in scarce data regimes. To address this shortcoming, this paper proposes employing a 3D model, which is derived from training images. Such a model can then be used to hallucinate nove… ▽ More Since the advent of deep learning, neural networks have demonstrated remarkable results in many visual recognition tasks, constantly pushing the limits. However, the state-of-the-art approaches are largely unsuitable in scarce data regimes. To address this shortcoming, this paper proposes employing a 3D model, which is derived from training images. Such a model can then be used to hallucinate novel viewpoints and poses for the scarce samples of the few-shot learning scenario. A self-paced learning approach allows for the selection of a diverse set of high-quality images, which facilitates the training of a classifier. The performance of the proposed approach is showcased on the fine-grained CUB-200-2011 dataset in a few-shot setting and significantly improves our baseline accuracy. △ Less

Submitted 4 January, 2019; originally announced January 2019.

Comments: To appear at WACV 2019. arXiv admin note: text overlap with arXiv:1811.09192

Interpreting Winograd Schemas Via the SP Theory of Intelligence and Its Realisation in the SP Computer Model

Authors: J Gerard Wolff

Abstract: In 'Winograd Schema' (WS) sentences like "The city councilmen refused the demonstrators a permit because they feared violence" and "The city councilmen refused the demonstrators a permit because they advocated revolution", it is easy for adults to understand what "they" refers to but can be difficult for AI systems. This paper describes how the SP System -- outlined in an appendix -- may solve thi… ▽ More In 'Winograd Schema' (WS) sentences like "The city councilmen refused the demonstrators a permit because they feared violence" and "The city councilmen refused the demonstrators a permit because they advocated revolution", it is easy for adults to understand what "they" refers to but can be difficult for AI systems. This paper describes how the SP System -- outlined in an appendix -- may solve this kind of problem of interpretation. The central idea is that a knowledge of discontinuous associations amongst linguistic features, and an ability to recognise such patterns of associations, provides a robust means of determining what a pronoun like "they" refers to. For any AI system to solve this kind of problem, it needs appropriate knowledge of relevant syntax and semantics which, ideally, it should learn for itself. Although the SP System has some strengths in unsupervised learning, its capabilities in this area are not yet good enough to learn the kind of knowledge needed to interpret WS examples, so it must be supplied with such knowledge at the outset. However, its existing strengths in unsupervised learning suggest that it has potential to learn the kind of knowledge needed for the interpretation of WS examples. In particular, it has potential to learn the kind of discontinuous association of linguistic features mentioned earlier. △ Less

Submitted 9 October, 2018; originally announced October 2018.

Mathematics as information compression via the matching and unification of patterns

Authors: J Gerard Wolff

Abstract: This paper describes a novel perspective on the foundations of mathematics: how mathematics may be seen to be largely about 'information compression via the matching and unification of patterns' (ICMUP). ICMUP is itself a novel approach to information compression, couched in terms of non-mathematical primitives, as is necessary in any investigation of the foundations of mathematics. This new persp… ▽ More This paper describes a novel perspective on the foundations of mathematics: how mathematics may be seen to be largely about 'information compression via the matching and unification of patterns' (ICMUP). ICMUP is itself a novel approach to information compression, couched in terms of non-mathematical primitives, as is necessary in any investigation of the foundations of mathematics. This new perspective on the foundations of mathematics has grown out of an extensive programme of research developing the "SP Theory of Intelligence" and its realisation in the "SP Computer Model", a system in which a generalised version of ICMUP -- the powerful concept of SP-multiple-alignment -- plays a central role. These ideas may be seen to be part of a "Big Picture" comprising six areas of interest, with information compression as a unifying theme. The paper describes the close relation between mathematics and information compression, and describes examples showing how variants of ICMUP may be seen in widely-used structures and operations in mathematics. Examples are also given to show how the mathematics-related disciplines of logic and computing may be understood as ICMUP. There are many potential benefits and applications of these ideas. △ Less

Submitted 9 October, 2018; v1 submitted 5 August, 2018; originally announced August 2018.

Introduction to the SP theory of intelligence

Authors: J Gerard Wolff

Abstract: This article provides a brief introduction to the "Theory of Intelligence" and its realisation in the "SP Computer Model". The overall goal of the SP programme of research, in accordance with long-established principles in science, has been the simplification and integration of observations and concepts across artificial intelligence, mainstream computing, mathematics, and human learning, percepti… ▽ More This article provides a brief introduction to the "Theory of Intelligence" and its realisation in the "SP Computer Model". The overall goal of the SP programme of research, in accordance with long-established principles in science, has been the simplification and integration of observations and concepts across artificial intelligence, mainstream computing, mathematics, and human learning, perception, and cognition. In broad terms, the SP system is a brain-like system that takes in "New" information through its senses and stores some or all of it as "Old" information. A central idea in the system is the powerful concept of "SP-multiple-alignment", borrowed and adapted from bioinformatics. This the key to the system's versatility in aspects of intelligence, in the representation of diverse kinds of knowledge, and in the seamless integration of diverse aspects of intelligence and diverse kinds of knowledge, in any combination. There are many potential benefits and applications of the SP system. It is envisaged that the system will be developed as the "SP Machine", which will initially be a software virtual machine, hosted on a high-performance computer, a vehicle for further research and a step towards the development of an industrial-strength SP Machine. △ Less

Submitted 24 February, 2018; originally announced February 2018.

Solutions to problems with deep learning

Authors: J Gerard Wolff

Abstract: Despite the several successes of deep learning systems, there are concerns about their limitations, discussed most recently by Gary Marcus. This paper discusses Marcus's concerns and some others, together with solutions to several of these problems provided by the "P theory of intelligence" and its realisation in the "SP computer model". The main advantages of the SP system are: relatively small r… ▽ More Despite the several successes of deep learning systems, there are concerns about their limitations, discussed most recently by Gary Marcus. This paper discusses Marcus's concerns and some others, together with solutions to several of these problems provided by the "P theory of intelligence" and its realisation in the "SP computer model". The main advantages of the SP system are: relatively small requirements for data and the ability to learn from a single experience; the ability to model both hierarchical and non-hierarchical structures; strengths in several kinds of reasoning, including `commonsense' reasoning; transparency in the representation of knowledge, and the provision of an audit trail for all processing; the likelihood that the SP system could not be fooled into bizarre or eccentric recognition of stimuli, as deep learning systems can be; the SP system provides a robust solution to the problem of `catastrophic forgetting' in deep learning systems; the SP system provides a theoretically-coherent solution to the problems of correcting over- and under-generalisations in learning, and learning correct structures despite errors in data; unlike most research on deep learning, the SP programme of research draws extensively on research on human learning, perception, and cognition; and the SP programme of research has an overarching theory, supported by evidence, something that is largely missing from research on deep learning. In general, the SP system provides a much firmer foundation than deep learning for the development of artificial general intelligence. △ Less

Submitted 8 January, 2018; originally announced January 2018.

Software engineering and the SP Theory of Intelligence

Authors: J Gerard Wolff

Abstract: This paper describes a novel approach to software engineering derived from the "SP Theory of Intelligence" and its realisation in the "SP Computer Model". Despite superficial appearances, it is shown that many of the key ideas in software engineering have counterparts in the structure and workings of the SP system. Potential benefits of this new approach to software engineering include: the automa… ▽ More This paper describes a novel approach to software engineering derived from the "SP Theory of Intelligence" and its realisation in the "SP Computer Model". Despite superficial appearances, it is shown that many of the key ideas in software engineering have counterparts in the structure and workings of the SP system. Potential benefits of this new approach to software engineering include: the automation or semi-automation of software development, with support for programming of the SP system where necessary; allowing programmers to concentrate on 'world-oriented' parallelism, without worries about parallelism to speed up processing; support for the long-term goal of programming the SP system via written or spoken natural language; reducing or eliminating the distinction between 'design' and 'implementation'; reducing or eliminating operations like compiling or interpretation; reducing or eliminating the need for verification of software; reducing the need for validation of software; no formal distinction between program and database; the potential for substantial reductions in the number of types of data file and the number of computer languages; benefits for version control; and reducing technical debt. △ Less

Submitted 5 August, 2018; v1 submitted 18 August, 2017; originally announced August 2017.

A Roadmap for the Development of the "SP Machine" for Artificial Intelligence

Authors: J Gerard Wolff

Abstract: This paper describes a roadmap for the development of the "SP Machine", based on the "SP Theory of Intelligence" and its realisation in the "SP Computer Model". The SP Machine will be developed initially as a software virtual machine with high levels of parallel processing, hosted on a high-performance computer. The system should help users visualise knowledge structures and processing. Research i… ▽ More This paper describes a roadmap for the development of the "SP Machine", based on the "SP Theory of Intelligence" and its realisation in the "SP Computer Model". The SP Machine will be developed initially as a software virtual machine with high levels of parallel processing, hosted on a high-performance computer. The system should help users visualise knowledge structures and processing. Research is needed into how the system may discover low-level features in speech and in images. Strengths of the SP System in the processing of natural language may be augmented, in conjunction with the further development of the SP System's strengths in unsupervised learning. Strengths of the SP System in pattern recognition may be developed for computer vision. Work is needed on the representation of numbers and the performance of arithmetic processes. A computer model is needed of "SP-Neural", the version of the SP Theory expressed in terms of neurons and their inter-connections. The SP Machine has potential in many areas of application, several of which may be realised on short-to-medium timescales. △ Less

Submitted 17 December, 2018; v1 submitted 28 June, 2017; originally announced July 2017.

Comments: Accepted for publication in the journal "Complexity"

The SP Theory of Intelligence as a Foundation for the Development of a General, Human-Level Thinking Machine

Authors: J Gerard Wolff

Abstract: This paper summarises how the "SP theory of intelligence" and its realisation in the "SP computer model" simplifies and integrates concepts across artificial intelligence and related areas, and thus provides a promising foundation for the development of a general, human-level thinking machine, in accordance with the main goal of research in artificial general intelligence. The key to this simpli… ▽ More This paper summarises how the "SP theory of intelligence" and its realisation in the "SP computer model" simplifies and integrates concepts across artificial intelligence and related areas, and thus provides a promising foundation for the development of a general, human-level thinking machine, in accordance with the main goal of research in artificial general intelligence. The key to this simplification and integration is the powerful concept of "multiple alignment", borrowed and adapted from bioinformatics. This concept has the potential to be the "double helix" of intelligence, with as much significance for human-level intelligence as has DNA for biological sciences. Strengths of the SP system include: versatility in the representation of diverse kinds of knowledge; versatility in aspects of intelligence (including: strengths in unsupervised learning; the processing of natural language; pattern recognition at multiple levels of abstraction that is robust in the face of errors in data; several kinds of reasoning (including: one-step `deductive' reasoning; chains of reasoning; abductive reasoning; reasoning with probabilistic networks and trees; reasoning with 'rules'; nonmonotonic reasoning and reasoning with default values; Bayesian reasoning with 'explaining away'; and more); planning; problem solving; and more); seamless integration of diverse kinds of knowledge and diverse aspects of intelligence in any combination; and potential for application in several areas (including: helping to solve nine problems with big data; helping to develop human-level intelligence in autonomous robots; serving as a database with intelligence and with versatility in the representation and integration of several forms of knowledge; serving as a vehicle for medical knowledge and as an aid to medical diagnosis; and several more). △ Less

Submitted 22 December, 2016; originally announced December 2016.

In the transmission of information, the great potential of model-based coding with the SP theory of intelligence

Authors: J Gerard Wolff

Abstract: Model-based coding, described by John Pierce in 1961, has great potential to reduce the volume of information that needs to be transmitted in moving big data, without loss of information, from one place to another, or in lossless communications via the internet. Compared with ordinary compression methods, this potential advantage of model-based coding in the transmission of data arises from the fa… ▽ More Model-based coding, described by John Pierce in 1961, has great potential to reduce the volume of information that needs to be transmitted in moving big data, without loss of information, from one place to another, or in lossless communications via the internet. Compared with ordinary compression methods, this potential advantage of model-based coding in the transmission of data arises from the fact that both the transmitter ("Alice") and the receiver ("Bob") are equipped with a grammar for the kind of data that is to be transmitted, which means that, to achieve lossless transmission of a body of data from Alice and Bob, a relatively small amount of information needs to be sent. Preliminary trials indicate that, with model-based coding, the volume of information to be sent from Alice to Bob to achieve lossless transmission of a given body of data may be less than $6\%$ of the volume of information that needs to be sent when ordinary compression methods are used. Until recently, it has not been feasible to convert John Pierce's vision into something that may be applied in practice. Now, with the development of the "SP theory of intelligence" and its realisation in the "SP computer model", there is clear potential to realise the three main functions that will be needed: unsupervised learning of a grammar for the kind of data that is to be transmitted using a relatively powerful computer that is independent of Alice and Bob; the encoding by Alice of any one example of such data in terms of the grammar; and, with the grammar, decoding of the encoding by Bob to retrieve the given example. It appears now to be feasible, within reasonable timescales, to bring these capabilities to a level where they may be applied to the transmission of realistically large bodies of data. △ Less

Submitted 6 December, 2016; originally announced December 2016.

The curse of variety in computing, and what can be done about it

Authors: J Gerard Wolff

Abstract: Excess freedom in how computers are used creates problems that include: bit rot, problems with big data, problems in the creation and debugging of software, and problems with cyber security. To tame excess freedom, "tough love" is needed in the form of a {\em universal framework for the representation and processing of diverse kinds of knowledge} (UFK). The "SP machine", based on the "SP theory of… ▽ More Excess freedom in how computers are used creates problems that include: bit rot, problems with big data, problems in the creation and debugging of software, and problems with cyber security. To tame excess freedom, "tough love" is needed in the form of a {\em universal framework for the representation and processing of diverse kinds of knowledge} (UFK). The "SP machine", based on the "SP theory of intelligence", has the potential to provide that framework and to help solve the problems above. There is potential to reduce the near-4000 different kinds of computer file to one, and to reduce the hundreds of different computer languages to one. △ Less

Submitted 25 September, 2016; originally announced September 2016.

Commonsense Reasoning, Commonsense Knowledge, and The SP Theory of Intelligence

Authors: J Gerard Wolff

Abstract: This paper describes how the "SP Theory of Intelligence" with the "SP Computer Model", outlined in an Appendix, may throw light on aspects of commonsense reasoning (CSR) and commonsense knowledge (CSK), as discussed in another paper by Ernest Davis and Gary Marcus (DM). In four main sections, the paper describes: 1) The main problems to be solved; 2) Other research on CSR and CSK; 3) Why the SP sy… ▽ More This paper describes how the "SP Theory of Intelligence" with the "SP Computer Model", outlined in an Appendix, may throw light on aspects of commonsense reasoning (CSR) and commonsense knowledge (CSK), as discussed in another paper by Ernest Davis and Gary Marcus (DM). In four main sections, the paper describes: 1) The main problems to be solved; 2) Other research on CSR and CSK; 3) Why the SP system may prove useful with CSR and CSK 4) How examples described by DM may be modelled in the SP system. With regard to successes in the automation of CSR described by DM, the SP system's strengths in simplification and integration may promote seamless integration across these areas, and seamless integration of those area with other aspects of intelligence. In considering challenges in the automation of CSR described by DM, the paper describes in detail, with examples of SP-multiple-alignments. how the SP system may model processes of interpretation and reasoning arising from the horse's head scene in "The Godfather" film. A solution is presented to the 'long tail' problem described by DM. The SP system has some potentially useful things to say about several of DM's objectives for research in CSR and CSK. △ Less

Submitted 4 August, 2018; v1 submitted 25 September, 2016; originally announced September 2016.

The SP theory of intelligence and the representation and processing of knowledge in the brain

Authors: J Gerard Wolff

Abstract: The "SP theory of intelligence", with its realisation in the "SP computer model", aims to simplify and integrate observations and concepts across AI-related fields, with information compression as a unifying theme. This paper describes how abstract structures and processes in the theory may be realised in terms of neurons, their interconnections, and the transmission of signals between neurons. Th… ▽ More The "SP theory of intelligence", with its realisation in the "SP computer model", aims to simplify and integrate observations and concepts across AI-related fields, with information compression as a unifying theme. This paper describes how abstract structures and processes in the theory may be realised in terms of neurons, their interconnections, and the transmission of signals between neurons. This part of the SP theory -- "SP-neural" -- is a tentative and partial model for the representation and processing of knowledge in the brain. In the SP theory (apart from SP-neural), all kinds of knowledge are represented with "patterns", where a pattern is an array of atomic symbols in one or two dimensions. In SP-neural, the concept of a "pattern" is realised as an array of neurons called a "pattern assembly", similar to Hebb's concept of a "cell assembly" but with important differences. Central to the processing of information in the SP system is the powerful concept of "multiple alignment", borrowed and adapted from bioinformatics. Processes such as pattern recognition, reasoning and problem solving are achieved via the building of multiple alignments, while unsupervised learning -- significantly different from the "Hebbian" kinds of learning -- is achieved by creating patterns from sensory information and also by creating patterns from multiple alignments in which there is a partial match between one pattern and another. Short-lived neural structures equivalent to multiple alignments will be created via an inter-play of excitatory and inhibitory neural signals. The paper discusses several associated issues, with relevant empirical evidence. △ Less

Submitted 12 May, 2016; v1 submitted 19 April, 2016; originally announced April 2016.

Journal ref: Frontiers in Psychology, 7, 1584, 2016

Proposal for the creation of a research facility for the development of the SP machine

Authors: J. Gerard Wolff, Vasile Palade

Abstract: This is a proposal to create a research facility for the development of a high-parallel version of the "SP machine", based on the "SP theory of intelligence". We envisage that the new version of the SP machine will be an open-source software virtual machine, derived from the existing "SP computer model", and hosted on an existing high-performance computer. It will be a means for researchers everyw… ▽ More This is a proposal to create a research facility for the development of a high-parallel version of the "SP machine", based on the "SP theory of intelligence". We envisage that the new version of the SP machine will be an open-source software virtual machine, derived from the existing "SP computer model", and hosted on an existing high-performance computer. It will be a means for researchers everywhere to explore what can be done with the system and to create new versions of it. The SP system is a unique attempt to simplify and integrate observations and concepts across artificial intelligence, mainstream computing, mathematics, and human perception and cognition, with information compression as a unifying theme. Potential benefits and applications include helping to solve problems associated with big data; facilitating the development of autonomous robots; unsupervised learning, natural language processing, several kinds of reasoning, fuzzy pattern recognition at multiple levels of abstraction, computer vision, best-match and semantic forms of information retrieval, software engineering, medical diagnosis, simplification of computing systems, and the seamless integration of diverse kinds of knowledge and diverse aspects of intelligence. Additional motivations include the potential of the SP system to help solve problems in defence, security, and the detection and prevention of crime; potential in terms of economic, social, environmental, and academic criteria, and in terms of publicity; and the potential for international influence in research. The main elements of the proposed facility are described, including support for the development of "SP-neural", a neural version of the SP machine. The facility should be permanent in the sense that it should be available for the foreseeable future, and it should be designed to facilitate its use by researchers anywhere in the world. △ Less

Submitted 19 August, 2015; originally announced August 2015.

Comments: arXiv admin note: text overlap with arXiv:1508.04087. substantial text overlap with arXiv:1409.8027

The SP theory of intelligence: distinctive features and advantages

Authors: J. G. Wolff

Abstract: This paper highlights distinctive features of the "SP theory of intelligence" and its apparent advantages compared with some AI-related alternatives. Distinctive features and advantages are: simplification and integration of observations and concepts; simplification and integration of structures and processes in computing systems; the theory is itself a theory of computing; it can be the basis for… ▽ More This paper highlights distinctive features of the "SP theory of intelligence" and its apparent advantages compared with some AI-related alternatives. Distinctive features and advantages are: simplification and integration of observations and concepts; simplification and integration of structures and processes in computing systems; the theory is itself a theory of computing; it can be the basis for new architectures for computers; information compression via the matching and unification of patterns and, more specifically, via multiple alignment, is fundamental; transparency in the representation and processing of knowledge; the discovery of 'natural' structures via information compression (DONSVIC); interpretations of mathematics; interpretations in human perception and cognition; and realisation of abstract concepts in terms of neurons and their inter-connections ("SP-neural"). These things relate to AI-related alternatives: minimum length encoding and related concepts; deep learning in neural networks; unified theories of cognition and related research; universal search; Bayesian networks and more; pattern recognition and vision; the analysis, production, and translation of natural language; Unsupervised learning of natural language; exact and inexact forms of reasoning; representation and processing of diverse forms of knowledge; IBM's Watson; software engineering; solving problems associated with big data, and in the development of intelligence in autonomous robots. In conclusion, the SP system can provide a firm foundation for the long-term development of AI, with many potential benefits and applications. It may also deliver useful results on relatively short timescales. A high-parallel, open-source version of the SP machine, derived from the SP computer model, would be a means for researchers everywhere to explore what can be done with the system, and to create new versions of it. △ Less

Submitted 15 March, 2016; v1 submitted 17 August, 2015; originally announced August 2015.

Journal ref: IEEE Access, 4, 216-246, 2016

Medical diagnosis as pattern recognition in a framework of information compression by multiple alignment, unification and search

Authors: J. Gerard Wolff

Abstract: This paper describes a novel approach to medical diagnosis based on the SP theory of computing and cognition. The main attractions of this approach are: a format for representing diseases that is simple and intuitive; an ability to cope with errors and uncertainties in diagnostic information; the simplicity of storing statistical information as frequencies of occurrence of diseases; a method for e… ▽ More This paper describes a novel approach to medical diagnosis based on the SP theory of computing and cognition. The main attractions of this approach are: a format for representing diseases that is simple and intuitive; an ability to cope with errors and uncertainties in diagnostic information; the simplicity of storing statistical information as frequencies of occurrence of diseases; a method for evaluating alternative diagnostic hypotheses that yields true probabilities; and a framework that should facilitate unsupervised learning of medical knowledge and the integration of medical diagnosis with other AI applications. △ Less

Submitted 29 September, 2014; originally announced September 2014.

Journal ref: Decision Support Systems 42, 608-625, 2006

Autonomous robots and the SP theory of intelligence

Authors: J. Gerard Wolff

Abstract: This article is about how the "SP theory of intelligence" and its realisation in the "SP machine" (both outlined in the article) may help to solve computer-related problems in the design of autonomous robots, meaning robots that do not depend on external intelligence or power supplies, are mobile, and are designed to exhibit as much human-like intelligence as possible. The article is about: how to… ▽ More This article is about how the "SP theory of intelligence" and its realisation in the "SP machine" (both outlined in the article) may help to solve computer-related problems in the design of autonomous robots, meaning robots that do not depend on external intelligence or power supplies, are mobile, and are designed to exhibit as much human-like intelligence as possible. The article is about: how to increase the computational and energy efficiency of computers and reduce their bulk; how to achieve human-like versatility in intelligence; and likewise for human-like adaptability in intelligence. The SP system has potential for substantial gains in computational and energy efficiency and reductions in the bulkiness of computers: by reducing the size of data to be processed; by exploiting statistical information that the system gathers; and via an updated version of Donald Hebb's concept of a "cell assembly". Towards human-like versatility in intelligence, the SP system has strengths in unsupervised learning, natural language processing, pattern recognition, information retrieval, several kinds of reasoning, planning, problem solving, and more, with seamless integration amongst structures and functions. The SP system's strengths in unsupervised learning and other aspects of intelligence may help to achieve human-like adaptability in intelligence via: the learning of natural language; learning to see; building 3D models of objects and of a robot's surroundings; learning regularities in the workings of a robot and in the robot's environment; exploration and play; learning major skills; and secondary forms of learning. Also discussed are: how the SP system may process parallel streams of information; generalisation of knowledge, correction of over-generalisations, and learning from dirty data; how to cut the cost of learning; and reinforcements, motivations, goals, and demonstration. △ Less

Submitted 23 January, 2015; v1 submitted 29 September, 2014; originally announced September 2014.

Journal ref: IEEE Access, 2, 1629-1651, 2014

A proof challenge: multiple alignment and information compression

Authors: J Gerard Wolff

Abstract: These notes pose a "proof challenge": a proof, or disproof, of the proposition that "For any given body of information, I, expressed as a one-dimensional sequence of atomic symbols, a multiple alignment concept, described in the document, provides a means of encoding all the redundancy that may exist in I. Aspects of the challenge are described. These notes pose a "proof challenge": a proof, or disproof, of the proposition that "For any given body of information, I, expressed as a one-dimensional sequence of atomic symbols, a multiple alignment concept, described in the document, provides a means of encoding all the redundancy that may exist in I. Aspects of the challenge are described. △ Less

Submitted 4 March, 2014; originally announced March 2014.

Smart machines and the SP theory of intelligence

Authors: J. Gerard Wolff

Abstract: These notes describe how the "SP theory of intelligence", and its embodiment in the "SP machine", may help to realise cognitive computing, as described in the book "Smart Machines". In the SP system, information compression and a concept of "multiple alignment" are centre stage. The system is designed to integrate such things as unsupervised learning, pattern recognition, probabilistic reasoning,… ▽ More These notes describe how the "SP theory of intelligence", and its embodiment in the "SP machine", may help to realise cognitive computing, as described in the book "Smart Machines". In the SP system, information compression and a concept of "multiple alignment" are centre stage. The system is designed to integrate such things as unsupervised learning, pattern recognition, probabilistic reasoning, and more. It may help to overcome the problem of variety in big data, it may serve in pattern recognition and in the unsupervised learning of structure in data, and it may facilitate the management and transmission of big data. There is potential, via information compression, for substantial gains in computational efficiency, especially in the use of energy. The SP system may help to realise data-centric computing, perhaps via a development of Hebb's concept of a "cell assembly", or via the use of light or DNA for the processing of information. It has potential in the management of errors and uncertainty in data, in medical diagnosis, in processing streams of data, and in promoting adaptability in robots. △ Less

Submitted 8 January, 2014; originally announced January 2014.

Comments: arXiv admin note: substantial text overlap with arXiv:1306.3890

Information Compression, Intelligence, Computing, and Mathematics

Authors: J. Gerard Wolff

Abstract: This paper presents evidence for the idea that much of artificial intelligence, human perception and cognition, mainstream computing, and mathematics, may be understood as compression of information via the matching and unification of patterns. This is the basis for the "SP theory of intelligence", outlined in the paper and fully described elsewhere. Relevant evidence may be seen: in empirical sup… ▽ More This paper presents evidence for the idea that much of artificial intelligence, human perception and cognition, mainstream computing, and mathematics, may be understood as compression of information via the matching and unification of patterns. This is the basis for the "SP theory of intelligence", outlined in the paper and fully described elsewhere. Relevant evidence may be seen: in empirical support for the SP theory; in some advantages of information compression (IC) in terms of biology and engineering; in our use of shorthands and ordinary words in language; in how we merge successive views of any one thing; in visual recognition; in binocular vision; in visual adaptation; in how we learn lexical and grammatical structures in language; and in perceptual constancies. IC via the matching and unification of patterns may be seen in both computing and mathematics: in IC via equations; in the matching and unification of names; in the reduction or removal of redundancy from unary numbers; in the workings of Post's Canonical System and the transition function in the Universal Turing Machine; in the way computers retrieve information from memory; in systems like Prolog; and in the query-by-example technique for information retrieval. The chunking-with-codes technique for IC may be seen in the use of named functions to avoid repetition of computer code. The schema-plus-correction technique may be seen in functions with parameters and in the use of classes in object-oriented programming. And the run-length coding technique may be seen in multiplication, in division, and in several other devices in mathematics and computing. The SP theory resolves the apparent paradox of "decompression by compression". And computing and cognition as IC is compatible with the uses of redundancy in such things as backup copies to safeguard data and understanding speech in a noisy environment. △ Less

Submitted 13 July, 2015; v1 submitted 31 October, 2013; originally announced October 2013.

The SP theory of intelligence: benefits and applications

Authors: J Gerard Wolff

Abstract: This article describes existing and expected benefits of the "SP theory of intelligence", and some potential applications. The theory aims to simplify and integrate ideas across artificial intelligence, mainstream computing, and human perception and cognition, with information compression as a unifying theme. It combines conceptual simplicity with descriptive and explanatory power across several a… ▽ More This article describes existing and expected benefits of the "SP theory of intelligence", and some potential applications. The theory aims to simplify and integrate ideas across artificial intelligence, mainstream computing, and human perception and cognition, with information compression as a unifying theme. It combines conceptual simplicity with descriptive and explanatory power across several areas of computing and cognition. In the "SP machine" -- an expression of the SP theory which is currently realized in the form of a computer model -- there is potential for an overall simplification of computing systems, including software. The SP theory promises deeper insights and better solutions in several areas of application including, most notably, unsupervised learning, natural language processing, autonomous robots, computer vision, intelligent databases, software engineering, information compression, medical diagnosis and big data. There is also potential in areas such as the semantic web, bioinformatics, structuring of documents, the detection of computer viruses, data fusion, new kinds of computer, and the development of scientific theories. The theory promises seamless integration of structures and functions within and between different areas of application. The potential value, worldwide, of these benefits and applications is at least $190 billion each year. Further development would be facilitated by the creation of a high-parallel, open-source version of the SP machine, available to researchers everywhere. △ Less

Submitted 23 December, 2013; v1 submitted 13 June, 2013; originally announced July 2013.

Comments: arXiv admin note: substantial text overlap with arXiv:1212.0229

Journal ref: J G Wolff, Information, 5 (1), 1-27, 2014

Big data and the SP theory of intelligence

Authors: J. Gerard Wolff

Abstract: This article is about how the "SP theory of intelligence" and its realisation in the "SP machine" may, with advantage, be applied to the management and analysis of big data. The SP system -- introduced in the article and fully described elsewhere -- may help to overcome the problem of variety in big data: it has potential as "a universal framework for the representation and processing of diverse k… ▽ More This article is about how the "SP theory of intelligence" and its realisation in the "SP machine" may, with advantage, be applied to the management and analysis of big data. The SP system -- introduced in the article and fully described elsewhere -- may help to overcome the problem of variety in big data: it has potential as "a universal framework for the representation and processing of diverse kinds of knowledge" (UFK), helping to reduce the diversity of formalisms and formats for knowledge and the different ways in which they are processed. It has strengths in the unsupervised learning or discovery of structure in data, in pattern recognition, in the parsing and production of natural language, in several kinds of reasoning, and more. It lends itself to the analysis of streaming data, helping to overcome the problem of velocity in big data. Central in the workings of the system is lossless compression of information: making big data smaller and reducing problems of storage and management. There is potential for substantial economies in the transmission of data, for big cuts in the use of energy in computing, for faster processing, and for smaller and lighter computers. The system provides a handle on the problem of veracity in big data, with potential to assist in the management of errors and uncertainties in data. It lends itself to the visualisation of knowledge structures and inferential processes. A high-parallel, open-source version of the SP machine would provide a means for researchers everywhere to explore what can be done with the system and to create new versions of it. △ Less

Submitted 31 March, 2014; v1 submitted 13 June, 2013; originally announced June 2013.

Comments: Accepted for publication in IEEE Access

Journal ref: J G Wolff, IEEE Access, 2, 301-315, 2014

The SP theory of intelligence: an overview

Authors: J. Gerard Wolff

Abstract: This article is an overview of the "SP theory of intelligence". The theory aims to simplify and integrate concepts across artificial intelligence, mainstream computing and human perception and cognition, with information compression as a unifying theme. It is conceived as a brain-like system that receives 'New' information and stores some or all of it in compressed form as 'Old' information. It is… ▽ More This article is an overview of the "SP theory of intelligence". The theory aims to simplify and integrate concepts across artificial intelligence, mainstream computing and human perception and cognition, with information compression as a unifying theme. It is conceived as a brain-like system that receives 'New' information and stores some or all of it in compressed form as 'Old' information. It is realised in the form of a computer model -- a first version of the SP machine. The concept of "multiple alignment" is a powerful central idea. Using heuristic techniques, the system builds multiple alignments that are 'good' in terms of information compression. For each multiple alignment, probabilities may be calculated. These provide the basis for calculating the probabilities of inferences. The system learns new structures from partial matches between patterns. Using heuristic techniques, the system searches for sets of structures that are 'good' in terms of information compression. These are normally ones that people judge to be 'natural', in accordance with the 'DONSVIC' principle -- the discovery of natural structures via information compression. The SP theory may be applied in several areas including 'computing', aspects of mathematics and logic, representation of knowledge, natural language processing, pattern recognition, several kinds of reasoning, information storage and retrieval, planning and problem solving, information compression, neuroscience, and human perception and cognition. Examples include the parsing and production of language including discontinuous dependencies in syntax, pattern recognition at multiple levels of abstraction and its integration with part-whole relations, nonmonotonic reasoning and reasoning with default values, reasoning in Bayesian networks including 'explaining away', causal diagnosis, and the solving of a geometric analogy problem. △ Less

Submitted 7 January, 2015; v1 submitted 13 June, 2013; originally announced June 2013.

Comments: arXiv admin note: text overlap with arXiv:cs/0401009, arXiv:1303.2071, arXiv:cs/0307010, arXiv:1212.0229, arXiv:1303.2013

Journal ref: J G Wolff, Information, 4 (3), 283-341, 2013

Application of the SP theory of intelligence to the understanding of natural vision and the development of computer vision

Authors: J. Gerard Wolff

Abstract: The SP theory of intelligence aims to simplify and integrate concepts in computing and cognition, with information compression as a unifying theme. This article discusses how it may be applied to the understanding of natural vision and the development of computer vision. The theory, which is described quite fully elsewhere, is described here in outline but with enough detail to ensure that the res… ▽ More The SP theory of intelligence aims to simplify and integrate concepts in computing and cognition, with information compression as a unifying theme. This article discusses how it may be applied to the understanding of natural vision and the development of computer vision. The theory, which is described quite fully elsewhere, is described here in outline but with enough detail to ensure that the rest of the article makes sense. Low level perceptual features such as edges or corners may be identified by the extraction of redundancy in uniform areas in a manner that is comparable with the run-length encoding technique for information compression. The concept of multiple alignment in the SP theory may be applied to the recognition of objects, and to scene analysis, with a hierarchy of parts and sub-parts, and at multiple levels of abstraction. The theory has potential for the unsupervised learning of visual objects and classes of objects, and suggests how coherent concepts may be derived from fragments. As in natural vision, both recognition and learning in the SP system is robust in the face of errors of omission, commission and substitution. The theory suggests how, via vision, we may piece together a knowledge of the three-dimensional structure of objects and of our environment, it provides an account of how we may see things that are not objectively present in an image, and how we recognise something despite variations in the size of its retinal image. And it has things to say about the phenomena of lightness constancy and colour constancy, the role of context in recognition, and ambiguities in visual perception. A strength of the SP theory is that it provides for the integration of vision with other sensory modalities and with other aspects of intelligence. △ Less

Submitted 23 January, 2015; v1 submitted 8 March, 2013; originally announced March 2013.

Comments: 40 pages, 16 figures

Journal ref: SpringerPlus, 3, 552, 2014

Computing as compression: the SP theory of intelligence

Authors: J Gerard Wolff

Abstract: This paper provides an overview of the SP theory of intelligence and its central idea that artificial intelligence, mainstream computing, and much of human perception and cognition, may be understood as information compression. The background and origins of the SP theory are described, and the main elements of the theory, including the key concept of multiple alignment, borrowed from bioinformat… ▽ More This paper provides an overview of the SP theory of intelligence and its central idea that artificial intelligence, mainstream computing, and much of human perception and cognition, may be understood as information compression. The background and origins of the SP theory are described, and the main elements of the theory, including the key concept of multiple alignment, borrowed from bioinformatics but with important differences. Associated with the SP theory is the idea that redundancy in information may be understood as repetition of patterns, that compression of information may be achieved via the matching and unification (merging) of patterns, and that computing and information compression are both fundamentally probabilistic. It appears that the SP system is Turing-equivalent in the sense that anything that may be computed with a Turing machine may, in principle, also be computed with an SP machine. One of the main strengths of the SP theory and the multiple alignment concept is in modelling concepts and phenomena in artificial intelligence. Within that area, the SP theory provides a simple but versatile means of representing different kinds of knowledge, it can model both the parsing and production of natural language, with potential for the understanding and translation of natural languages, it has strengths in pattern recognition, with potential in computer vision, it can model several kinds of reasoning, and it has capabilities in planning, problem solving, and unsupervised learning. The paper includes two examples showing how alternative parsings of an ambiguous sentence may be modelled as multiple alignments, and another example showing how the concept of multiple alignment may be applied in medical diagnosis. △ Less

Submitted 8 March, 2013; originally announced March 2013.

Comments: 8 pages, 2 figures. arXiv admin note: text overlap with arXiv:1212.0229

Simplification and integration in computing and cognition: the SP theory and the multiple alignment concept

Authors: James Gerard Wolff

Abstract: The main purpose of this article is to describe potential benefits and applications of the SP theory, a unique attempt to simplify and integrate ideas across artificial intelligence, mainstream computing and human cognition, with information compression as a unifying theme. The theory, including a concept of multiple alignment, combines conceptual simplicity with descriptive and explanatory power… ▽ More The main purpose of this article is to describe potential benefits and applications of the SP theory, a unique attempt to simplify and integrate ideas across artificial intelligence, mainstream computing and human cognition, with information compression as a unifying theme. The theory, including a concept of multiple alignment, combines conceptual simplicity with descriptive and explanatory power in several areas including representation of knowledge, natural language processing, pattern recognition, several kinds of reasoning, the storage and retrieval of information, planning and problem solving, unsupervised learning, information compression, and human perception and cognition. In the SP machine -- an expression of the SP theory which is currently realised in the form of computer models -- there is potential for an overall simplification of computing systems, including software. As a theory with a broad base of support, the SP theory promises useful insights in many areas and the integration of structures and functions, both within a given area and amongst different areas. There are potential benefits in natural language processing (with potential for the understanding and translation of natural languages), the need for a versatile intelligence in autonomous robots, computer vision, intelligent databases, maintaining multiple versions of documents or web pages, software engineering, criminal investigations, the management of big data and gaining benefits from it, the semantic web, medical diagnosis, the detection of computer viruses, the economical transmission of data, and data fusion. Further development of these ideas would be facilitated by the creation of a high-parallel, web-based, open-source version of the SP machine, with a good user interface. This would provide a means for researchers to explore what can be done with the system and to refine it. △ Less

Submitted 2 December, 2012; originally announced December 2012.

Comments: 32 pages, 5 figures

Unifying Computing and Cognition: The SP Theory and its Applications

Authors: J Gerard Wolff

Abstract: This book develops the conjecture that all kinds of information processing in computers and in brains may usefully be understood as "information compression by multiple alignment, unification and search". This "SP theory", which has been under development since 1987, provides a unified view of such things as the workings of a universal Turing machine, the nature of 'knowledge', the interpretatio… ▽ More This book develops the conjecture that all kinds of information processing in computers and in brains may usefully be understood as "information compression by multiple alignment, unification and search". This "SP theory", which has been under development since 1987, provides a unified view of such things as the workings of a universal Turing machine, the nature of 'knowledge', the interpretation and production of natural language, pattern recognition and best-match information retrieval, several kinds of probabilistic reasoning, planning and problem solving, unsupervised learning, and a range of concepts in mathematics and logic. The theory also provides a basis for the design of an 'SP' computer with several potential advantages compared with traditional digital computers. △ Less

Submitted 13 January, 2004; originally announced January 2004.

ACM Class: F.1.0; I.2.0

Unsupervised Grammar Induction in a Framework of Information Compression by Multiple Alignment, Unification and Search

Authors: J Gerard Wolff

Abstract: This paper describes a novel approach to grammar induction that has been developed within a framework designed to integrate learning with other aspects of computing, AI, mathematics and logic. This framework, called "information compression by multiple alignment, unification and search" (ICMAUS), is founded on principles of Minimum Length Encoding pioneered by Solomonoff and others. Most of the… ▽ More This paper describes a novel approach to grammar induction that has been developed within a framework designed to integrate learning with other aspects of computing, AI, mathematics and logic. This framework, called "information compression by multiple alignment, unification and search" (ICMAUS), is founded on principles of Minimum Length Encoding pioneered by Solomonoff and others. Most of the paper describes SP70, a computer model of the ICMAUS framework that incorporates processes for unsupervised learning of grammars. An example is presented to show how the model can infer a plausible grammar from appropriate input. Limitations of the current model and how they may be overcome are briefly discussed. △ Less

Submitted 27 November, 2003; originally announced November 2003.

ACM Class: I.2.6

Journal ref: Proceedings of the Workshop and Tutorial on Learning Context-Free Grammars (in association with the 14th European Conference on Machine Learning and the 7th European Conference on Principles and Practice of Knowledge Discovery in Databases (ECML/PKDD 2003), September 2003, Cavtat-Dubrovnik, Croata), editors: C. de la Higuera and P. Adriaans and M. van Zaanen and J. Oncina, pp 113-124

Towards an Intelligent Database System Founded on the SP Theory of Computing and Cognition

Authors: J. Gerard Wolff

Abstract: The SP theory of computing and cognition, described in previous publications, is an attractive model for intelligent databases because it provides a simple but versatile format for different kinds of knowledge, it has capabilities in artificial intelligence, and it can also function like established database models when that is required. This paper describes how the SP model can emulate other… ▽ More The SP theory of computing and cognition, described in previous publications, is an attractive model for intelligent databases because it provides a simple but versatile format for different kinds of knowledge, it has capabilities in artificial intelligence, and it can also function like established database models when that is required. This paper describes how the SP model can emulate other models used in database applications and compares the SP model with those other models. The artificial intelligence capabilities of the SP model are reviewed and its relationship with other artificial intelligence systems is described. Also considered are ways in which current prototypes may be translated into an 'industrial strength' working system. △ Less

Submitted 21 November, 2003; originally announced November 2003.

ACM Class: H.2.1

Journal ref: J G Wolff, Data & Knowledge Engineering 60, 596-624, 2007

Mathematics and Logic as Information Compression by Multiple Alignment, Unification and Search

Authors: J Gerard Wolff

Abstract: This article introduces the conjecture that "mathematics, logic and related disciplines may usefully be understood as information compression (IC) by 'multiple alignment', 'unification' and 'search' (ICMAUS)". As a preparation for the two main sections of the article, concepts of information and information compression are reviewed. Related areas of research are also described including IC in… ▽ More This article introduces the conjecture that "mathematics, logic and related disciplines may usefully be understood as information compression (IC) by 'multiple alignment', 'unification' and 'search' (ICMAUS)". As a preparation for the two main sections of the article, concepts of information and information compression are reviewed. Related areas of research are also described including IC in brains and nervous systems, and IC in relation to inductive inference, Minimum Length Encoding and probabilistic reasoning. The ICMAUS concepts and a computer model in which they are embodied are briefly described. The first of the two main sections describes how many of the commonly-used forms and structures in mathematics, logic and related disciplines (such as theoretical linguistics and computer programming) may be seen as devices for IC. In some cases, these forms and structures may be interpreted in terms of the ICMAUS framework. The second main section describes a selection of examples where processes of calculation and inference in mathematics, logic and related disciplines may be understood as IC. In many cases, these examples may be understood more specifically in terms of the ICMAUS concepts. △ Less

Submitted 15 August, 2003; originally announced August 2003.

MSC Class: 00A30; 03A05

An Alternative to RDF-Based Languages for the Representation and Processing of Ontologies in the Semantic Web

Authors: J Gerard Wolff

Abstract: This paper describes an approach to the representation and processing of ontologies in the Semantic Web, based on the ICMAUS theory of computation and AI. This approach has strengths that complement those of languages based on the Resource Description Framework (RDF) such as RDF Schema and DAML+OIL. The main benefits of the ICMAUS approach are simplicity and comprehensibility in the representati… ▽ More This paper describes an approach to the representation and processing of ontologies in the Semantic Web, based on the ICMAUS theory of computation and AI. This approach has strengths that complement those of languages based on the Resource Description Framework (RDF) such as RDF Schema and DAML+OIL. The main benefits of the ICMAUS approach are simplicity and comprehensibility in the representation of ontologies, an ability to cope with errors and uncertainties in knowledge, and a versatile reasoning system with capabilities in the kinds of probabilistic reasoning that seem to be required in the Semantic Web. △ Less

Submitted 29 July, 2003; originally announced July 2003.

ACM Class: I.2.0

Neural realisation of the SP theory: cell assemblies revisited

Authors: J. Gerard Wolff

Abstract: This paper describes how the elements of the SP theory (Wolff, 2003a) may be realised with neural structures and processes. To the extent that this is successful, the insights that have been achieved in the SP theory - the integration and simplification of a range of phenomena in perception and cognition - may be incorporated in a neural view of brain function. These proposals may be seen as a… ▽ More This paper describes how the elements of the SP theory (Wolff, 2003a) may be realised with neural structures and processes. To the extent that this is successful, the insights that have been achieved in the SP theory - the integration and simplification of a range of phenomena in perception and cognition - may be incorporated in a neural view of brain function. These proposals may be seen as a development of Hebb's (1949) concept of a 'cell assembly'. By contrast with that concept and variants of it, the version described in this paper proposes that any one neuron can belong in one assembly and only one assembly. A distinctive feature of the present proposals is that any neuron or cluster of neurons within a cell assembly may serve as a proxy or reference for another cell assembly or class of cell assemblies. This device provides solutions to many of the problems associated with cell assemblies, it allows information to be stored in a compressed form, and it provides a robust mechanism by which assemblies may be connected to form hierarchies, grammars and other kinds of knowledge structure. Drawing on insights derived from the SP theory, the paper also describes how unsupervised learning may be achieved with neural structures and processes. This theory of learning overcomes weaknesses in the Hebbian concept of learning and it is, at the same time, compatible with the observations that Hebb's theory was designed to explain. △ Less

Submitted 13 May, 2004; v1 submitted 27 July, 2003; originally announced July 2003.

Comments: This is a radical revision with new content which is shorter and, I hope, clearer

ACM Class: I.2.0

Information Compression by Multiple Alignment, Unification and Search as a Unifying Principle in Computing and Cognition

Authors: J Gerard Wolff

Abstract: This article presents an overview of the idea that "information compression by multiple alignment, unification and search" (ICMAUS) may serve as a unifying principle in computing (including mathematics and logic) and in such aspects of human cognition as the analysis and production of natural language, fuzzy pattern recognition and best-match information retrieval, concept hierarchies with inher… ▽ More This article presents an overview of the idea that "information compression by multiple alignment, unification and search" (ICMAUS) may serve as a unifying principle in computing (including mathematics and logic) and in such aspects of human cognition as the analysis and production of natural language, fuzzy pattern recognition and best-match information retrieval, concept hierarchies with inheritance of attributes, probabilistic reasoning, and unsupervised inductive learning. The ICMAUS concepts are described together with an outline of the SP61 software model in which the ICMAUS concepts are currently realised. A range of examples is presented, illustrated with output from the SP61 model. △ Less

Submitted 10 July, 2003; originally announced July 2003.

ACM Class: I.2.0

Journal ref: Artificial Intelligence Review 19(3), 193-230, 2003

Syntax, Parsing and Production of Natural Language in a Framework of Information Compression by Multiple Alignment, Unification and Search

Authors: J Gerard Wolff

Abstract: This article introduces the idea that "information compression by multiple alignment, unification and search" (ICMAUS) provides a framework within which natural language syntax may be represented in a simple format and the parsing and production of natural language may be performed in a transparent manner. The ICMAUS concepts are embodied in a software model, SP61. The organisation and operati… ▽ More This article introduces the idea that "information compression by multiple alignment, unification and search" (ICMAUS) provides a framework within which natural language syntax may be represented in a simple format and the parsing and production of natural language may be performed in a transparent manner. The ICMAUS concepts are embodied in a software model, SP61. The organisation and operation of the model are described and a simple example is presented showing how the model can achieve parsing of natural language. Notwithstanding the apparent paradox of 'decompression by compression', the ICMAUS framework, without any modification, can produce a sentence by decoding a compressed code for the sentence. This is illustrated with output from the SP61 model. The article includes four other examples - one of the parsing of a sentence in French and three from the domain of English auxiliary verbs. These examples show how the ICMAUS framework and the SP61 model can accommodate 'context sensitive' features of syntax in a relatively simple and direct manner. △ Less

Submitted 7 July, 2003; originally announced July 2003.

ACM Class: I.2.7

Journal ref: Journal of Universal Computer Science 6(8), 781--829, 2000

'Computing' as Information Compression by Multiple Alignment, Unification and Search

Authors: J Gerard Wolff

Abstract: This paper argues that the operations of a 'Universal Turing Machine' (UTM) and equivalent mechanisms such as the 'Post Canonical System' (PCS) - which are widely accepted as definitions of the concept of `computing' - may be interpreted as *information compression by multiple alignment, unification and search* (ICMAUS). The motivation for this interpretation is that it suggests ways in which… ▽ More This paper argues that the operations of a 'Universal Turing Machine' (UTM) and equivalent mechanisms such as the 'Post Canonical System' (PCS) - which are widely accepted as definitions of the concept of `computing' - may be interpreted as *information compression by multiple alignment, unification and search* (ICMAUS). The motivation for this interpretation is that it suggests ways in which the UTM/PCS model may be augmented in a proposed new computing system designed to exploit the ICMAUS principles as fully as possible. The provision of a relatively sophisticated search mechanism in the proposed 'SP' system appears to open the door to the integration and simplification of a range of functions including unsupervised inductive learning, best-match pattern recognition and information retrieval, probabilistic reasoning, planning and problem solving, and others. Detailed consideration of how the ICMAUS principles may be applied to these functions is outside the scope of this article but relevant sources are cited in this article. △ Less

Submitted 5 July, 2003; originally announced July 2003.

ACM Class: F.0

Journal ref: Journal of Universal Computer Science 5(11), 776--815, 1999

Probabilistic Reasoning as Information Compression by Multiple Alignment, Unification and Search: An Introduction and Overview

Authors: J Gerard Wolff

Abstract: This article introduces the idea that probabilistic reasoning (PR) may be understood as "information compression by multiple alignment, unification and search" (ICMAUS). In this context, multiple alignment has a meaning which is similar to but distinct from its meaning in bio-informatics, while unification means a simple merging of matching patterns, a meaning which is related to but simpler tha… ▽ More This article introduces the idea that probabilistic reasoning (PR) may be understood as "information compression by multiple alignment, unification and search" (ICMAUS). In this context, multiple alignment has a meaning which is similar to but distinct from its meaning in bio-informatics, while unification means a simple merging of matching patterns, a meaning which is related to but simpler than the meaning of that term in logic. A software model, SP61, has been developed for the discovery and formation of 'good' multiple alignments, evaluated in terms of information compression. The model is described in outline. Using examples from the SP61 model, this article describes in outline how the ICMAUS framework can model various kinds of PR including: PR in best-match pattern recognition and information retrieval; one-step 'deductive' and 'abductive' PR; inheritance of attributes in a class hierarchy; chains of reasoning (probabilistic decision networks and decision trees, and PR with 'rules'); geometric analogy problems; nonmonotonic reasoning and reasoning with default values; modelling the function of a Bayesian network. △ Less

Submitted 6 July, 2003; v1 submitted 4 July, 2003; originally announced July 2003.

ACM Class: I.2.3

Journal ref: Journal of Universal Computer Science 5 (7), 418--462, 1999

Unsupervised Learning in a Framework of Information Compression by Multiple Alignment, Unification and Search

Authors: J. G. Wolff

Abstract: This paper describes a novel approach to unsupervised learning that has been developed within a framework of "information compression by multiple alignment, unification and search" (ICMAUS), designed to integrate learning with other AI functions such as parsing and production of language, fuzzy pattern recognition, probabilistic and exact forms of reasoning, and others. This paper describes a novel approach to unsupervised learning that has been developed within a framework of "information compression by multiple alignment, unification and search" (ICMAUS), designed to integrate learning with other AI functions such as parsing and production of language, fuzzy pattern recognition, probabilistic and exact forms of reasoning, and others. △ Less

Submitted 12 February, 2003; originally announced February 2003.

Comments: 39 pages, 1 JPEG figure

ACM Class: I.2.4; I.2.6; I.2.7