-
Ultra-Quantisation: Efficient Embedding Search via 1.58-bit Encodings
Authors:
Richard Connor,
Alan Dearle,
Ben Claydon
Abstract:
Many modern search domains comprise high-dimensional vectors of floating point numbers derived from neural networks, in the form of embeddings. Typical embeddings range in size from hundreds to thousands of dimensions, making the size of the embeddings, and the speed of comparison, a significant issue.
Quantisation is a class of mechanism which replaces the floating point values with a smaller r…
▽ More
Many modern search domains comprise high-dimensional vectors of floating point numbers derived from neural networks, in the form of embeddings. Typical embeddings range in size from hundreds to thousands of dimensions, making the size of the embeddings, and the speed of comparison, a significant issue.
Quantisation is a class of mechanism which replaces the floating point values with a smaller representation, for example a short integer. This gives an approximation of the embedding space in return for a smaller data representation and a faster comparison function.
Here we take this idea almost to its extreme: we show how vectors of arbitrary-precision floating point values can be replaced by vectors whose elements are drawn from the set {-1,0,1}. This yields very significant savings in space and metric evaluation cost, while maintaining a strong correlation for similarity measurements.
This is achieved by way of a class of convex polytopes which exist in the high-dimensional space. In this article we give an outline description of these objects, and show how they can be used for the basis of such radical quantisation while maintaining a surprising degree of accuracy.
△ Less
Submitted 31 May, 2025;
originally announced June 2025.
-
Workflow Partitioning and Deployment on the Cloud using Orchestra
Authors:
Ward Jaradat,
Alan Dearle,
Adam Barker
Abstract:
Orchestrating service-oriented workflows is typically based on a design model that routes both data and control through a single point - the centralised workflow engine. This causes scalability problems that include the unnecessary consumption of the network bandwidth, high latency in transmitting data between the services, and performance bottlenecks. These problems are highly prominent when orch…
▽ More
Orchestrating service-oriented workflows is typically based on a design model that routes both data and control through a single point - the centralised workflow engine. This causes scalability problems that include the unnecessary consumption of the network bandwidth, high latency in transmitting data between the services, and performance bottlenecks. These problems are highly prominent when orchestrating workflows that are composed from services dispersed across distant geographical locations. This paper presents a novel workflow partitioning approach, which attempts to improve the scalability of orchestrating large-scale workflows. It permits the workflow computation to be moved towards the services providing the data in order to garner optimal performance results. This is achieved by decomposing the workflow into smaller sub workflows for parallel execution, and determining the most appropriate network locations to which these sub workflows are transmitted and subsequently executed. This paper demonstrates the efficiency of our approach using a set of experimental workflows that are orchestrated over Amazon EC2 and across several geographic network regions.
△ Less
Submitted 29 September, 2014;
originally announced September 2014.
-
Minimising virtual machine support for concurrency
Authors:
Simon Dobson,
Alan Dearle,
Barry Porter
Abstract:
Co-operative and pre-emptive scheduling are usually considered to be complementary models of threading. In the case of virtual machines, we show that they can be unified using a single concept, the bounded execution of a thread of control, essentially providing a first-class representation of a computation as it is reduced. Furthermore this technique can be used to surface the thread scheduler of…
▽ More
Co-operative and pre-emptive scheduling are usually considered to be complementary models of threading. In the case of virtual machines, we show that they can be unified using a single concept, the bounded execution of a thread of control, essentially providing a first-class representation of a computation as it is reduced. Furthermore this technique can be used to surface the thread scheduler of a language into the language itself, allowing programs to provide their own schedulers without any additional support in the virtual machine, and allowing the same virtual machine to support different thread models simultaneously and without re-compilation.
△ Less
Submitted 10 December, 2013;
originally announced December 2013.
-
An Architecture for Decentralised Orchestration of Web Service Workflows
Authors:
Ward Jaradat,
Alan Dearle,
Adam Barker
Abstract:
Service-oriented workflows are typically executed using a centralised orchestration approach that presents significant scalability challenges. These challenges include the consumption of network bandwidth, degradation of performance, and single-points of failure. We provide a decentralised orchestration architecture that attempts to address these challenges. Our architecture adopts a design model…
▽ More
Service-oriented workflows are typically executed using a centralised orchestration approach that presents significant scalability challenges. These challenges include the consumption of network bandwidth, degradation of performance, and single-points of failure. We provide a decentralised orchestration architecture that attempts to address these challenges. Our architecture adopts a design model that permits the computation to be moved "closer" to services in a workflow. This is achieved by partitioning workflows specified using our simple dataflow language into smaller fragments, which may be sent to remote locations for execution.
△ Less
Submitted 8 May, 2013;
originally announced May 2013.
-
A Dataflow Language for Decentralised Orchestration of Web Service Workflows
Authors:
Ward Jaradat,
Alan Dearle,
Adam Barker
Abstract:
Orchestrating centralised service-oriented workflows presents significant scalability challenges that include: the consumption of network bandwidth, degradation of performance, and single points of failure. This paper presents a high-level dataflow specification language that attempts to address these scalability challenges. This language provides simple abstractions for orchestrating large-scale…
▽ More
Orchestrating centralised service-oriented workflows presents significant scalability challenges that include: the consumption of network bandwidth, degradation of performance, and single points of failure. This paper presents a high-level dataflow specification language that attempts to address these scalability challenges. This language provides simple abstractions for orchestrating large-scale web service workflows, and separates between the workflow logic and its execution. It is based on a data-driven model that permits parallelism to improve the workflow performance. We provide a decentralised architecture that allows the computation logic to be moved "closer" to services involved in the workflow. This is achieved through partitioning the workflow specification into smaller fragments that may be sent to remote orchestration services for execution. The orchestration services rely on proxies that exploit connectivity to services in the workflow. These proxies perform service invocations and compositions on behalf of the orchestration services, and carry out data collection, retrieval, and mediation tasks. The evaluation of our architecture implementation concludes that our decentralised approach reduces the execution time of workflows, and scales accordingly with the increasing size of data sets.
△ Less
Submitted 8 May, 2013;
originally announced May 2013.
-
First Smart Spaces
Authors:
Graham Kirby,
Alan Dearle,
Andrew McCarthy,
Ron Morrison,
Kevin Mullen,
Yanyan Yang,
Richard Connor,
Paula Welen,
Andy Wilson
Abstract:
This document describes the Gloss software currently implemented. The description of the Gloss demonstrator for multi-surface interaction can be found in D17. The ongoing integration activity for the work described in D17 and D8 constitutes our development of infrastructure for a first smart space. In this report, the focus is on infrastructure to support the implementation of location aware servi…
▽ More
This document describes the Gloss software currently implemented. The description of the Gloss demonstrator for multi-surface interaction can be found in D17. The ongoing integration activity for the work described in D17 and D8 constitutes our development of infrastructure for a first smart space. In this report, the focus is on infrastructure to support the implementation of location aware services. A local architecture provides a framework for constructing Gloss applications, termed assemblies, that run on individual physical nodes. A global architecture defines an overlay network for linking individual assemblies. Both local and global architectures are under active development.
△ Less
Submitted 1 July, 2010;
originally announced July 2010.
-
Second Set of Spaces
Authors:
Evangelos Zirintsis,
Graham Kirby,
Alan Dearle,
Ben Allen,
Rob MacInnis,
Andrew McCarthy,
Ron Morrison,
Paddy Nixon,
Andrew Jamieson,
Chris Nicholson,
Steven Harris
Abstract:
This document describes the Gloss infrastructure supporting implementation of location-aware services. The document is in two parts. The first part describes software architecture for the smart space. As described in D8, a local architecture provides a framework for constructing Gloss applications, termed assemblies, that run on individual physical nodes, whereas a global architecture defines an o…
▽ More
This document describes the Gloss infrastructure supporting implementation of location-aware services. The document is in two parts. The first part describes software architecture for the smart space. As described in D8, a local architecture provides a framework for constructing Gloss applications, termed assemblies, that run on individual physical nodes, whereas a global architecture defines an overlay network for linking individual assemblies. The second part outlines the hardware installation for local sensing. This describes the first phase of the installation in Strathclyde University.
△ Less
Submitted 29 June, 2010;
originally announced June 2010.
-
An Information Flow Architecture for Global Smart Spaces
Authors:
Alan Dearle,
Graham Kirby,
Andrew McCarthy,
Juan-Carlos Diaz y Carballo
Abstract:
In this paper we describe an architecture which: Permits the deployment and execution of components in appropriate geographical locations. Provides security mechanisms that prevent misuse of the architecture. Supports a programming model that is familiar to application programmers. Permits installed components to share data. Permits the deployed components to communicate via communication channels…
▽ More
In this paper we describe an architecture which: Permits the deployment and execution of components in appropriate geographical locations. Provides security mechanisms that prevent misuse of the architecture. Supports a programming model that is familiar to application programmers. Permits installed components to share data. Permits the deployed components to communicate via communication channels. Provides evolution mechanisms permitting the dynamic rearrangement of inter-connection topologies the components that they connect. Supports the specification and deployment of distributed component deployments.
△ Less
Submitted 29 June, 2010;
originally announced June 2010.
-
Report on the XBase Project
Authors:
Evangelos Zirintsis,
Graham Kirby,
Alan Dearle,
Ron Morrison
Abstract:
This project addressed the conceptual fundamentals of data storage, investigating techniques for provision of highly generic storage facilities that can be tailored to produce various individually customised storage infrastructures, compliant to the needs of particular applications. This requires the separation of mechanism and policy wherever possible. Aspirations include: actors, whether users o…
▽ More
This project addressed the conceptual fundamentals of data storage, investigating techniques for provision of highly generic storage facilities that can be tailored to produce various individually customised storage infrastructures, compliant to the needs of particular applications. This requires the separation of mechanism and policy wherever possible. Aspirations include: actors, whether users or individual processes, should be able to bind to, update and manipulate data and programs transparently with respect to their respective locations; programs should be expressed independently of the storage and network technology involved in their execution; storage facilities should be structure-neutral so that actors can impose multiple interpretations over information, simultaneously and safely; information should not be discarded so that arbitrary historical views are supported; raw stored information should be open to all; where security restrictions on its use are required this should be achieved using cryptographic techniques. The key advances of the research were: 1) the identification of a candidate set of minimal storage system building blocks, which are sufficiently simple to avoid encapsulating policy where it cannot be customised by applications, and composable to build highly flexible storage architectures 2) insight into the nature of append-only storage components, and the issues arising from their application to common storage use-cases.
△ Less
Submitted 30 June, 2010;
originally announced June 2010.
-
Working Document on Gloss Ontology
Authors:
Joelle Coutaz,
Alan Dearle,
Sophie Dupuy-Chessa,
Graham Kirby,
Christophe Lachenal,
Ron Morrison,
Gaetan Rey,
Evangelos Zirintsis
Abstract:
This document describes the Gloss Ontology. The ontology and associated class model are organised into several packages. Section 2 describes each package in detail, while Section 3 contains a summary of the whole ontology.
This document describes the Gloss Ontology. The ontology and associated class model are organised into several packages. Section 2 describes each package in detail, while Section 3 contains a summary of the whole ontology.
△ Less
Submitted 29 June, 2010;
originally announced June 2010.
-
A Reflective Approach to Providing Flexibility in Application Distribution
Authors:
Álvaro Rebón Portillo,
Scott Walker,
Graham Kirby,
Alan Dearle
Abstract:
Current middleware systems suffer from drawbacks. Often one is forced to make decisions early in the design process about which classes may participate in inter-machine communication. Further, application level and middleware specific semantics cannot be separated forcing an unnatural design. The RAFDA project proposes to adress these deficiencies by creating an adaptive, reflective framework that…
▽ More
Current middleware systems suffer from drawbacks. Often one is forced to make decisions early in the design process about which classes may participate in inter-machine communication. Further, application level and middleware specific semantics cannot be separated forcing an unnatural design. The RAFDA project proposes to adress these deficiencies by creating an adaptive, reflective framework that enables the transformation of non-distributed applications into isomorphic applications whose distribution architecture is flexible. This paper describes the code transformation techniques that have been developed as part of the project. The system enables the distribution of a program according to a flexible configuration without user intervention. Proxy objects can then be substituted, permitting cross-address space communication. The distributed program can adapt to its environment by dynamically altering its distribution boundaries.
△ Less
Submitted 29 June, 2010;
originally announced June 2010.
-
A Generic Storage API
Authors:
Graham Kirby,
Evangelos Zirintsis,
Alan Dearle,
Ron Morrison
Abstract:
We present a generic API suitable for provision of highly generic storage facilities that can be tailored to produce various individually customised storage infrastructures. The paper identifies a candidate set of minimal storage system building blocks, which are sufficiently simple to avoid encapsulating policy where it cannot be customised by applications, and composable to build highly flexible…
▽ More
We present a generic API suitable for provision of highly generic storage facilities that can be tailored to produce various individually customised storage infrastructures. The paper identifies a candidate set of minimal storage system building blocks, which are sufficiently simple to avoid encapsulating policy where it cannot be customised by applications, and composable to build highly flexible storage architectures. Four main generic components are defined: the store, the namer, the caster and the interpreter. It is hypothesised that these are sufficiently general that they could act as building blocks for any information storage and retrieval system. The essential characteristics of each are defined by an interface, which may be implemented by multiple implementing classes.
△ Less
Submitted 24 June, 2010;
originally announced June 2010.
-
Architectural Support for Global Smart Spaces
Authors:
Alan Dearle,
Graham Kirby,
Ron Morrison,
Andrew McCarthy,
Kevin Mullen,
Yanyan Yang,
Richard Connor,
Paula Welen,
Andy Wilson
Abstract:
A GLObal Smart Space (GLOSS) provides support for interaction amongst people, artefacts and places while taking account of both context and movement on a global scale. Crucial to the definition of a GLOSS is the provision of a set of location-aware services that detect, convey, store and exploit location information. We use one of these services, hearsay, to illustrate the implementation dimension…
▽ More
A GLObal Smart Space (GLOSS) provides support for interaction amongst people, artefacts and places while taking account of both context and movement on a global scale. Crucial to the definition of a GLOSS is the provision of a set of location-aware services that detect, convey, store and exploit location information. We use one of these services, hearsay, to illustrate the implementation dimensions of a GLOSS. The focus of the paper is on both local and global software architecture to support the implementation of such services. The local architecture is based on XML pipe-lines and is used to construct location-aware components. The global architecture is based on a hybrid peer-to-peer routing scheme and provides the local architectures with the means to communicate in the global context.
△ Less
Submitted 24 June, 2010;
originally announced June 2010.
-
Active Architecture for Pervasive Contextual Services
Authors:
Graham Kirby,
Alan Dearle,
Ron Morrison,
Mark Dunlop,
Richard Connor,
Paddy Nixon
Abstract:
Pervasive services may be defined as services that are available "to any client (anytime, anywhere)". Here we focus on the software and network infrastructure required to support pervasive contextual services operating over a wide area. One of the key requirements is a matching service capable of as-similating and filtering information from various sources and determining matches relevant to those…
▽ More
Pervasive services may be defined as services that are available "to any client (anytime, anywhere)". Here we focus on the software and network infrastructure required to support pervasive contextual services operating over a wide area. One of the key requirements is a matching service capable of as-similating and filtering information from various sources and determining matches relevant to those services. We consider some of the challenges in engineering a globally distributed matching service that is scalable, manageable, and able to evolve incrementally as usage patterns, data formats, services, network topologies and deployment technologies change. We outline an approach based on the use of a peer-to-peer architecture to distribute user events and data, and to support the deployment and evolution of the infrastructure itself.
△ Less
Submitted 24 June, 2010;
originally announced June 2010.
-
A Middleware Framework for Constraint-Based Deployment and Autonomic Management of Distributed Applications
Authors:
Alan Dearle,
Graham Kirby,
Andrew McCarthy
Abstract:
We propose a middleware framework for deployment and subsequent autonomic management of component-based distributed applications. An initial deployment goal is specified using a declarative constraint language, expressing constraints over aspects such as component-host mappings and component interconnection topology. A constraint solver is used to find a configuration that satisfies the goal, and…
▽ More
We propose a middleware framework for deployment and subsequent autonomic management of component-based distributed applications. An initial deployment goal is specified using a declarative constraint language, expressing constraints over aspects such as component-host mappings and component interconnection topology. A constraint solver is used to find a configuration that satisfies the goal, and the configuration is deployed automatically. The deployed application is instrumented to allow subsequent autonomic management. If, during execution, the manager detects that the original goal is no longer being met, the satisfy/deploy process can be repeated automatically in order to generate a revised deployment that does meet the goal.
△ Less
Submitted 24 June, 2010;
originally announced June 2010.
-
A Framework for Constraint-Based Deployment and Autonomic Management of Distributed Applications (Extended Abstract)
Authors:
Alan Dearle,
Graham Kirby,
Andrew McCarthy
Abstract:
We propose a framework for the deployment and subsequent autonomic management of component-based distributed applications. An initial deployment goal is specified using a declarative constraint language, expressing constraints over aspects such as component-host mappings and component interconnection topology. A constraint solver is used to find a configuration that satisfies the goal, and the con…
▽ More
We propose a framework for the deployment and subsequent autonomic management of component-based distributed applications. An initial deployment goal is specified using a declarative constraint language, expressing constraints over aspects such as component-host mappings and component interconnection topology. A constraint solver is used to find a configuration that satisfies the goal, and the configuration is deployed automatically. The deployed application is instrumented to allow subsequent autonomic management. If, during execution, the manager detects that the original goal is no longer being met, the satisfy/deploy process can be repeated automatically in order to generate a revised deployment that does meet the goal.
△ Less
Submitted 24 June, 2010;
originally announced June 2010.
-
A Framework for Constraint-Based Deployment and Autonomic Management of Distributed Applications
Authors:
Alan Dearle,
Graham Kirby,
Andrew McCarthy
Abstract:
We propose a framework for deployment and subsequent autonomic management of component-based distributed applications. An initial deployment goal is specified using a declarative constraint language, expressing constraints over aspects such as component-host mappings and component interconnection topology. A constraint solver is used to find a configuration that satisfies the goal, and the configu…
▽ More
We propose a framework for deployment and subsequent autonomic management of component-based distributed applications. An initial deployment goal is specified using a declarative constraint language, expressing constraints over aspects such as component-host mappings and component interconnection topology. A constraint solver is used to find a configuration that satisfies the goal, and the configuration is deployed automatically. The deployed application is instrumented to allow subsequent autonomic management. If, during execution, the manager detects that the original goal is no longer being met, the satisfy/deploy process can be repeated automatically in order to generate a revised deployment that does meet the goal.
△ Less
Submitted 23 June, 2010;
originally announced June 2010.
-
A Flexible and Secure Deployment Framework for Distributed Applications
Authors:
Alan Dearle,
Graham Kirby,
Andrew McCarthy,
Juan-Carlos Diaz y Carballo
Abstract:
This paper describes an implemented system which is designed to support the deployment of applications offering distributed services, comprising a number of distributed components. This is achieved by creating high level placement and topology descriptions which drive tools that deploy applications consisting of components running on multiple hosts. The system addresses issues of heterogeneity by…
▽ More
This paper describes an implemented system which is designed to support the deployment of applications offering distributed services, comprising a number of distributed components. This is achieved by creating high level placement and topology descriptions which drive tools that deploy applications consisting of components running on multiple hosts. The system addresses issues of heterogeneity by providing abstractions over host-specific attributes yielding a homogeneous run-time environment into which components may be deployed. The run-time environments provide secure binding mechanisms that permit deployed components to bind to stored data and services on the hosts on which they are running.
△ Less
Submitted 23 June, 2010;
originally announced June 2010.
-
Exposing Application Components as Web Services
Authors:
Scott Walker,
Alan Dearle,
Graham Kirby,
Stuart Norcross
Abstract:
This paper explores technology permitting arbitrary application components to be exposed for remote access from other software. Using this, the application and its constituent components can be written without concern for its distribution. Software running in different address spaces, on different machines, can perform operations on the remotely accessible components. This is of utility in the cre…
▽ More
This paper explores technology permitting arbitrary application components to be exposed for remote access from other software. Using this, the application and its constituent components can be written without concern for its distribution. Software running in different address spaces, on different machines, can perform operations on the remotely accessible components. This is of utility in the creation of distributed applications and in permitting tools such as debuggers, component browsers, observers or remote probes access to application components. Current middleware systems do not allow arbitrary exposure of application components: instead, the programmer is forced to decide statically which classes of component will support remote accessibility. In the work described here, arbitrary components of any class can be dynamically exposed via Web Services. Traditional Web Services are extended with a remote reference scheme. This extension permits application components to be invoked using either the traditional pass-by-value semantics supported by Web Services or pass-by-reference semantics. The latter permits the preservation of local call semantics across address space boundaries.
△ Less
Submitted 23 June, 2010;
originally announced June 2010.
-
RAFDA: Middleware Supporting the Separation of Application Logic from Distribution Policy
Authors:
Alan Dearle,
Scott Walker,
Stuart Norcross,
Graham Kirby,
Andrew McCarthy
Abstract:
Middleware technologies often limit the way in which object classes may be used in distributed applications due to the fixed distribution policies imposed by the Middleware system. These policies permeate the applications developed using them and force an unnatural encoding of application level semantics. For example, the application programmer has no direct control over inter-address-space parame…
▽ More
Middleware technologies often limit the way in which object classes may be used in distributed applications due to the fixed distribution policies imposed by the Middleware system. These policies permeate the applications developed using them and force an unnatural encoding of application level semantics. For example, the application programmer has no direct control over inter-address-space parameter passing semantics since it is fixed by the application's distribution topology which is dictated early in the design cycle by the Middleware. This creates applications that are brittle with respect to changes in the way in which the applications are distributed. This paper explores technology permitting arbitrary objects in an application to be dynamically exposed for remote access. Using this, the application can be written without concern for its distribution with object placement and distribution boundaries decided late in the design cycle and even dynamically. Inter-address-space parameter passing semantics may also be decided independently of object implementation and at varying times in the design cycle, again, possibly as late as run-time. Furthermore, transmission policy may be defined on a per-class, per-method or per-parameter basis maximizing plasticity. This flexibility is of utility in the development of new distributed applications and the creation of management and monitoring infrastructures for existing applications.
△ Less
Submitted 18 June, 2010;
originally announced June 2010.
-
Promoting Component Reuse by Separating Transmission Policy from Implementation
Authors:
Scott Walker,
Alan Dearle,
Graham Kirby,
Stuart Norcross
Abstract:
In this paper we present a methodology and set of tools which assist the construction of applications from components, by separating the issues of transmission policy from component definition and implementation. This promotes a greater degree of software reuse than is possible using traditional middleware environments. Whilst component technologies are usually presented as a mechanism for promoti…
▽ More
In this paper we present a methodology and set of tools which assist the construction of applications from components, by separating the issues of transmission policy from component definition and implementation. This promotes a greater degree of software reuse than is possible using traditional middleware environments. Whilst component technologies are usually presented as a mechanism for promoting reuse, reuse is often limited due to design choices that permeate component implementation. The programmer has no direct control over inter-address-space parameter passing semantics: it is fixed by the distributed application's structure, based on the remote accessibility of the components. Using traditional middleware tools and environments, the application designer may be forced to use an unnatural encoding of application level semantics since application parameter passing semantics are tightly coupled with the component deployment topology. This paper describes how inter-address-space parameter passing semantics may be decided independently of component implementation. Transmission policy may be dynamically defined on a per-class, per-method or per-parameter basis.
△ Less
Submitted 18 June, 2010;
originally announced June 2010.
-
Towards Adaptable and Adaptive Policy-Free Middleware
Authors:
Alan Dearle,
Graham Kirby,
Stuart Norcross,
Angus Macdonald,
Greg Bigwood
Abstract:
We believe that to fully support adaptive distributed applications, middleware must itself be adaptable, adaptive and policy-free. In this paper we present a new language-independent adaptable and adaptive policy framework suitable for integration in a wide variety of middleware systems. This framework facilitates the construction of adaptive distributed applications. The framework addresses adapt…
▽ More
We believe that to fully support adaptive distributed applications, middleware must itself be adaptable, adaptive and policy-free. In this paper we present a new language-independent adaptable and adaptive policy framework suitable for integration in a wide variety of middleware systems. This framework facilitates the construction of adaptive distributed applications. The framework addresses adaptability through its ability to represent a wide range of specific middleware policies. Adaptiveness is supported by a rich contextual model, through which an application programmer may control precisely how policies should be selected for any particular interaction with the middleware. A contextual pattern mechanism facilitates the succinct expression of both coarse- and fine-grain policy contexts. Policies may be specified and altered dynamically, and may themselves take account of dynamic conditions. The framework contains no hard-wired policies; instead, all policies can be configured.
△ Less
Submitted 18 June, 2010;
originally announced June 2010.
-
RAFDA: A Policy-Aware Middleware Supporting the Flexible Separation of Application Logic from Distribution
Authors:
Scott Walker,
Alan Dearle,
Stuart Norcross,
Graham Kirby,
Andrew McCarthy
Abstract:
Middleware technologies often limit the way in which object classes may be used in distributed applications due to the fixed distribution policies that they impose. These policies permeate applications developed using existing middleware systems and force an unnatural encoding of application level semantics. For example, the application programmer has no direct control over inter-address-space par…
▽ More
Middleware technologies often limit the way in which object classes may be used in distributed applications due to the fixed distribution policies that they impose. These policies permeate applications developed using existing middleware systems and force an unnatural encoding of application level semantics. For example, the application programmer has no direct control over inter-address-space parameter passing semantics. Semantics are fixed by the distribution topology of the application, which is dictated early in the design cycle. This creates applications that are brittle with respect to changes in distribution. This paper explores technology that provides control over the extent to which inter-address-space communication is exposed to programmers, in order to aid the creation, maintenance and evolution of distributed applications. The described system permits arbitrary objects in an application to be dynamically exposed for remote access, allowing applications to be written without concern for distribution. Programmers can conceal or expose the distributed nature of applications as required, permitting object placement and distribution boundaries to be decided late in the design cycle and even dynamically. Inter-address-space parameter passing semantics may also be decided independently of object implementation and at varying times in the design cycle, again possibly as late as run-time. Furthermore, transmission policy may be defined on a per-class, per-method or per-parameter basis, maximizing plasticity. This flexibility is of utility in the development of new distributed applications, and the creation of management and monitoring infrastructures for existing applications.
△ Less
Submitted 18 June, 2010;
originally announced June 2010.
-
A Peer-to-Peer Middleware Framework for Resilient Persistent Programming
Authors:
Alan Dearle,
Graham Kirby,
Stuart Norcross,
Andrew McCarthy
Abstract:
The persistent programming systems of the 1980s offered a programming model that integrated computation and long-term storage. In these systems, reliable applications could be engineered without requiring the programmer to write translation code to manage the transfer of data to and from non-volatile storage. More importantly, it simplified the programmer's conceptual model of an application, and…
▽ More
The persistent programming systems of the 1980s offered a programming model that integrated computation and long-term storage. In these systems, reliable applications could be engineered without requiring the programmer to write translation code to manage the transfer of data to and from non-volatile storage. More importantly, it simplified the programmer's conceptual model of an application, and avoided the many coherency problems that result from multiple cached copies of the same information. Although technically innovative, persistent languages were not widely adopted, perhaps due in part to their closed-world model. Each persistent store was located on a single host, and there were no flexible mechanisms for communication or transfer of data between separate stores. Here we re-open the work on persistence and combine it with modern peer-to-peer techniques in order to provide support for orthogonal persistence in resilient and potentially long-running distributed applications. Our vision is of an infrastructure within which an application can be developed and distributed with minimal modification, whereupon the application becomes resilient to certain failure modes. If a node, or the connection to it, fails during execution of the application, the objects are re-instantiated from distributed replicas, without their reference holders being aware of the failure. Furthermore, we believe that this can be achieved within a spectrum of application programmer intervention, ranging from minimal to totally prescriptive, as desired. The same mechanisms encompass an orthogonally persistent programming model. We outline our approach to implementing this vision, and describe current progress.
△ Less
Submitted 18 June, 2010;
originally announced June 2010.
-
Hosting Byzantine Fault Tolerant Services on a Chord Ring
Authors:
Alan Dearle,
Graham Kirby,
Stuart Norcross
Abstract:
In this paper we demonstrate how stateful Byzantine Fault Tolerant services may be hosted on a Chord ring. The strategy presented is fourfold: firstly a replication scheme that dissociates the maintenance of replicated service state from ring recovery is developed. Secondly, clients of the ring based services are made replication aware. Thirdly, a consensus protocol is introduced that supports the…
▽ More
In this paper we demonstrate how stateful Byzantine Fault Tolerant services may be hosted on a Chord ring. The strategy presented is fourfold: firstly a replication scheme that dissociates the maintenance of replicated service state from ring recovery is developed. Secondly, clients of the ring based services are made replication aware. Thirdly, a consensus protocol is introduced that supports the serialization of updates. Finally Byzantine fault tolerant replication protocols are developed that ensure the integrity of service data hosted on the ring.
△ Less
Submitted 17 June, 2010;
originally announced June 2010.
-
Applying Constraint Solving to the Management of Distributed Applications
Authors:
Andrew McCarthy,
Alan Dearle,
Graham Kirby
Abstract:
We present our approach for deploying and managing distributed component-based applications. A Desired State Description (DSD), written in a high-level declarative language, specifies requirements for a distributed application. Our infrastructure accepts a DSD as input, and from it automatically configures and deploys the distributed application. Subsequent violations of the original requirements…
▽ More
We present our approach for deploying and managing distributed component-based applications. A Desired State Description (DSD), written in a high-level declarative language, specifies requirements for a distributed application. Our infrastructure accepts a DSD as input, and from it automatically configures and deploys the distributed application. Subsequent violations of the original requirements are detected and, where possible, automatically rectified by reconfiguration and redeployment of the necessary application components. A constraint solving tool is used to plan deployments that meet the application requirements.
△ Less
Submitted 17 June, 2010;
originally announced June 2010.
-
Generating a Family of Byzantine Tolerant Protocol Implementations Using a Meta-Model Architecture
Authors:
Graham Kirby,
Alan Dearle,
Stuart Norcross
Abstract:
We describe an approach to modelling a Byzantine tolerant distributed algorithm as a family of related finite state machines, generated from a single meta-model. Various artefacts are generated from each state machine, including diagrams and source-level protocol implementations. The approach allows a state machine formulation to be applied to problems for which it would not otherwise be suitable,…
▽ More
We describe an approach to modelling a Byzantine tolerant distributed algorithm as a family of related finite state machines, generated from a single meta-model. Various artefacts are generated from each state machine, including diagrams and source-level protocol implementations. The approach allows a state machine formulation to be applied to problems for which it would not otherwise be suitable, increasing confidence in correctness.
△ Less
Submitted 17 June, 2010;
originally announced June 2010.
-
Orthogonal Persistence Revisited
Authors:
Alan Dearle,
Graham Kirby,
Ron Morrison
Abstract:
The social and economic importance of large bodies of programs and data that are potentially long-lived has attracted much attention in the commercial and research communities. Here we concentrate on a set of methodologies and technologies called persistent programming. In particular we review programming language support for the concept of orthogonal persistence, a technique for the uniform treat…
▽ More
The social and economic importance of large bodies of programs and data that are potentially long-lived has attracted much attention in the commercial and research communities. Here we concentrate on a set of methodologies and technologies called persistent programming. In particular we review programming language support for the concept of orthogonal persistence, a technique for the uniform treatment of objects irrespective of their types or longevity. While research in persistent programming has become unfashionable, we show how the concept is beginning to appear as a major component of modern systems. We relate these attempts to the original principles of orthogonal persistence and give a few hints about how the concept may be utilised in the future.
△ Less
Submitted 17 June, 2010;
originally announced June 2010.
-
H2O: An Autonomic, Resource-Aware Distributed Database System
Authors:
Angus Macdonald,
Alan Dearle,
Graham Kirby
Abstract:
This paper presents the design of an autonomic, resource-aware distributed database which enables data to be backed up and shared without complex manual administration. The database, H2O, is designed to make use of unused resources on workstation machines. Creating and maintaining highly-available, replicated database systems can be difficult for untrained users, and costly for IT departments. H2O…
▽ More
This paper presents the design of an autonomic, resource-aware distributed database which enables data to be backed up and shared without complex manual administration. The database, H2O, is designed to make use of unused resources on workstation machines. Creating and maintaining highly-available, replicated database systems can be difficult for untrained users, and costly for IT departments. H2O reduces the need for manual administration by autonomically replicating data and load-balancing across machines in an enterprise. Provisioning hardware to run a database system can be unnecessarily costly as most organizations already possess large quantities of idle resources in workstation machines. H2O is designed to utilize this unused capacity by using resource availability information to place data and plan queries over workstation machines that are already being used for other tasks. This paper discusses the requirements for such a system and presents the design and implementation of H2O.
△ Less
Submitted 16 June, 2010;
originally announced June 2010.
-
Autonomic Management of Maintenance Scheduling in Chord
Authors:
Markus Tauber,
Graham Kirby,
Alan Dearle
Abstract:
This paper experimentally evaluates the effects of applying autonomic management to the scheduling of maintenance operations in a deployed Chord network, for various membership churn and workload patterns. Two versions of an autonomic management policy were compared with a static configuration. The autonomic policies varied with respect to the aggressiveness with which they responded to peer acces…
▽ More
This paper experimentally evaluates the effects of applying autonomic management to the scheduling of maintenance operations in a deployed Chord network, for various membership churn and workload patterns. Two versions of an autonomic management policy were compared with a static configuration. The autonomic policies varied with respect to the aggressiveness with which they responded to peer access error rates and to wasted maintenance operations. In most experiments, significant improvements due to autonomic management were observed in the performance of routing operations and the quantity of data transmitted between network members. Of the autonomic policies, the more aggressive version gave slightly better results.
△ Less
Submitted 8 June, 2010;
originally announced June 2010.
-
An Approach to Ad hoc Cloud Computing
Authors:
Graham Kirby,
Alan Dearle,
Angus Macdonald,
Alvaro Fernandes
Abstract:
We consider how underused computing resources within an enterprise may be harnessed to improve utilization and create an elastic computing infrastructure. Most current cloud provision involves a data center model, in which clusters of machines are dedicated to running cloud infrastructure software. We propose an additional model, the ad hoc cloud, in which infrastructure software is distributed…
▽ More
We consider how underused computing resources within an enterprise may be harnessed to improve utilization and create an elastic computing infrastructure. Most current cloud provision involves a data center model, in which clusters of machines are dedicated to running cloud infrastructure software. We propose an additional model, the ad hoc cloud, in which infrastructure software is distributed over resources harvested from machines already in existence within an enterprise. In contrast to the data center cloud model, resource levels are not established a priori, nor are resources dedicated exclusively to the cloud while in use. A participating machine is not dedicated to the cloud, but has some other primary purpose such as running interactive processes for a particular user. We outline the major implementation challenges and one approach to tackling them.
△ Less
Submitted 25 February, 2010;
originally announced February 2010.
