-
Genomic reproducibility in the bioinformatics era
Authors:
Pelin Icer Baykal,
Paweł P. Łabaj,
Florian Markowetz,
Lynn M. Schriml,
Daniel J. Stekhoven,
Serghei Mangul,
Niko Beerenwinkel
Abstract:
In biomedical research, validation of a new scientific discovery is tied to the reproducibility of its experimental results. However, in genomics, the definition and implementation of reproducibility still remain imprecise. Here, we argue that genomic reproducibility, defined as the ability of bioinformatics tools to maintain consistent genomics results across technical replicates, is key to gener…
▽ More
In biomedical research, validation of a new scientific discovery is tied to the reproducibility of its experimental results. However, in genomics, the definition and implementation of reproducibility still remain imprecise. Here, we argue that genomic reproducibility, defined as the ability of bioinformatics tools to maintain consistent genomics results across technical replicates, is key to generating scientific knowledge and enabling medical applications. We first discuss different concepts of reproducibility and then focus on reproducibility in the context of genomics, aiming to establish clear definitions of relevant terms. We then focus on the role of bioinformatics tools and their impact on genomic reproducibility and assess methods of evaluating bioinformatics tools in terms of genomic reproducibility. Lastly, we suggest best practices for enhancing genomic reproducibility, with an emphasis on assessing the performance of bioinformatics tools through rigorous testing across multiple technical replicates.
△ Less
Submitted 18 August, 2023;
originally announced August 2023.
-
Practical and scalable simulations of non-Markovian stochastic processes and temporal networks with individual node properties
Authors:
Aurelien Pelissier,
Miroslav Phan,
Didier Le Bail,
Niko Beerenwinkel,
Maria Rodriguez Martinez
Abstract:
Discrete stochastic processes are prevalent in natural systems, with applications in physics, biochemistry, epidemiology, sociology, and finance. While analytic solutions often cannot be derived, existing simulation frameworks can generate stochastic trajectories compatible with the dynamical laws underlying the random phenomena. Still, most simulation algorithms assume the system dynamics are mem…
▽ More
Discrete stochastic processes are prevalent in natural systems, with applications in physics, biochemistry, epidemiology, sociology, and finance. While analytic solutions often cannot be derived, existing simulation frameworks can generate stochastic trajectories compatible with the dynamical laws underlying the random phenomena. Still, most simulation algorithms assume the system dynamics are memoryless (Markovian assumption), under which assumption, future occurrences only depend on the system's present state. This enables efficient and exact simulation via the Gillespie algorithm. However, many real-world systems are inherently non-Markovian and exhibit memory effects. Such systems are difficult to study analytically, and current numerical methods are often computationally expensive or limited by strong simplifying assumptions that conflict with empirical data. To address these limitations, we introduce the Rejection-based Gillespie algorithm for non-Markovian Reactions (REGIR), a general and scalable framework for simulating non-Markovian stochastic systems with arbitrary inter-event time distributions. REGIR provides user-defined accuracy while preserving the same asymptotic computational complexity as the classical Gillespie algorithm. We derive a lower bound on REGIR's approximation accuracy and demonstrate its capabilities across three representative classes of non-Markovian systems: (1) reaction channels with delays, (2) stochastic processes driven by individual reactant properties, and (3) temporal networks governed by node activity. In all cases, REGIR accurately captures memory-dependent dynamics and outperforms existing approaches in terms of flexibility and efficiency.
△ Less
Submitted 1 April, 2025; v1 submitted 8 December, 2022;
originally announced December 2022.
-
Bayesian reconstruction of HIV transmission trees from viral sequences and uncertain infection times
Authors:
Hesam Montazeri,
Susan Little,
Niko Beerenwinkel,
Victor DeGruttola
Abstract:
Genetic sequence data of pathogens are increasingly used to investigate transmission dynamics in both endemic diseases and disease outbreaks; such research can aid in development of appropriate interventions and in design of studies to evaluate them. Several methods have been proposed to infer transmission chains from sequence data; however, existing methods do not generally reliably reconstruct t…
▽ More
Genetic sequence data of pathogens are increasingly used to investigate transmission dynamics in both endemic diseases and disease outbreaks; such research can aid in development of appropriate interventions and in design of studies to evaluate them. Several methods have been proposed to infer transmission chains from sequence data; however, existing methods do not generally reliably reconstruct transmission trees because genetic sequence data or inferred phylogenetic trees from such data are insufficient for accurate inference regarding transmission chains. In this paper, we demonstrate the lack of a one-to-one relationship between phylogenies and transmission trees, and also show that information regarding infection times together with genetic sequences permit accurate reconstruction of transmission trees. We propose a Bayesian inference method for this purpose and demonstrate that precision of inference regarding these transmission trees depends on precision of the estimated times of infection. We also illustrate the use of these methods to study features of epidemic dynamics, such as the relationship between characteristics of nodes and average number of outbound edges or inbound edges-- signifying possible transmission events from and to nodes. We study the performance of the proposed method in simulation experiments and demonstrate its superiority in comparison to an alternative method. We apply them to a transmission cluster in San Diego and investigate the impact of biological, behavioral, and demographic factors.
△ Less
Submitted 23 January, 2018;
originally announced January 2018.
-
Probabilistic modeling of occurring substitutions in PAR-CLIP data
Authors:
Monica Golumbeanu,
Pejman Mohammadi,
Niko Beerenwinkel
Abstract:
Photoactivatable ribonucleoside-enhanced cross-linking and immunoprecipitation (PAR-CLIP) is an experimental method based on next-generation sequencing for identifying the RNA interaction sites of a given protein. The method deliberately inserts T-to-C substitutions at the RNA-protein interaction sites, which provides a second layer of evidence compared to other CLIP methods. However, the experime…
▽ More
Photoactivatable ribonucleoside-enhanced cross-linking and immunoprecipitation (PAR-CLIP) is an experimental method based on next-generation sequencing for identifying the RNA interaction sites of a given protein. The method deliberately inserts T-to-C substitutions at the RNA-protein interaction sites, which provides a second layer of evidence compared to other CLIP methods. However, the experiment includes several sources of noise which cause both low-frequency errors and spurious high-frequency alterations. Therefore, rigorous statistical analysis is required in order to separate true T-to-C base changes, following cross-linking, from noise. So far, most of the existing PAR-CLIP data analysis methods focus on discarding the low-frequency errors and rely on high-frequency substitutions to report binding sites, not taking into account the possibility of high-frequency false positive substitutions. Here, we introduce BMix, a new probabilistic method which explicitly accounts for the sources of noise in PAR- CLIP data and distinguishes cross-link induced T-to-C substitutions from low and high-frequency erroneous alterations. We demonstrate the superior speed and accuracy of our method compared to existing approaches on both simulated and real, publicly available human datasets. The model is implemented in the Matlab toolbox BMix, freely available at www.cbg.bsse.ethz.ch/software/BMix.
△ Less
Submitted 7 April, 2015;
originally announced April 2015.
-
TiMEx: A Waiting Time Model for Mutually Exclusive Groups of Cancer Alterations
Authors:
Simona Constantinescu,
Ewa Szczurek,
Pejman Mohammadi,
Jörg Rahnenführer,
Niko Beerenwinkel
Abstract:
Despite recent technological advances in genomic sciences, our understanding of cancer progression and its driving genetic alterations remains incomplete. Here, we introduce TiMEx, a generative probabilistic model for detecting patterns of various degrees of mutual exclusivity across genetic alterations, which can indicate pathways involved in cancer progression. TiMEx explicitly accounts for the…
▽ More
Despite recent technological advances in genomic sciences, our understanding of cancer progression and its driving genetic alterations remains incomplete. Here, we introduce TiMEx, a generative probabilistic model for detecting patterns of various degrees of mutual exclusivity across genetic alterations, which can indicate pathways involved in cancer progression. TiMEx explicitly accounts for the temporal interplay between the waiting times to alterations and the observation time. In simulation studies, we show that our model outperforms previous methods for detecting mutual exclusivity. On large-scale biological datasets, TiMEx identifies gene groups with strong functional biological relevance, while also proposing many new candidates for biological validation. TiMEx possesses several advantages over previous methods, including a novel generative probabilistic model of tumorigenesis, direct estimation of the probability of mutual exclusivity interaction, computational efficiency, as well as high sensitivity in detecting gene groups involving low-frequency alterations. R code is available at www.cbg.bsse.ethz.ch/software/TiMEx.
△ Less
Submitted 27 October, 2015; v1 submitted 3 April, 2015;
originally announced April 2015.
-
Evolutionary Games with Affine Fitness Functions: Applications to Cancer
Authors:
Moritz Gerstung,
Hani Nakhoul,
Niko Beerenwinkel
Abstract:
We analyze the dynamics of evolutionary games in which fitness is defined as an affine function of the expected payoff and a constant contribution. The resulting inhomogeneous replicator equation has an homogeneous equivalent with modified payoffs. The affine terms also influence the stochastic dynamics of a two-strategy Moran model of a finite population. We then apply the affine fitness function…
▽ More
We analyze the dynamics of evolutionary games in which fitness is defined as an affine function of the expected payoff and a constant contribution. The resulting inhomogeneous replicator equation has an homogeneous equivalent with modified payoffs. The affine terms also influence the stochastic dynamics of a two-strategy Moran model of a finite population. We then apply the affine fitness function in a model for tumor-normal cell interactions to determine which are the most successful tumor strategies. In order to analyze the dynamics of concurrent strategies within a tumor population, we extend the model to a three-strategy game involving distinct tumor cell types as well as normal cells. In this model, interaction with normal cells, in combination with an increased constant fitness, is the most effective way of establishing a population of tumor cells in normal tissue.
△ Less
Submitted 30 August, 2011; v1 submitted 25 January, 2011;
originally announced January 2011.
-
Competition-colonization trade-off promotes coexistence of low-virulence viral strains
Authors:
Samuel Ojosnegros,
Edgar Delgado-Eckert,
Niko Beerenwinkel
Abstract:
RNA viruses exist as genetically diverse populations displaying different phenotypes, including diverse degrees of virulence. The evolution of virulence in viral populations is, however, poorly understood. Based on the experimental observation of an RNA virus clone in cell culture diversifying into two subpopulations of different virulence, we study the dynamics of heterogeneous virus populations…
▽ More
RNA viruses exist as genetically diverse populations displaying different phenotypes, including diverse degrees of virulence. The evolution of virulence in viral populations is, however, poorly understood. Based on the experimental observation of an RNA virus clone in cell culture diversifying into two subpopulations of different virulence, we study the dynamics of heterogeneous virus populations and the evolution of virulence. We introduce a competition-colonization trade-off into standard mathematical models of intra-host viral infection. Colonizers are fast spreading, virulent strains, whereas competitors are less virulent variants that are more successful within coinfected cells. We observe biphasic dynamics of the population: Early in the infection the population is dominated by colonizers, which later will be outcompeted by competitors. The simulations suggest the existence of a steady state where few low-virulence variants coexist. This equilibrium implies collective virulence attenuation in the population, in contrast to previous models predicting development of the population towards increased virulence. Nevertheless, the attenuation effect disappears if we include a highly simplified immune response in our models. Thus, the competition-colonization trade-off indicates a role for virulence in the modulation of the viral population diversity. The evolution of virulence is a dynamic feature of the population shaped by interactions between individuals and by the structure of the patchy habitat.
△ Less
Submitted 24 January, 2011; v1 submitted 16 January, 2011;
originally announced January 2011.
-
The evolution of virulence in RNA viruses under a competition-colonization trade-off
Authors:
Edgar Delgado-Eckert,
Samuel Ojosnegros,
Niko Beerenwinkel
Abstract:
RNA viruses exist in large intra-host populations which display great genotypic and phenotypic diversity. We analyze a model of viral competition between two different viral strains infecting a constantly replenished cell pool, in which we assume a trade-off between the virus' colonization skills (cell killing ability or virulence) and its local competition skills (replication performance within…
▽ More
RNA viruses exist in large intra-host populations which display great genotypic and phenotypic diversity. We analyze a model of viral competition between two different viral strains infecting a constantly replenished cell pool, in which we assume a trade-off between the virus' colonization skills (cell killing ability or virulence) and its local competition skills (replication performance within coinfected cells). We characterize the conditions that allow for viral spread by means of the basic reproductive number and show that a local coexistence equilibrium exists, which is asymptotically stable. At this equilibrium, the less virulent competitor has a reproductive advantage over the more virulent colonizer. The equilibria at which one strain outcompetes the other one are unstable, i.e., a second viral strain is always able to permanently invade. One generalization of the model is to consider multiple viral strains, each one displaying a different virulence. However, to account for the large phenotypic diversity in viral populations, we consider a continuous spectrum of virulences and present a continuum limit of this multiple viral strains model that describes the time evolution of an initial continuous distribution of virulence. We provide a proof of the existence of solutions of the model's equations and present numerical approximations of solutions for different initial distributions. Our simulations suggest that initial continuous distributions of virulence evolve towards a stationary distribution that is extremely skewed in favor of competitors. Consequently, collective virulence attenuation takes place. This finding may contribute to understanding the phenomenon of virulence attenuation, which has been reported in previous experimental studies.
△ Less
Submitted 18 January, 2010;
originally announced January 2010.
-
Multiple Sequence Alignment System for Pyrosequencing Reads
Authors:
Fahad Saeed,
Ashfaq Khokhar,
Osvaldo Zagordi,
Niko Beerenwinkel
Abstract:
Pyrosequencing is among the emerging sequencing techniques, capable of generating upto 100,000 overlapping reads in a single run. This technique is much faster and cheaper than the existing state of the art sequencing technique such as Sanger. However, the reads generated by pyrosequencing are short in size and contain numerous errors. Furthermore, each read has a specific position in the refere…
▽ More
Pyrosequencing is among the emerging sequencing techniques, capable of generating upto 100,000 overlapping reads in a single run. This technique is much faster and cheaper than the existing state of the art sequencing technique such as Sanger. However, the reads generated by pyrosequencing are short in size and contain numerous errors. Furthermore, each read has a specific position in the reference genome. In order to use these reads for any subsequent analysis, the reads must be aligned . Existing multiple sequence alignment methods cannot be used as they do not take into account the specific positions of the sequences with respect to the genome, and are highly inefficient for large number of sequences. Therefore, the common practice has been to use either simple pairwise alignment despite its poor accuracy for error prone pyroreads, or use computationally expensive techniques based on sequential gap propagation. In this paper, we develop a computationally efficient method based on domain decomposition, referred to as pyro-align, to align such large number of reads. The proposed alignment algorithm accurately aligns the erroneous reads in a short period of time, which is orders of magnitude faster than any existing method. The accuracy of the alignment is confirmed from the consensus obtained from the multiple alignments.
△ Less
Submitted 18 January, 2009;
originally announced January 2009.
-
Waiting time models of cancer progression
Authors:
Moritz Gerstung,
Niko Beerenwinkel
Abstract:
Cancer progression is an evolutionary process that is driven by mutation and selection in a population of tumor cells. We discuss mathematical models of cancer progression, starting from traditional multistage theory. Each stage is associated with the occurrence of genetic alterations and their fixation in the population. We describe the accumulation of mutations using conjunctive Bayesian netwo…
▽ More
Cancer progression is an evolutionary process that is driven by mutation and selection in a population of tumor cells. We discuss mathematical models of cancer progression, starting from traditional multistage theory. Each stage is associated with the occurrence of genetic alterations and their fixation in the population. We describe the accumulation of mutations using conjunctive Bayesian networks, an exponential family of waiting time models in which the occurrence of mutations is constrained to a partial temporal order. Two opposing limit cases arise if mutations either follow a linear order or occur independently. We derive exact analytical expressions for the waiting time until a specific number of mutations have accumulated in these limit cases as well as for the general conjunctive Bayesian network. Finally, we analyze a stochastic population genetics model that explicitly accounts for mutation and selection. In this model, waves of clonal expansions sweep through the population at equidistant intervals. We present an approximate analytical expression for the waiting time in this model and compare it to the results obtained for the conjunctive Bayesian networks.
△ Less
Submitted 3 February, 2009; v1 submitted 23 July, 2008;
originally announced July 2008.
-
Markov models for accumulating mutations
Authors:
Niko Beerenwinkel,
Seth Sullivant
Abstract:
We introduce and analyze a waiting time model for the accumulation of genetic changes. The continuous time conjunctive Bayesian network is defined by a partially ordered set of mutations and by the rate of fixation of each mutation. The partial order encodes constraints on the order in which mutations can fixate in the population, shedding light on the mutational pathways underlying the evolutio…
▽ More
We introduce and analyze a waiting time model for the accumulation of genetic changes. The continuous time conjunctive Bayesian network is defined by a partially ordered set of mutations and by the rate of fixation of each mutation. The partial order encodes constraints on the order in which mutations can fixate in the population, shedding light on the mutational pathways underlying the evolutionary process. We study a censored version of the model and derive equations for an EM algorithm to perform maximum likelihood estimation of the model parameters. We also show how to select the maximum likelihood poset. The model is applied to genetic data from different cancers and from drug resistant HIV samples, indicating implications for diagnosis and treatment.
△ Less
Submitted 17 September, 2007;
originally announced September 2007.
-
Genetic progression and the waiting time to cancer
Authors:
Niko Beerenwinkel,
Tibor Antal,
David Dingli,
Arne Traulsen,
Kenneth W. Kinzler,
Victor E. Velculescu,
Bert Vogelstein,
Martin A. Nowak
Abstract:
Cancer results from genetic alterations that disturb the normal cooperative behavior of cells. Recent high-throughput genomic studies of cancer cells have shown that the mutational landscape of cancer is complex and that individual cancers may evolve through mutations in as many as 20 different cancer-associated genes. We use data published by Sjoblom et al. (2006) to develop a new mathematical…
▽ More
Cancer results from genetic alterations that disturb the normal cooperative behavior of cells. Recent high-throughput genomic studies of cancer cells have shown that the mutational landscape of cancer is complex and that individual cancers may evolve through mutations in as many as 20 different cancer-associated genes. We use data published by Sjoblom et al. (2006) to develop a new mathematical model for the somatic evolution of colorectal cancers. We employ the Wright-Fisher process for exploring the basic parameters of this evolutionary process and derive an analytical approximation for the expected waiting time to the cancer phenotype. Our results highlight the relative importance of selection over both the size of the cell population at risk and the mutation rate. The model predicts that the observed genetic diversity of cancer genomes can arise under a normal mutation rate if the average selective advantage per mutation is on the order of 1%. Increased mutation rates due to genetic instability would allow even smaller selective advantages during tumorigenesis. The complexity of cancer progression thus can be understood as the result of multiple sequential mutations, each of which has a relatively small but positive effect on net cell growth.
△ Less
Submitted 25 July, 2007;
originally announced July 2007.
-
Viral population estimation using pyrosequencing
Authors:
Nicholas Eriksson,
Lior Pachter,
Yumi Mitsuya,
Soo-Yon Rhee,
Chunlin Wang,
Baback Gharizadeh,
Mostafa Ronaghi,
Robert W. Shafer,
Niko Beerenwinkel
Abstract:
The diversity of virus populations within single infected hosts presents a major difficulty for the natural immune response as well as for vaccine design and antiviral drug therapy. Recently developed pyrophosphate based sequencing technologies (pyrosequencing) can be used for quantifying this diversity by ultra-deep sequencing of virus samples. We present computational methods for the analysis…
▽ More
The diversity of virus populations within single infected hosts presents a major difficulty for the natural immune response as well as for vaccine design and antiviral drug therapy. Recently developed pyrophosphate based sequencing technologies (pyrosequencing) can be used for quantifying this diversity by ultra-deep sequencing of virus samples. We present computational methods for the analysis of such sequence data and apply these techniques to pyrosequencing data obtained from HIV populations within patients harboring drug resistant virus strains. Our main result is the estimation of the population structure of the sample from the pyrosequencing reads. This inference is based on a statistical approach to error correction, followed by a combinatorial algorithm for constructing a minimal set of haplotypes that explain the data. Using this set of explaining haplotypes, we apply a statistical model to infer the frequencies of the haplotypes in the population via an EM algorithm. We demonstrate that pyrosequencing reads allow for effective population reconstruction by extensive simulations and by comparison to 165 sequences obtained directly from clonal sequencing of four independent, diverse HIV populations. Thus, pyrosequencing can be used for cost-effective estimation of the structure of virus populations, promising new insights into viral evolutionary dynamics and disease control strategies.
△ Less
Submitted 21 January, 2008; v1 submitted 1 July, 2007;
originally announced July 2007.
-
Epistasis and Shapes of Fitness Landscapes
Authors:
Niko Beerenwinkel,
Lior Pachter,
Bernd Sturmfels
Abstract:
The relationship between the shape of a fitness landscape and the underlying gene interactions, or epistasis, has been extensively studied in the two-locus case. Gene interactions among multiple loci are usually reduced to two-way interactions. We present a geometric theory of shapes of fitness landscapes for multiple loci. A central concept is the genotope, which is the convex hull of all possi…
▽ More
The relationship between the shape of a fitness landscape and the underlying gene interactions, or epistasis, has been extensively studied in the two-locus case. Gene interactions among multiple loci are usually reduced to two-way interactions. We present a geometric theory of shapes of fitness landscapes for multiple loci. A central concept is the genotope, which is the convex hull of all possible allele frequencies in populations. Triangulations of the genotope correspond to different shapes of fitness landscapes and reveal all the gene interactions. The theory is applied to fitness data from HIV and Drosophila melanogaster. In both cases, our findings refine earlier analyses and reveal previously undetected gene interactions.
△ Less
Submitted 14 April, 2006; v1 submitted 29 March, 2006;
originally announced March 2006.
-
A Mutagenetic Tree Hidden Markov Model for Longitudinal Clonal HIV Sequence Data
Authors:
Niko Beerenwinkel,
Mathias Drton
Abstract:
RNA viruses provide prominent examples of measurably evolving populations. In HIV infection, the development of drug resistance is of particular interest, because precise predictions of the outcome of this evolutionary process are a prerequisite for the rational design of antiretroviral treatment protocols. We present a mutagenetic tree hidden Markov model for the analysis of longitudinal clonal…
▽ More
RNA viruses provide prominent examples of measurably evolving populations. In HIV infection, the development of drug resistance is of particular interest, because precise predictions of the outcome of this evolutionary process are a prerequisite for the rational design of antiretroviral treatment protocols. We present a mutagenetic tree hidden Markov model for the analysis of longitudinal clonal sequence data. Using HIV mutation data from clinical trials, we estimate the order and rate of occurrence of seven amino acid changes that are associated with resistance to the reverse transcriptase inhibitor efavirenz.
△ Less
Submitted 27 March, 2006;
originally announced March 2006.
-
Parametric inference of recombination in HIV genomes
Authors:
Niko Beerenwinkel,
Colin N. Dewey,
Kevin M. Woods
Abstract:
Recombination is an important event in the evolution of HIV. It affects the global spread of the pandemic as well as evolutionary escape from host immune response and from drug therapy within single patients. Comprehensive computational methods are needed for detecting recombinant sequences in large databases, and for inferring the parental sequences.
We present a hidden Markov model to annota…
▽ More
Recombination is an important event in the evolution of HIV. It affects the global spread of the pandemic as well as evolutionary escape from host immune response and from drug therapy within single patients. Comprehensive computational methods are needed for detecting recombinant sequences in large databases, and for inferring the parental sequences.
We present a hidden Markov model to annotate a query sequence as a recombinant of a given set of aligned sequences. Parametric inference is used to determine all optimal annotations for all parameters of the model. We show that the inferred annotations recover most features of established hand-curated annotations. Thus, parametric analysis of the hidden Markov model is feasible for HIV full-length genomes, and it improves the detection and annotation of recombinant forms.
All computational results, reference alignments, and C++ source code are available at http://bio.math.berkeley.edu/recombination/.
△ Less
Submitted 8 December, 2005;
originally announced December 2005.
-
Evolution on distributive lattices
Authors:
Niko Beerenwinkel,
Nicholas Eriksson,
Bernd Sturmfels
Abstract:
We consider the directed evolution of a population after an intervention that has significantly altered the underlying fitness landscape. We model the space of genotypes as a distributive lattice; the fitness landscape is a real-valued function on that lattice. The risk of escape from intervention, i.e., the probability that the population develops an escape mutant before extinction, is encoded…
▽ More
We consider the directed evolution of a population after an intervention that has significantly altered the underlying fitness landscape. We model the space of genotypes as a distributive lattice; the fitness landscape is a real-valued function on that lattice. The risk of escape from intervention, i.e., the probability that the population develops an escape mutant before extinction, is encoded in the risk polynomial. Tools from algebraic combinatorics are applied to compute the risk polynomial in terms of the fitness landscape. In an application to the development of drug resistance in HIV, we study the risk of viral escape from treatment with the protease inhibitors ritonavir and indinavir.
△ Less
Submitted 12 May, 2006; v1 submitted 23 November, 2005;
originally announced November 2005.