• Media type: Doctoral Thesis; Electronic Thesis; E-Book
  • Title: Markov Processes Beyond Equilibrium and Optimal Control ; Theory, Applications, and Examples ; Markovprozesse im Nichtgleichgewicht und Optimalsteuerung ; Theorie, Anwendungen und Beispiele
  • Contributor: Banisch, Ralf [Author]
  • Published: Freie Universität Berlin: Refubium (FU Berlin), 2015
  • Extent: XV, 157 S.
  • Language: English
  • DOI: https://doi.org/10.17169/refubium-16968
  • Keywords: Cycle Decompositions ; Optimal Control ; Transition Path Theory ; Irreversible Markov Process ; Metastability
  • Origination:
  • Footnote: Diese Datenquelle enthält auch Bestandsnachweise, die nicht zu einem Volltext führen.
  • Description: This thesis is concerned with the long-term dynamics of irreversible Markov processes in discrete and continuous state spaces. In the first part, we study how the long-term dynamics of a reversible Markov process changes if an external force that destroys detailed balance is added. We derive an intuitive and general comparison result in terms of commuting times which indicates that under certain constraints the external driving force will always accelerate the long-term dynamics. We argue that non-trivial cycles in the probability flow are the key feature of irreversible processes and explain two ways of obtaining cycle decompositions in detail. We study how cycles can be used to construct reversible surrogates of irreversible processes that represent the long-term dynamics more faithfully than simple symmetrization, and apply this to the problem of module detection in directed networks. %This problem is equivalent to the problem of finding metastable sets for Markov processes on discrete state spaces. As a second application, we consider the problem of computing transition pathways between metastable states. This is done by considering the current of reactive trajectories which is computed by Transition Path Theory. We show that this current has cycles if the dynamics is irreversible, and compare two possible Hodge-Helmholtz like splittings of the current into simpler parts. One method is based on a projection, the second is based on cycle decompositions. We show that the second method allows for a computation of the statistics of transition pathways. In the second part, we study optimal control problems that arise if the external force can be adjusted by a controller who wants to minimize a certain objective function. We focus on linear quadratic (LQ) control problems and show that they are dual to sampling problems which appear e.g. in Molecular Dynamics. A numerical method to approximately solve LQ control problems is derived. The method uses a logarithmic transformation together with a Galerkin projection ...
  • Access State: Open Access