CaMKII activation supports reward-based neural network optimization through Hamiltonian sampling

Zhaofei Yu, David Kappel, Robert Legenstein, Sen Song, Feng Chen, Wolfgang Maass

Research output: Contribution to journalArticleResearchpeer-review

Abstract

Experimental data show that synaptic connections are subject to stochastic processes, and that neural codes drift on larger time scales. These data suggest to consider besides maximum likelihood learning also sampling models for network plasticity (synaptic sampling), where the current network connectivity and parameter values are viewed as a sample from a Markov chain, whose stationary distribution captures the invariant properties of network plasticity. However convergence to this stationary distribution may be rather slow if synaptic sampling carries out Langevin sampling. We show here that data on the molecular basis of synaptic plasticity, specifically on the role of CaMKII in its activated form, support a substantially more efficient Hamiltonian sampling of network configurations. We apply this new conceptual and mathematical framework to the analysis of rewardgated network plasticity, and show in a concrete example based on experimental data that Hamiltonian sampling speeds up the convergence to well-functioning network configurations. We also show that a regulation of the temperature of the sampling process provides a link between reinforcement learning and global network optimization through simulated annealing.
Original languageEnglish
JournalarXiv.org e-Print archive
Publication statusPublished - 1 Jun 2016

Fingerprint

Network Optimization
Reward
Activation
Neural Networks
Plasticity
Stationary Distribution
Experimental Data
Configuration
Network Connectivity
Reinforcement Learning
Simulated Annealing
Global Optimization
Maximum Likelihood
Stochastic Processes
Markov chain
Time Scales
Speedup
Invariant

Keywords

  • cs.NE
  • q-bio.NC

Cite this

CaMKII activation supports reward-based neural network optimization through Hamiltonian sampling. / Yu, Zhaofei; Kappel, David; Legenstein, Robert; Song, Sen; Chen, Feng; Maass, Wolfgang.

In: arXiv.org e-Print archive, 01.06.2016.

Research output: Contribution to journalArticleResearchpeer-review

@article{4204bfc23c954a42ba0aabd1c56dd760,
title = "CaMKII activation supports reward-based neural network optimization through Hamiltonian sampling",
abstract = "Experimental data show that synaptic connections are subject to stochastic processes, and that neural codes drift on larger time scales. These data suggest to consider besides maximum likelihood learning also sampling models for network plasticity (synaptic sampling), where the current network connectivity and parameter values are viewed as a sample from a Markov chain, whose stationary distribution captures the invariant properties of network plasticity. However convergence to this stationary distribution may be rather slow if synaptic sampling carries out Langevin sampling. We show here that data on the molecular basis of synaptic plasticity, specifically on the role of CaMKII in its activated form, support a substantially more efficient Hamiltonian sampling of network configurations. We apply this new conceptual and mathematical framework to the analysis of rewardgated network plasticity, and show in a concrete example based on experimental data that Hamiltonian sampling speeds up the convergence to well-functioning network configurations. We also show that a regulation of the temperature of the sampling process provides a link between reinforcement learning and global network optimization through simulated annealing.",
keywords = "cs.NE, q-bio.NC",
author = "Zhaofei Yu and David Kappel and Robert Legenstein and Sen Song and Feng Chen and Wolfgang Maass",
note = "10pages,1 figure",
year = "2016",
month = "6",
day = "1",
language = "English",
journal = "arXiv.org e-Print archive",
publisher = "Cornell University Library",

}

TY - JOUR

T1 - CaMKII activation supports reward-based neural network optimization through Hamiltonian sampling

AU - Yu, Zhaofei

AU - Kappel, David

AU - Legenstein, Robert

AU - Song, Sen

AU - Chen, Feng

AU - Maass, Wolfgang

N1 - 10pages,1 figure

PY - 2016/6/1

Y1 - 2016/6/1

N2 - Experimental data show that synaptic connections are subject to stochastic processes, and that neural codes drift on larger time scales. These data suggest to consider besides maximum likelihood learning also sampling models for network plasticity (synaptic sampling), where the current network connectivity and parameter values are viewed as a sample from a Markov chain, whose stationary distribution captures the invariant properties of network plasticity. However convergence to this stationary distribution may be rather slow if synaptic sampling carries out Langevin sampling. We show here that data on the molecular basis of synaptic plasticity, specifically on the role of CaMKII in its activated form, support a substantially more efficient Hamiltonian sampling of network configurations. We apply this new conceptual and mathematical framework to the analysis of rewardgated network plasticity, and show in a concrete example based on experimental data that Hamiltonian sampling speeds up the convergence to well-functioning network configurations. We also show that a regulation of the temperature of the sampling process provides a link between reinforcement learning and global network optimization through simulated annealing.

AB - Experimental data show that synaptic connections are subject to stochastic processes, and that neural codes drift on larger time scales. These data suggest to consider besides maximum likelihood learning also sampling models for network plasticity (synaptic sampling), where the current network connectivity and parameter values are viewed as a sample from a Markov chain, whose stationary distribution captures the invariant properties of network plasticity. However convergence to this stationary distribution may be rather slow if synaptic sampling carries out Langevin sampling. We show here that data on the molecular basis of synaptic plasticity, specifically on the role of CaMKII in its activated form, support a substantially more efficient Hamiltonian sampling of network configurations. We apply this new conceptual and mathematical framework to the analysis of rewardgated network plasticity, and show in a concrete example based on experimental data that Hamiltonian sampling speeds up the convergence to well-functioning network configurations. We also show that a regulation of the temperature of the sampling process provides a link between reinforcement learning and global network optimization through simulated annealing.

KW - cs.NE

KW - q-bio.NC

M3 - Article

JO - arXiv.org e-Print archive

JF - arXiv.org e-Print archive

ER -