A dynamic connectome supports the emergence of stable computational function of neural circuits through reward-based learning

David Kappel, Robert Legenstein, Stefan Habenschuss, Michael Hsieh, Wolfgang Maass

Research output: Contribution to journalArticleResearchpeer-review

Abstract

Synaptic connections between neurons in the brain are dynamic because of continuously ongoing spine dynamics, axonal sprouting, and other processes. In fact, it was recently shown that the spontaneous synapseautonomous component of spine dynamics is at least as large as the component that depends on the history of pre- and postsynaptic neural activity. These data are inconsistent with common models for network plasticity and raise the following questions: how can neural circuits maintain a stable computational function in spite of these continuously ongoing processes, and what could be functional uses of these ongoing processes? Here, we present a rigorous theoretical framework for these seemingly stochastic spine dynamics and rewiring processes in the context of reward-based learning tasks. We show that spontaneous synapse-autonomous processes, in combination with reward signals such as dopamine, can explain the capability of networks of neurons in the brain to configure themselves for specific computational tasks, and to compensate automatically for later changes in the network or task. Furthermore, we show theoretically and through computer simulations that stable computational performance is compatible with continuously ongoing synapse-autonomous changes. After reaching good computational performance it causes primarily a slow drift of network architecture and dynamics in task-irrelevant dimensions, as observed for neural activity in motor cortex and other areas. On the more abstract level of reinforcement learning the resulting model gives rise to an understanding of reward-driven network plasticity as continuous sampling of network configurations.

Original languageEnglish
Article numbere0301-17.2018
Number of pages27
JournaleNeuro
Volume5
Issue number2
DOIs
Publication statusPublished - 1 Mar 2018

Fingerprint

Connectome
Reward
Spine
Learning
Synapses
Neurons
Motor Cortex
Brain
Computer Simulation
Dopamine
History

Keywords

  • Reward-modulated STDP
  • Spine dynamics
  • Stochastic synaptic plasticity
  • Synapse-autonomous processes
  • Synaptic rewiring
  • Task-irrelevant dimensions in motor control

ASJC Scopus subject areas

  • Neuroscience(all)

Cite this

A dynamic connectome supports the emergence of stable computational function of neural circuits through reward-based learning. / Kappel, David; Legenstein, Robert; Habenschuss, Stefan; Hsieh, Michael; Maass, Wolfgang.

In: eNeuro, Vol. 5, No. 2, e0301-17.2018, 01.03.2018.

Research output: Contribution to journalArticleResearchpeer-review

@article{a6f17d7b142149a2822ca656152bad52,
title = "A dynamic connectome supports the emergence of stable computational function of neural circuits through reward-based learning",
abstract = "Synaptic connections between neurons in the brain are dynamic because of continuously ongoing spine dynamics, axonal sprouting, and other processes. In fact, it was recently shown that the spontaneous synapseautonomous component of spine dynamics is at least as large as the component that depends on the history of pre- and postsynaptic neural activity. These data are inconsistent with common models for network plasticity and raise the following questions: how can neural circuits maintain a stable computational function in spite of these continuously ongoing processes, and what could be functional uses of these ongoing processes? Here, we present a rigorous theoretical framework for these seemingly stochastic spine dynamics and rewiring processes in the context of reward-based learning tasks. We show that spontaneous synapse-autonomous processes, in combination with reward signals such as dopamine, can explain the capability of networks of neurons in the brain to configure themselves for specific computational tasks, and to compensate automatically for later changes in the network or task. Furthermore, we show theoretically and through computer simulations that stable computational performance is compatible with continuously ongoing synapse-autonomous changes. After reaching good computational performance it causes primarily a slow drift of network architecture and dynamics in task-irrelevant dimensions, as observed for neural activity in motor cortex and other areas. On the more abstract level of reinforcement learning the resulting model gives rise to an understanding of reward-driven network plasticity as continuous sampling of network configurations.",
keywords = "Reward-modulated STDP, Spine dynamics, Stochastic synaptic plasticity, Synapse-autonomous processes, Synaptic rewiring, Task-irrelevant dimensions in motor control",
author = "David Kappel and Robert Legenstein and Stefan Habenschuss and Michael Hsieh and Wolfgang Maass",
year = "2018",
month = "3",
day = "1",
doi = "10.1523/ENEURO.0301-17.2018",
language = "English",
volume = "5",
journal = "eNeuro",
issn = "2373-2822",
publisher = "Society for Neuroscience",
number = "2",

}

TY - JOUR

T1 - A dynamic connectome supports the emergence of stable computational function of neural circuits through reward-based learning

AU - Kappel, David

AU - Legenstein, Robert

AU - Habenschuss, Stefan

AU - Hsieh, Michael

AU - Maass, Wolfgang

PY - 2018/3/1

Y1 - 2018/3/1

N2 - Synaptic connections between neurons in the brain are dynamic because of continuously ongoing spine dynamics, axonal sprouting, and other processes. In fact, it was recently shown that the spontaneous synapseautonomous component of spine dynamics is at least as large as the component that depends on the history of pre- and postsynaptic neural activity. These data are inconsistent with common models for network plasticity and raise the following questions: how can neural circuits maintain a stable computational function in spite of these continuously ongoing processes, and what could be functional uses of these ongoing processes? Here, we present a rigorous theoretical framework for these seemingly stochastic spine dynamics and rewiring processes in the context of reward-based learning tasks. We show that spontaneous synapse-autonomous processes, in combination with reward signals such as dopamine, can explain the capability of networks of neurons in the brain to configure themselves for specific computational tasks, and to compensate automatically for later changes in the network or task. Furthermore, we show theoretically and through computer simulations that stable computational performance is compatible with continuously ongoing synapse-autonomous changes. After reaching good computational performance it causes primarily a slow drift of network architecture and dynamics in task-irrelevant dimensions, as observed for neural activity in motor cortex and other areas. On the more abstract level of reinforcement learning the resulting model gives rise to an understanding of reward-driven network plasticity as continuous sampling of network configurations.

AB - Synaptic connections between neurons in the brain are dynamic because of continuously ongoing spine dynamics, axonal sprouting, and other processes. In fact, it was recently shown that the spontaneous synapseautonomous component of spine dynamics is at least as large as the component that depends on the history of pre- and postsynaptic neural activity. These data are inconsistent with common models for network plasticity and raise the following questions: how can neural circuits maintain a stable computational function in spite of these continuously ongoing processes, and what could be functional uses of these ongoing processes? Here, we present a rigorous theoretical framework for these seemingly stochastic spine dynamics and rewiring processes in the context of reward-based learning tasks. We show that spontaneous synapse-autonomous processes, in combination with reward signals such as dopamine, can explain the capability of networks of neurons in the brain to configure themselves for specific computational tasks, and to compensate automatically for later changes in the network or task. Furthermore, we show theoretically and through computer simulations that stable computational performance is compatible with continuously ongoing synapse-autonomous changes. After reaching good computational performance it causes primarily a slow drift of network architecture and dynamics in task-irrelevant dimensions, as observed for neural activity in motor cortex and other areas. On the more abstract level of reinforcement learning the resulting model gives rise to an understanding of reward-driven network plasticity as continuous sampling of network configurations.

KW - Reward-modulated STDP

KW - Spine dynamics

KW - Stochastic synaptic plasticity

KW - Synapse-autonomous processes

KW - Synaptic rewiring

KW - Task-irrelevant dimensions in motor control

UR - http://www.scopus.com/inward/record.url?scp=85046435025&partnerID=8YFLogxK

U2 - 10.1523/ENEURO.0301-17.2018

DO - 10.1523/ENEURO.0301-17.2018

M3 - Article

VL - 5

JO - eNeuro

JF - eNeuro

SN - 2373-2822

IS - 2

M1 - e0301-17.2018

ER -