A dynamic connectome supports the emergence of stable computational function of neural circuits through reward-based learning

David Kappel, Robert Legenstein, Stefan Habenschuss, Michael Hsieh, Wolfgang Maass

Publikation: Beitrag in einer FachzeitschriftArtikelForschungBegutachtung

Abstract

Synaptic connections between neurons in the brain are dynamic because of continuously ongoing spine dynamics, axonal sprouting, and other processes. In fact, it was recently shown that the spontaneous synapseautonomous component of spine dynamics is at least as large as the component that depends on the history of pre- and postsynaptic neural activity. These data are inconsistent with common models for network plasticity and raise the following questions: how can neural circuits maintain a stable computational function in spite of these continuously ongoing processes, and what could be functional uses of these ongoing processes? Here, we present a rigorous theoretical framework for these seemingly stochastic spine dynamics and rewiring processes in the context of reward-based learning tasks. We show that spontaneous synapse-autonomous processes, in combination with reward signals such as dopamine, can explain the capability of networks of neurons in the brain to configure themselves for specific computational tasks, and to compensate automatically for later changes in the network or task. Furthermore, we show theoretically and through computer simulations that stable computational performance is compatible with continuously ongoing synapse-autonomous changes. After reaching good computational performance it causes primarily a slow drift of network architecture and dynamics in task-irrelevant dimensions, as observed for neural activity in motor cortex and other areas. On the more abstract level of reinforcement learning the resulting model gives rise to an understanding of reward-driven network plasticity as continuous sampling of network configurations.

Originalspracheenglisch
Aufsatznummere0301-17.2018
Seitenumfang27
FachzeitschrifteNeuro
Jahrgang5
Ausgabenummer2
DOIs
PublikationsstatusVeröffentlicht - 1 Mär 2018

Fingerprint

Connectome
Reward
Spine
Learning
Synapses
Neurons
Motor Cortex
Brain
Computer Simulation
Dopamine
History

Schlagwörter

    ASJC Scopus subject areas

    • !!Neuroscience(all)

    Dies zitieren

    A dynamic connectome supports the emergence of stable computational function of neural circuits through reward-based learning. / Kappel, David; Legenstein, Robert; Habenschuss, Stefan; Hsieh, Michael; Maass, Wolfgang.

    in: eNeuro, Jahrgang 5, Nr. 2, e0301-17.2018, 01.03.2018.

    Publikation: Beitrag in einer FachzeitschriftArtikelForschungBegutachtung

    @article{a6f17d7b142149a2822ca656152bad52,
    title = "A dynamic connectome supports the emergence of stable computational function of neural circuits through reward-based learning",
    abstract = "Synaptic connections between neurons in the brain are dynamic because of continuously ongoing spine dynamics, axonal sprouting, and other processes. In fact, it was recently shown that the spontaneous synapseautonomous component of spine dynamics is at least as large as the component that depends on the history of pre- and postsynaptic neural activity. These data are inconsistent with common models for network plasticity and raise the following questions: how can neural circuits maintain a stable computational function in spite of these continuously ongoing processes, and what could be functional uses of these ongoing processes? Here, we present a rigorous theoretical framework for these seemingly stochastic spine dynamics and rewiring processes in the context of reward-based learning tasks. We show that spontaneous synapse-autonomous processes, in combination with reward signals such as dopamine, can explain the capability of networks of neurons in the brain to configure themselves for specific computational tasks, and to compensate automatically for later changes in the network or task. Furthermore, we show theoretically and through computer simulations that stable computational performance is compatible with continuously ongoing synapse-autonomous changes. After reaching good computational performance it causes primarily a slow drift of network architecture and dynamics in task-irrelevant dimensions, as observed for neural activity in motor cortex and other areas. On the more abstract level of reinforcement learning the resulting model gives rise to an understanding of reward-driven network plasticity as continuous sampling of network configurations.",
    keywords = "Reward-modulated STDP, Spine dynamics, Stochastic synaptic plasticity, Synapse-autonomous processes, Synaptic rewiring, Task-irrelevant dimensions in motor control",
    author = "David Kappel and Robert Legenstein and Stefan Habenschuss and Michael Hsieh and Wolfgang Maass",
    year = "2018",
    month = "3",
    day = "1",
    doi = "10.1523/ENEURO.0301-17.2018",
    language = "English",
    volume = "5",
    journal = "eNeuro",
    issn = "2373-2822",
    publisher = "Society for Neuroscience",
    number = "2",

    }

    TY - JOUR

    T1 - A dynamic connectome supports the emergence of stable computational function of neural circuits through reward-based learning

    AU - Kappel, David

    AU - Legenstein, Robert

    AU - Habenschuss, Stefan

    AU - Hsieh, Michael

    AU - Maass, Wolfgang

    PY - 2018/3/1

    Y1 - 2018/3/1

    N2 - Synaptic connections between neurons in the brain are dynamic because of continuously ongoing spine dynamics, axonal sprouting, and other processes. In fact, it was recently shown that the spontaneous synapseautonomous component of spine dynamics is at least as large as the component that depends on the history of pre- and postsynaptic neural activity. These data are inconsistent with common models for network plasticity and raise the following questions: how can neural circuits maintain a stable computational function in spite of these continuously ongoing processes, and what could be functional uses of these ongoing processes? Here, we present a rigorous theoretical framework for these seemingly stochastic spine dynamics and rewiring processes in the context of reward-based learning tasks. We show that spontaneous synapse-autonomous processes, in combination with reward signals such as dopamine, can explain the capability of networks of neurons in the brain to configure themselves for specific computational tasks, and to compensate automatically for later changes in the network or task. Furthermore, we show theoretically and through computer simulations that stable computational performance is compatible with continuously ongoing synapse-autonomous changes. After reaching good computational performance it causes primarily a slow drift of network architecture and dynamics in task-irrelevant dimensions, as observed for neural activity in motor cortex and other areas. On the more abstract level of reinforcement learning the resulting model gives rise to an understanding of reward-driven network plasticity as continuous sampling of network configurations.

    AB - Synaptic connections between neurons in the brain are dynamic because of continuously ongoing spine dynamics, axonal sprouting, and other processes. In fact, it was recently shown that the spontaneous synapseautonomous component of spine dynamics is at least as large as the component that depends on the history of pre- and postsynaptic neural activity. These data are inconsistent with common models for network plasticity and raise the following questions: how can neural circuits maintain a stable computational function in spite of these continuously ongoing processes, and what could be functional uses of these ongoing processes? Here, we present a rigorous theoretical framework for these seemingly stochastic spine dynamics and rewiring processes in the context of reward-based learning tasks. We show that spontaneous synapse-autonomous processes, in combination with reward signals such as dopamine, can explain the capability of networks of neurons in the brain to configure themselves for specific computational tasks, and to compensate automatically for later changes in the network or task. Furthermore, we show theoretically and through computer simulations that stable computational performance is compatible with continuously ongoing synapse-autonomous changes. After reaching good computational performance it causes primarily a slow drift of network architecture and dynamics in task-irrelevant dimensions, as observed for neural activity in motor cortex and other areas. On the more abstract level of reinforcement learning the resulting model gives rise to an understanding of reward-driven network plasticity as continuous sampling of network configurations.

    KW - Reward-modulated STDP

    KW - Spine dynamics

    KW - Stochastic synaptic plasticity

    KW - Synapse-autonomous processes

    KW - Synaptic rewiring

    KW - Task-irrelevant dimensions in motor control

    UR - http://www.scopus.com/inward/record.url?scp=85046435025&partnerID=8YFLogxK

    U2 - 10.1523/ENEURO.0301-17.2018

    DO - 10.1523/ENEURO.0301-17.2018

    M3 - Article

    VL - 5

    JO - eNeuro

    JF - eNeuro

    SN - 2373-2822

    IS - 2

    M1 - e0301-17.2018

    ER -