Emergent Inference of Hidden Markov Models in Spiking Neural Networks Through Winner-Take-All

Zhaofei Yu, Shangqi Guo, Fei Deng, Qi Yan, Keke Huang, Jian K. Liu, Feng Chen

Research output: Contribution to journalArticleResearchpeer-review

Abstract

Hidden Markov models (HMMs) underpin the solution to many problems in computational neuroscience. However, it is still unclear how to implement inference of HMMs with a network of neurons in the brain. The existing methods suffer from the problem of being nonspiking and inaccurate. Here, we build a precise equivalence between the inference equation of HMMs with time-invariant hidden variables and the dynamics of spiking winner-take-all (WTA) neural networks. We show that the membrane potential of each spiking neuron in the WTA circuit encodes the logarithm of the posterior probability of the hidden variable in each state, and the firing rate of each neuron is proportional to the posterior probability of the HMMs. We prove that the time course of the neural firing rate can implement posterior inference of HMMs. Theoretical analysis and experimental results show that the proposed WTA circuit can get accurate inference results of HMMs.

Original languageEnglish
JournalIEEE Transactions on Cybernetics
Early online date3 Oct 2018
DOIs
Publication statusE-pub ahead of print - 3 Oct 2018

Fingerprint

Hidden Markov models
Neural networks
Neurons
Networks (circuits)
Brain
Membranes

Keywords

  • Biological neural networks
  • Brain modeling
  • Cybernetics
  • Hidden Markov models
  • Hidden Markov models (HMMs)
  • Markov processes
  • Mathematical model
  • neural implementation
  • Neurons
  • posterior inference
  • spiking neural network
  • winner-take-all (WTA) circuits

ASJC Scopus subject areas

  • Software
  • Control and Systems Engineering
  • Information Systems
  • Human-Computer Interaction
  • Computer Science Applications
  • Electrical and Electronic Engineering

Cite this

Emergent Inference of Hidden Markov Models in Spiking Neural Networks Through Winner-Take-All. / Yu, Zhaofei; Guo, Shangqi; Deng, Fei; Yan, Qi; Huang, Keke; Liu, Jian K.; Chen, Feng.

In: IEEE Transactions on Cybernetics, 03.10.2018.

Research output: Contribution to journalArticleResearchpeer-review

Yu, Zhaofei ; Guo, Shangqi ; Deng, Fei ; Yan, Qi ; Huang, Keke ; Liu, Jian K. ; Chen, Feng. / Emergent Inference of Hidden Markov Models in Spiking Neural Networks Through Winner-Take-All. In: IEEE Transactions on Cybernetics. 2018.
@article{1a8770470f7145a682ed59856bf0c51b,
title = "Emergent Inference of Hidden Markov Models in Spiking Neural Networks Through Winner-Take-All",
abstract = "Hidden Markov models (HMMs) underpin the solution to many problems in computational neuroscience. However, it is still unclear how to implement inference of HMMs with a network of neurons in the brain. The existing methods suffer from the problem of being nonspiking and inaccurate. Here, we build a precise equivalence between the inference equation of HMMs with time-invariant hidden variables and the dynamics of spiking winner-take-all (WTA) neural networks. We show that the membrane potential of each spiking neuron in the WTA circuit encodes the logarithm of the posterior probability of the hidden variable in each state, and the firing rate of each neuron is proportional to the posterior probability of the HMMs. We prove that the time course of the neural firing rate can implement posterior inference of HMMs. Theoretical analysis and experimental results show that the proposed WTA circuit can get accurate inference results of HMMs.",
keywords = "Biological neural networks, Brain modeling, Cybernetics, Hidden Markov models, Hidden Markov models (HMMs), Markov processes, Mathematical model, neural implementation, Neurons, posterior inference, spiking neural network, winner-take-all (WTA) circuits",
author = "Zhaofei Yu and Shangqi Guo and Fei Deng and Qi Yan and Keke Huang and Liu, {Jian K.} and Feng Chen",
year = "2018",
month = "10",
day = "3",
doi = "10.1109/TCYB.2018.2871144",
language = "English",
journal = "IEEE Transactions on Cybernetics",
issn = "2168-2267",
publisher = "Institute of Electrical and Electronics Engineers",

}

TY - JOUR

T1 - Emergent Inference of Hidden Markov Models in Spiking Neural Networks Through Winner-Take-All

AU - Yu, Zhaofei

AU - Guo, Shangqi

AU - Deng, Fei

AU - Yan, Qi

AU - Huang, Keke

AU - Liu, Jian K.

AU - Chen, Feng

PY - 2018/10/3

Y1 - 2018/10/3

N2 - Hidden Markov models (HMMs) underpin the solution to many problems in computational neuroscience. However, it is still unclear how to implement inference of HMMs with a network of neurons in the brain. The existing methods suffer from the problem of being nonspiking and inaccurate. Here, we build a precise equivalence between the inference equation of HMMs with time-invariant hidden variables and the dynamics of spiking winner-take-all (WTA) neural networks. We show that the membrane potential of each spiking neuron in the WTA circuit encodes the logarithm of the posterior probability of the hidden variable in each state, and the firing rate of each neuron is proportional to the posterior probability of the HMMs. We prove that the time course of the neural firing rate can implement posterior inference of HMMs. Theoretical analysis and experimental results show that the proposed WTA circuit can get accurate inference results of HMMs.

AB - Hidden Markov models (HMMs) underpin the solution to many problems in computational neuroscience. However, it is still unclear how to implement inference of HMMs with a network of neurons in the brain. The existing methods suffer from the problem of being nonspiking and inaccurate. Here, we build a precise equivalence between the inference equation of HMMs with time-invariant hidden variables and the dynamics of spiking winner-take-all (WTA) neural networks. We show that the membrane potential of each spiking neuron in the WTA circuit encodes the logarithm of the posterior probability of the hidden variable in each state, and the firing rate of each neuron is proportional to the posterior probability of the HMMs. We prove that the time course of the neural firing rate can implement posterior inference of HMMs. Theoretical analysis and experimental results show that the proposed WTA circuit can get accurate inference results of HMMs.

KW - Biological neural networks

KW - Brain modeling

KW - Cybernetics

KW - Hidden Markov models

KW - Hidden Markov models (HMMs)

KW - Markov processes

KW - Mathematical model

KW - neural implementation

KW - Neurons

KW - posterior inference

KW - spiking neural network

KW - winner-take-all (WTA) circuits

UR - http://www.scopus.com/inward/record.url?scp=85054527553&partnerID=8YFLogxK

U2 - 10.1109/TCYB.2018.2871144

DO - 10.1109/TCYB.2018.2871144

M3 - Article

JO - IEEE Transactions on Cybernetics

JF - IEEE Transactions on Cybernetics

SN - 2168-2267

ER -