Implementation of Bayesian Inference in Distributed Neural Networks

Zhaofei Yu, Tiejun Huang, Jian K. Liu

Research output: Chapter in Book/Report/Conference proceedingConference contributionResearchpeer-review

Abstract

Numerous neuroscience experiments have suggested that the cognitive process of human brain is realized as probability reasoning and further modeled as Bayesian inference. It is still unclear how Bayesian inference could be implemented by neural underpinnings in the brain. Here we present a novel Bayesian inference algorithm based on importance sampling. By distributed sampling through a deep tree structure with simple and stackable basic motifs for any given neural circuit, one can perform local inference while guaranteeing the accuracy of global inference. We show that these task-independent motifs can be used in parallel for fast inference without iteration and scale-limitation. Furthermore, experimental simulations with a small-scale neural network demonstrate that our distributed sampling-based algorithm, consisting with our theoretical analysis, can approximate Bayesian inference. Taken all together, we provide a proofof-principle to use distributed neural networks to implement Bayesian inference, which gives a road-map for large-scale Bayesian network implementation based on spiking neural networks with computer hardwares, including neuromorphic chips.

Original languageEnglish
Title of host publicationProceedings - 26th Euromicro International Conference on Parallel, Distributed, and Network-Based Processing, PDP 2018
PublisherInstitute of Electrical and Electronics Engineers
Pages666-673
Number of pages8
ISBN (Electronic)9781538649756
DOIs
Publication statusPublished - 6 Jun 2018
Event26th Euromicro International Conference on Parallel, Distributed, and Network-Based Processing, PDP 2018 - Cambridge, United Kingdom
Duration: 21 Mar 201823 Mar 2018

Conference

Conference26th Euromicro International Conference on Parallel, Distributed, and Network-Based Processing, PDP 2018
CountryUnited Kingdom
CityCambridge
Period21/03/1823/03/18

    Fingerprint

Keywords

  • Bayesian inference
  • distributed neural network
  • importance sampling
  • neural implementation

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Hardware and Architecture

Cite this

Yu, Z., Huang, T., & Liu, J. K. (2018). Implementation of Bayesian Inference in Distributed Neural Networks. In Proceedings - 26th Euromicro International Conference on Parallel, Distributed, and Network-Based Processing, PDP 2018 (pp. 666-673). Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/PDP2018.2018.00111