Decoding Word Information from Spatiotemporal Activity of Sensory Neurons

Abstract

Spatiotemporal activity of neurons is ubiquitous in sensory coding in the CNS. It is a fundamental problem for sensory perception to understand how sensory information is decoded from the spatiotemporal activity. However, little is known about the decoding mechanism. To address this issue, we are concerned with auditory system as a model system exhibiting spatiotemporal activity. We present here a model of auditory cortex, which performs a hierarchical processing of auditory information. The model consists of three layers of two-dimensional networks. The first layer represents auditory stimulus as a spatiotemporal activity of neurons. The second layer consists of feature-detecting neurons, which extract the features of phonemes and their overlaps from the spatiotemporal activity of the first layer. The third layer combines information of the sound features encoded by the second layer and decodes word information about the sound stimulus as a temporal sequence of attractors. Using the model, we show how the information of phonemes and words emerge in the hierarchical processing of the auditory cortex. We also show that the overlap between phonemes plays a crucial role in linking the attractors of phonemes. The present study may provide a clue for understanding the mechanism by which word information is decoded from spatiotemporal activity of neurons.

Appendix: Model of FR Layer

As the model of the FR layer, we used a simple network model of wave propagation in the auditory cortex, proposed by Yamaguchi et al. [48]. The network model of the FR layer consists of a two-dimensional array of population units as shown in Fig. 1b. The unit consists of a pair of excitatory and inhibitory neuron as shown in Fig. 1c. The network model has the two axes, one is the tonotopical axis representing frequency selectivity of input, or T-axis, and the other is propagation axis, or P-axis, along which excitatory wave is propagated. Each population unit is numbered by ij, where i and j, respectively, denote the position on T- and P-axis (i = 1 − N _T and j = 1 − N _P ). The time evolution of the excitatory and inhibitory neurons in (i, j)th unit is given by the following equations:

$$\tau_E \frac{{\text{d}} x_{ij}}{{\text{d}}t} = - x_{ij} - \alpha P(y_{ij}) + \kappa P(x_{ij}) + \Upgamma + \sum\limits_{mn} w_{ijmn} P(x_{mn}) + \delta_j I_i - H_{ij},$$

(8)

$$\tau_I \frac{{\text{d}} y_{ij}}{{\text{d}}t} = - y_{ij} + \beta P(x_{ij}) - \beta_0,$$

(9)

where x _ij and y _ij are the internal variables of excitatory and inhibitory neuron in (i, j)th unit, τ _E and τ _I are the time constants, and $\alpha, \beta, \kappa, \Upgamma$, and β ₀ are constant parameters. The output function P is given by

$$P(z) = \frac{1}{2}(\tanh(\lambda_z (z - \mu_z)) + 1),$$

(10)

where λ _z is the gradient parameter, and μ _z is the threshold value.

The external input to jth neurons on the ith tonotopy axis is given by δ _j I _i . The factor δ _j equals 1 if j = 1, and 0 otherwise, and the term I _i is the magnitude of the input to ith neuron along T-axis.

The synaptic weight from (m, n)th neuron to (i, j)th neuron in the FR layer, w _ijmn is given by

$$w_{ijmn} = W_0(I_i + I_m) + w_0, \quad \hbox{for}\,i-1 \leq m \leq i+1 \,\hbox{and}\,n = j - 1,$$

(11)

where w ₀ gives the synaptic weight in the absence of external input. The inputs I _i and I _m increase the synaptic weights of the connections between ith and mth frequency bands, and W ₀ is a constant parameter. Thus, neurons in the ith frequency band are characterized by the common input I _i . It gives the excitatory inputs at the left edge of P-axis, while neurons in other columns receive no direct input but input elicited by instantaneous synaptic modulation.

The last term of the righthand side of Eq. (8) indicates the lateral inhibitions between neighboring neurons, given by

$$H_{ij} = \sum\limits_{q \in Q} h P(x_{qj}).$$

(12)

The range of competition is represented by Q = {q| q = i − 1, i + 1}, and h is the magnitudes of the connections.

The parameter values used are as follows: $\tau_E=30\,\hbox{ms}, \tau_I=30\,\hbox{ms}, \alpha=8.0, \beta=8.0, \beta_0=1.0, \kappa=4.0, \Upgamma=1.6, h=0.025, W_0=2.0, w_0=0.1, \lambda_z=9.0, \mu_z=2.0, I_i=1.0, I_m=1.0, N_P=50$, and N _T  = 20.