Geek Repo

Github PK Tool

# 简易神经网络

## 原理

#### 概述

$$W_{mn} = \begin{bmatrix} w_{11} & w_{12} & \cdots & w_{1n} \\ \vdots & \vdots & \ddots & \vdots \\ w_{m1} & w_{m2} & \cdots & w_{mn} \end{bmatrix}$$

#### 激活函数

$sigmoid(x) \in (0, 1)$，因此保持输入值在 $(0, 1)$为佳

$$X_{out} = sigmoid(X_{in})$$

#### 信号传递

$X$层、$Y$层之间信号传递表示为 $$Y_{in} = W_{XY}^TX_{out}$$

#### 误差

$$e_{hidden, i} = \frac{\sum_k w_{ik}e_{out,k}}{\sum_k w_{ik}}$$

$$e_{hidden, i} = \sum_k w_{ik}e_{out,k}$$

$$E_{hidden} = W_{ho}E_{out}$$

#### 权重修正

$$\frac{\partial e_j}{\partial w_{ij}} = \frac{\partial e_j}{\partial output_j} \cdot \frac{\partial output_j}{\partial w_{ij}} = -2(target_j - output_j) \cdot \frac{\partial sigmoid(\sum_k w_{kj} \cdot output_k)}{\partial w_{ij}}$$

$$\frac{\partial sigmoid(x)}{\partial x} = sigmoid(x) \cdot (1 - sidmoid(x))$$

$$sigmoid(\sum_k w_{kj} \cdot output_k) = sigmoid(input_j) = output_j$$

$$\frac{\partial \sum_k w_{kj} \cdot output_k}{\partial w_{ij}} = output_i$$

$$\frac{\partial e_j}{\partial w_{ij}} = -2(target_j - output_j) \cdot sigmoid(\sum_k w_{kj} \cdot output_k) \cdot (1 - sigmoid(\sum_k w_{kj} \cdot output_k)) \cdot output_i$$

$$= -2(target_j - output_j) \cdot output_j \cdot (1 - output_j) \cdot output_i$$

$$W_{ij} = W_{ij} + \alpha \cdot E_j \cdot Output_j(I - Output_j) \cdot Output_i^T$$

$$I = \begin{bmatrix} 1 \\ 1 \\ \vdots \\ 1 \end{bmatrix}$$