1. Some materials are taken from machine learning course of Victor Kitov
Human brain consists of multiple interconnected neuron cells
This is structure of multilayer perceptron - acyclic directed graph.
Output of neuron $j$: $O_{j}=f(I_{j})$.
Number of layers usually denotes all layers except input layer (hidden layers+output layer)
Classification:
We may optimize neural network using gradient descent:
k=0
initialize randomly w_0 # small values for sigmoid and tangh
while stop criteria not met:
w_k+1 := w_k - alpha * grad(L(w_k))
k := k+1
Standardization of features makes gradient descend converge faster
Different optima will correspond to:
So we may solve task many times for different conditions and then
And/Or use some complex optimization methods
x = np.linspace(-10, 10, 1000)
gr_sigm = sigmoid(x)*(1-sigmoid(x))
plt.plot(x, gr_sigm)
[<matplotlib.lines.Line2D at 0x120210da0>]
Advantages of neural networks:
Disadvantages of neural networks: