1. Some materials are taken from machine learning course of Victor Kitov
+1
|-1
)In the end the description of the segments could look like this
[Gender = F][Age > 21][Age <= 25][Education = Higher][Have domestic animals = No]
- like in 82% of cases[Gender = M][Age > 25][Age <= 30][Current Career = Manager]
- don't like in 75% of casesPrediction is performed by tree $T$ (directed, acyclic graph)
Node types
Prediction process for tree $T$:
while $t$ is not a terminal node:
{python}
1. function decision_tree(X, y):
2. if termination_criterion(X, y) == True:
3. S = create_leaf_with_prediction(y)
4. else:
5. S = create_node()
6. (X_1, y_1) .. (X_L, y_L) = best_split(X, y)
7. for i in 1..L:
8. C = decision_tree(X_i, y_i)
9. connect_nodes(S, C)
10. return S
$h(t)\in\{x_{1}^{i(t)},x_{2}^{i(t)},...x_{N}^{i(t)}\}$
Then impurity function $\phi(t)=\phi(p_{1},p_{2},...p_{C})$ should satisfy:
Gini criterion
Entropy
Classification error
plot_impurities()
Define $\Delta I(t)$ - is the quality of the split of node $t$ into child nodes $t_{1},...t_{C}$. $$ \Delta I(t)=I(t)-\sum_{i=1}^{C}I(t_{i})\frac{N(t_{i})}{N(t)} $$ $$ \Delta I(t)=I(t)-\left(I(t_{L})\frac{N(t_{L})}{N(t)} + I(t_{R})\frac{N(t_{R})}{N(t)}\right) $$
$$\begin{cases} \text{left child }t_{1}, & \text{if }x^{i_{t}}\le h_{t}\\ \text{right child }t_{2}, & \text{if }x^{i_{t}}>h_{t} \end{cases}$$
Remarks
wine_demo()
fig = interact(demo_dec_tree, depth=IntSlider(min=1, max=5, value=1))
fig = interact(plot_dec_reg, depth=IntSlider(min=1, max=5, value=1), criterion=['mse', 'mae'])
Advantages:
Disadvantages:
Null
, try $x^{i_t}$Advantages:
ID 3
С 4.5