Graphical Top Ten Classical Machine Learning Algorithms

Weak AI has made great breakthroughs in recent years, quietly, it has become an indispensable part of everyone's life. Take our smartphone as an example to see how many magic tricks of artificial intelligence are hidden in it.

The following is a typical smartphone installation of some common applications, many people may not guess, artificial intelligence technology has been the core driver of many applications on the mobile phone.

Graphical Top Ten Classical Machine Learning Algorithms

Figure 1 Relevant applications on smartphones

Traditional machine learning algorithms include decision tree, clustering, Bayesian classification, support vector machine, EM, Adaboost and so on. This article will give a general introduction to common algorithms, no code, no complicated theoretical derivation, that is, to figure out what these algorithms are and how they are applied.

The field of artificial intelligence has a wide range of knowledge. We recommend Deep Blue College, a platform that focuses on online education of artificial intelligence. Deep Blue College was founded by the Ph. D. team graduated from the Institute of Automation, Chinese Academy of Sciences. Although it has been established for half a year, it has gained a good reputation in the industry.

decision tree

According to some features, each node asks a question, divides the data into two categories by judgment, and then continues to ask questions. These problems are learnt from the existing data. When new data is put into the tree, the data can be divided into suitable leaves according to the problems on the tree.

Graphical Top Ten Classical Machine Learning Algorithms

Figure 2 Schematic diagram of decision tree

Random Forest

Data are randomly selected from the source data to form several subsets:

Graphical Top Ten Classical Machine Learning Algorithms

Figure 3-1 schematic diagram of random forest

S matrix is the source data, with 1-N data, A, B, C are features, and the last column C is categories:

Graphical Top Ten Classical Machine Learning Algorithms

M sub-matrices are randomly generated from S:

Graphical Top Ten Classical Machine Learning Algorithms

This M subset obtains M decision trees: the new data is put into the M tree and M classification results are obtained. Counting to see which category is the most predicted, the category is regarded as the final prediction result.

Graphical Top Ten Classical Machine Learning Algorithms

Figure 3-2 Showcase of Random Forest Effects

logistic regression

When the prediction target is probability and the range needs to be greater than or equal to 0, less than or equal to 1, the simple linear model can not do this, because the range is beyond the prescribed range when the definition range is not within a certain range.

Graphical Top Ten Classical Machine Learning Algorithms

Figure 4-1 Linear Model Diagram

So it would be better to have a model of this shape at this time:

Graphical Top Ten Classical Machine Learning Algorithms

Figure 4-2

So how can we get such a model?

This model needs to satisfy two conditions: "greater than or equal to 0" and "less than or equal to 1". Models larger than or equal to 0 can choose absolute values, square values, where the exponential function must be greater than 0; when less than or equal to 1, the molecule is itself, and the denominator itself plus 1, that must be less than 1.

Graphical Top Ten Classical Machine Learning Algorithms

Figure 4-3

After another deformation, we get the logistic regressions model:

Graphical Top Ten Classical Machine Learning Algorithms

Figure 4-4

The corresponding coefficients can be obtained by calculating the source data.

Graphical Top Ten Classical Machine Learning Algorithms

Figure 4-5

Graphical Top Ten Classical Machine Learning Algorithms

Figure 4-6 LR Model Curve

Support Vector Machine

To separate the two classes, in order to get a hyperplane, the optimal hyperplane is to maximize the margin of the two classes. Margin is the distance between the hyperplane and the nearest point to it. As shown in the following figure, Z2 > Z1, so the green hyperplane is better.

Graphical Top Ten Classical Machine Learning Algorithms

Figure 5 Schematic diagram of classification problem

The hyperplane is expressed as a linear equation. The one above the line is greater than or equal to 1, and the other is less than or equal to -1:

Graphical Top Ten Classical Machine Learning Algorithms

The distance from the point to the surface is calculated according to the formula in the figure.

Graphical Top Ten Classical Machine Learning Algorithms

So we get the expression of total margin as follows. The goal is to maximize the margin, so we need to minimize the denominator. So it becomes an optimization problem.

Graphical Top Ten Classical Machine Learning Algorithms

For example, we find the optimal hyperplane at three points and define weight vector = (2,3) - (1,1):

Graphical Top Ten Classical Machine Learning Algorithms

The weight vector is obtained as (a, 2a). Two points are substituted into the equation and (2, 3) into (2, 3) and (1, 1) into (1, 1) and (1, 1) into (1, 1). The values of a and truncation W0 are solved, and then the expression of hyperplane is obtained.

Graphical Top Ten Classical Machine Learning Algorithms

When A is calculated, the support vector machine is obtained by substituting (a, 2a), and the equation of substituting a and W0 into the hyperplane is the support vector machine.

Naive Bayes

For example, in the application of NLP: give a text, return to emotional classification, the attitude of this text is positive or negative:

Graphical Top Ten Classical Machine Learning Algorithms

Figure 6-1 Question Case

To solve this problem, we can only look at some of the words:

Graphical Top Ten Classical Machine Learning Algorithms

This passage will be represented by only a few words and their counts:

Graphical Top Ten Classical Machine Learning Algorithms

The original question is: To give you a word, what kind does it belong to? Through Bayes rules, it becomes a relatively simple and easy problem to solve:

Graphical Top Ten Classical Machine Learning Algorithms

The question becomes, what is the probability of this sentence in this category, and of course, don't forget the other two probabilities in the formula.

Please read the Chinese version for details.