Three Frameworks of Deep Learning (Contrast)

The tide of artificial intelligence is sweeping the world, and many words are lingering in our ears, such as artificial intelligence, machine learning, in-depth learning and so on. The concept of "artificial intelligence" was put forward as early as 1956. As its name implies, computers are used to construct complex machines with the same essential characteristics as human intelligence. After decades of development, in 2012, thanks to the rise of data, the improvement of computing power and the emergence of machine learning algorithms (in-depth learning), artificial intelligence began to explode. However, the current research work is focused on the weak AI part, that is to say, the machine has the ability of observation and perception, can understand and reasoning to a certain extent, and it is expected that some major breakthroughs will be made in this field. Artificial intelligence in movies is mostly about strong artificial intelligence, that is, to enable machines to acquire adaptive ability to solve some problems that have not been encountered before, and this part is difficult to really achieve in the current real world. If AI is expected to make a breakthrough, how can it be realized? Where does "intelligence" come from? This is mainly due to a method of realizing artificial intelligence - machine learning. 1. Machine Learning Concept Machine Learning is a method of realizing artificial intelligence. The most basic approach of machine learning is to use algorithms to parse data and learn from it, and then make decisions and predictions about real world events. Unlike traditional hard-coded software programs for specific tasks, machine learning uses a large amount of data to "train" and learns how to accomplish tasks from data through various algorithms. Machine learning originated from the early field of artificial intelligence. Traditional algorithms include decision tree, clustering, Bayesian classification, support vector machine, EM, Adaboost and so on. In terms of learning methods, machine learning algorithms can be divided into supervised learning (such as classification problems), unsupervised learning (such as clustering problems), semi-supervised learning, ensemble learning, in-depth learning and reinforcement learning. Traditional machine learning algorithms in fingerprint recognition, Haar-based face detection, HoG-based object detection and other fields basically meet the requirements of commercialization or the level of commercialization of specific scenarios, but each step is extremely difficult until the emergence of deep learning algorithms. 2. Deep learning concept deep learning is a technology to realize machine learning. It is not an independent learning method in itself, and supervised and unsupervised learning methods are also used to train deep neural networks. However, due to the rapid development of this field in recent years, some unique learning methods have been proposed one after another (such as residual network), so more and more people regard it as a learning method alone. Initial deep learning is a learning process that uses deep neural network to solve feature expression. Deep neural network is not a new concept in itself. It can be roughly understood as a neural network structure with multiple hidden layers. In order to improve the training effect of deep-seated neural network, people have made corresponding adjustments to the connection method and activation function of neurons. In fact, there were many ideas in the early years, but because of the insufficient training data and backward computing ability, the ultimate effect was not satisfactory. Deep learning, as the hottest machine learning method at present, does not mean the end of machine learning. At least at present, there are the following problems: 1. Deep learning model needs a lot of training data to show the magic effect, but in real life, it often encounters small sample problems. At this time, deep learning methods can not start, traditional machine learning methods can be dealt with; 2. In some areas, the traditional simple machine learning methods can be used to solve well, but not yet. It is necessary to use complex deep learning methods. 3. The thought of deep learning comes from the inspiration of human brain, but it is not the simulation of human brain. Therefore, there are differences between machine learning frameworks and deep learning frameworks. In essence, machine learning framework covers various learning methods for classification, regression, clustering, anomaly detection and data preparation, as well as neural network methods. Deep learning or deep neural network (DNN) framework covers a variety of neural network topologies with many hidden layers, including multi-step process of pattern recognition. The more layers in the network, the more complex it is to extract features for clustering and classification. Common Caffe, CNTK, Deep Learning 4j, Keras, MXNet and TensorFlow are deep learning frameworks. Scikit-learning and Park MLlib are machine learning frameworks. Theano crosses these two categories. In the next part of this paper, we will focus on three frameworks of deep learning: caffe, tensorflow and keras. It is more appropriate to use scikit-learning and spark MLlib only when traditional machine learning basic algorithms are needed. Thirdly, the deep learning framework comparison neural network generally includes two stages: training and testing. Training is the process of extracting model parameters from training data and neural network models (AlexNet, RNN, etc.) using CPU or GPU. Testing is to use the trained model (neural network model + model parameters) to run the test data and see the results. Caffe, keras and tensorflow are the unified abstraction of the link data involved in the training process to form a usable framework. (1) Caffe1 and Concept Caffe are clear and efficient deep learning frameworks. They are also widely used open source deep learning frameworks. Before the emergence of Tensorflow, Github star was the largest project in the field of deep learning. The main advantages are: easy to use, network

Please read the Chinese version for details.