Jia Yangqing: How does AI reshape the traditional software industry?

AI is reshaping every aspect of human society, such as developing new drugs and improving people's living habits. So in this new era, how does AI reshape the software engineering industry?

Editor's Note: This article is from the Wechat public number "OReillyAI" (ID: OReillyAI), author's big data abstract, compiled effy. 36 krypton is reproduced by authorization.

Jia Yangqing: How does AI reshape the traditional software industry?

Special thanks to the big data digest subtitle group

Compiler: Effy

In this paper, Jia Yangqing, founder of Cafe and one of the core authors of TensorFlow, gave a speech at the San Francisco Station of AI Conference co-organized by O'Reilly and Intel, "Rethinking Software Engineering in the AI Era". By revealing the pain points of traditional software engineering, he hopes to share with you his own experience in the era of artificial intelligence and software engineering. Reflections on industry development.

For a long time, we have taken for granted the ability to distinguish objects, colors and shapes. What does that mean? For example, when Xiaobian mentioned "chopping a man's color", I believe many ladies and sisters understand it, but how do you give a TA who does not know the color, or a computer description / description of "chopping a man's color"?

Note: Cut off the male color, it is said that lipstick painted with this color can make the world invincible, pull countless younger brothers, and cut off all straight men's hearts.

In the field of image recognition and processing, traditional software engineering studies the problem of computer vision by designing rules, that is, using hard codes to describe object features. Before further study, the histogram of gradients (HOG) was widely used in the north and south of the Yangtze River.

What is HOG? Simply put, HOG tries to collect statistical data locally in an image, or, as it can be understood, it tries to find boundaries in all directions of an object. For example, when you look carefully at the picture under the arrow, you can see the outline of an object similar to a car.

Jia Yangqing: How does AI reshape the traditional software industry?

Is it like going back to the childhood physical examination to see the color-blind card? Another one. Is this much better?

Jia Yangqing: How does AI reshape the traditional software industry?

Therefore, traditional software engineering needs to tell the computer what the rules are, or what the characteristics are, and step by step'tell'the computer what to do. It is conceivable that this design itself limits the further development of computer vision.

From Logical Programming to Self-adaptation Model: From Giving Fish to Giving Data

The error rate of traditional image recognition technology has stagnated at 26%, which has badly affected many scientists, scholars and researchers.

Jia Yangqing: How does AI reshape the traditional software industry?

At this time, a little brother named Alex Krizhevsky proposed to write modules in a more abstract way, and designed what we now call a "convolutional neural network" model. It is constructed and trained with a large amount of data to realize the transformation of image recognition from traditional logical programming to modeling.

Jia Yangqing: How does AI reshape the traditional software industry?

Compared with the traditional image recognition technology, this method achieves a leap in accuracy. AlexNet scored the top five with a 15% error rate in ImageNet in 2012.

AlexNet consists of five convolution layers and three full junction layers, 650,000 neurons and 60,000,000 parameters. Among them, convolution layer plays the role of abstracting and extracting features.

Jia Yangqing: How does AI reshape the traditional software industry?

The traditional method of image recognition is to put the artificially designated features extracted by HOG into the classifier for recognition. AlexNet achieves self-learning of image features through convolution layer, and determines the categorized "one-stop" service through the output probability of full connection layer. It's really excellent!

Reference: http://www.image-net.org/challenges/LSVRC/2012/supervision.pdf

Interestingly, AlexNet also has back-end support for biology. It has been found that in our visual cortex, neurons make stratified inferences. Information is transmitted from one layer to the next, and then to the next, so that the brain can extract more and more complex information.

Data gold deposit

How do you write software in the traditional way? When we write software, we put the source code into the compiler, which converts the source code into a low-level machine language that the computer can interpret and run.

In the field of artificial intelligence, we have written programs or models that are different from traditional software engineering compilation. It is no longer a set of logic, but will get different programs and models according to different training data and target data. These models can derive a general rule, and then use a lot of data and calculations to get accurate results.

For example, for a data set of about 1.2 million pictures, one challenge in the traditional way of processing these compilations is that it requires about 1 trillion times of computation (exaflop) to train an image network model.

What is this concept? If every Londoner were to do a floating-point operation every second, it would take about four thousand years to train the model.

In fact, we have realized that the calculation of AI is very barbaric to some extent. Especially in convolutional neural networks, or superimposed neural networks, we need to do a lot of float operations.

So a few years ago, we started building and developing more efficient hardware. Furthermore, we have established data centers, scale clusters or environments.

Please read the Chinese version for details.