On AI, Machine Learning, DL etc: Things some people on this forum rave about but dont understand...

[Shamsher1990]

Hello folks!

It seems ML, AI, Neural Networks and Deep Learning are the rave of this decade. I have seen miles long threads on this forum with people copy pasting things blindly without understanding what they are posting and journalists writing rad stories about AI without bothering to know what it is.

So, I want to use this thread as a place to write about AI/ML/DL/NN etc in simple to understand language, so that we all can discuss about it without invoking nationalism and politics. I will like to keep this thread free from any name calling and as a reference and clearing house to all things AI/DL/ML etc. Briefly, I am an AI enthusiast and I am preparing to participate in Kaggle : An open international Machine Learning competition with prizes of order of 10K-100K USD or sometimes more. I don't know everything about AI/ML/DL/NN etc but there are things that I know and there are many many things that I don't know.

Scope of this thread is not to train anyone in AI --just like most of the avid posters of this forums will never be real service-men-- but to help you understand what usually AI/ML/DL/NN etc entails and especially what it DOES NOT entails.

I welcome everyone to write about the things they know but I have few ground rules :

1. Don't copy paste articles/news postings. There are too many other threads for that.

2. Please do not resort to name calling or nation-name calling. There are again too many threads for that too.....

3. Smileys are fun but don't make a smiley-fest. Again, read the reason for the same.

4. Avoid copy pasting equations but tell intuitive feel behind the concepts in simple words. We have text books for that.

5. It will be hard to present a rigorous mathematical proof of many ideas discussed in this thread. Again, refer to above ground rule.

6. It is not possible for most of us to know every last concept or even every main concept and architecture and model, so don't flame if some model is not discussed. Feel free to write about it. Again, refer to above suggestions.

So lets begin... I will post the first part ... What is AI, ML, DL, NN etc.

 

[Shamsher1990]

What is AI?
AI, simply put, is practically any technology which allows us to construct or mostly develop software which can perform tasks which usually require human intelligence or at times skills. Most common examples: Playing Chess, Making summary of an article, driving cars etc. That said, Human Intelligence itself is not exactly a very well defined term, so, many tasks like sorting a pack of cards may seem to require human skills or intelligence but usually people dont call these tasks AI: For the reason there are existing well defined mathematical procedures (called Algorithms) which can do them. So, usually, AI often applies to things for which there is no known algorithm.

Actually I lied in the last sentence. The correct version is 'there is no known efficient algorithm'. The term efficient algorithm is a rather deep and technical concept but in short, there are no known algorithms and methods which are fast enough so that they don't double or triple or more time each time you increase the problem size by just one more variable.

What is ML?
ML or Machine Learning is one way to achieve artificial intelligence. Its not the only one surely. It is usually a statistical technique which can improve as you feed it more example of correct solutions. In that sense it 'learns'. An example will be to develop a system which can tell cats apart from say dogs based on their picture. If you have a machine learning solution, it will begin with making lots of error but if you feed it with many many images of cats with a label 'this is cat' and dogs with a label 'this is NOT a cat', it will reduce its error over attempts. It learns from examples.

A contrasting thing will be writing a program to play tic-tac-toe. You don't need to exactly give such a program examples of successful and failed plays but simply you program a known winning strategy in steps which a computer can follow. The computer does not 'learn' it from its mistakes but solution is given upfront. A similar situation can arise in some chess playing programs as well.

Is ML different from AI?
ML is one way to achieve AI. It is one way to make softwares or machines which show human/animal like intelligence.

What is DL or Deep Learning?
As you might have noticed the word 'Learning' in its name; it is a specific kind of technique under the fold of Machine Learning. It is actually under the fold of a bigger technique under ML called ANNs or Artificial Neural Networks. The word 'Deep' in this technique implies that ANNs under the fold of this techniques are 'Deep' or have numerous layers. More on this later.

What is a Model?

You will see this term being used a lot in AI/DL context. A model, as its name suggests, is a model of reality. More specifically it is a bunch of equations which can tell us how something real, say a ball, will behave if we know something about that real thing and something about what is happening to it. Example, how a far ball will drop if you know its mass, the impulse from the cannon applied on it, the angle at which the cannon was from the ground. You can write equations which can be solved to find the distance at which the ball will fall. This set of equations with information about ball and what is happening to it is a model.

In ML, usually a model means a model of a task requiring intelligence. Like the model above, this model is a set of equations and their constants and configurable parameters and given an input these model can also be solved or calculated to a decision or value which perhaps a human will also arrive at.

Example: A ML model after learning can evaluate if a given image is of a cat or not a cat. This is usually called as 'Model Inference'.

Most often, these ML models have internal value which need to be automatically changed and adjusted during learning so that it predicts correctly. This is usually called as model training. It entails given model example data so that it can adjust its internal values or weights.

There is another type of variables or values in a model, which a ML 'Scientist' adjusts. These values are called Hyper Parameters.

What is Model-Learning or training?
Model learning or training is the core task in Machine Learning. It simply means construct an algorithm or a mathematica procedure programmed as a software which automatically adjusts weights of a model to arrive at a model which is better than before. This usually entails feeding the training algorithm a bunch of examples --like images of cats and dogs and the algorithm then adjusts weights of model based on errors it makes automatically.

 

[gangsta_rap]

fake

 

[EpiiC]

fake

Whats fake?

 

[majorytyrant]

dude the main problem with deep learning is data. if you want to show up in kaggle top 25 go for ensemble learning. usually, the competition that they post has unstructured and non-linear data so applying the model without knowing which gives you best is like hitting the wall. many noobies start applying PCA and other compressing technique without feature engineering never seen themselves above 6k rank.

 

[Shamsher1990]

dude the main problem with deep learning is data. if you want to show up in kaggle top 25 go for ensemble learning. usually, the competition that they post has unstructured and non-linear data so applying the model without knowing which gives you best is like hitting the wall. many noobies start applying PCA and other compressing technique without feature engineering never seen themselves above 6k rank.

Yes, data is the biggest issue in deep-learning and one of the reasons I started this thread. I want to impress upon hoards of forum posters going ga-ga over 'AI' what it entails.

Also, it depends on the problem and how much data you get. Currently I am solving a problem for which feature engineering is a fools errand. You cann't engineer features good enough in it and only way to solve this issue is to 'learn features'. In other words transfer learning. You adjust the domain of pre-trained features.

 

[jamahir]

There is another type of variables or values in a model, which a ML 'Scientist' adjusts. These values are called Hyper Parameters.

Please explain more about "Hyper Parameters".

Model learning or training is the core task in Machine Learning. It simply means construct an algorithm or a mathematica procedure programmed as a software which automatically adjusts weights of a model to arrive at a model which is better than before.

Please explain more about "mathematica".

dude the main problem with deep learning is data. if you want to show up in kaggle top 25 go for ensemble learning. usually, the competition that they post has unstructured and non-linear data so applying the model without knowing which gives you best is like hitting the wall. many noobies start applying PCA and other compressing technique without feature engineering never seen themselves above 6k rank.

@Shamsher1990 , please explain the above in simple words.

Also, would a hardware implementation of AI ( in microprocessor ) be better than in software ??

 

[Nefarious]

Geeks

 

[Shamsher1990]

Please explain more about "mathematica".

That was a typo. It was supposed to be 'Mathematical'. For some reason, I cann't edit my posts.

Please explain more about "Hyper Parameters".

It will be explained in more details in later posts.

@Shamsher1990 , please explain the above in simple words.

There are quite a bit of background required before this can be explained in anyway. Things like "Dimensionality Reduction", which he has referred to as compression via Principal Component Analysis (PCA), "Features" and art of choose feature or "transforming" and "combining" features called "Feature Engineering". "Ensambling" or combining a number of "weak" models for a "strong" model. All of these require a much larger discussion. They will come but there needs to be a bit of background before that.

 

[Shamsher1990]

Different types of Problems solved in 'AI'.

1. Classification or telling things apart
This type of problem usually mean given a picture or sound or description --textual or numerical-- of any real world things you want to machine or software to tell --out of a known group of categories -- which category or class it is from. Example Spam filtering. Given an email, you want your ML software to categorize it as "Spam" or "Not Spam".

Spam filtering is actually an important type of Classification Problem. They are called binary classification problems: ie there are two classes only, 'Something' or 'Not Something'. Another example are 'Detectors', like face detectors used in Face Recognition or Face search in a picture.

The boxes in this picture is the handy work of this Face Detector. It basically can tell, given a section of image if it contains face or not.

Another important example of Classification problem is Recognition by classifying into a number of known classes. Imagine each face is as a category or class. So now after detecting you can re-classify which 'face' is detected face is. So you can now where and which person's face appears in the picture.

2. Regression or predicting

This is another important type of AI/ML problem. You have some sample values of a known thing and you want to predict others. The approach is to take a model and train it on known values and then feed unknown inputs to predict. Example :

Lets say we want to predict how much grain is produced. So we try to first reason out which are the things on which grain crops usually depend. Let say we arrive at following list.

1. Rain fall in mm during the year
2. Highest temperature in the year in C
3. Lowest temperature during the year in C
4. Amount of fertilizer shipped during the year in tonnes


So we take past several years of data of above AND the grains produced. We fit a model, ex, linear regression model with the above data ie relating above 4 values to how much crop is produced. Then we put the above four values for an year we want to predict and if it is fitted properly, the model will allow us to predict the grain tonnage.

3. Sequence to Sequence Mapping
This is another interesting type of problem usually solved in ML. Given a sequence of variables map it to another. Example will be :
Given a sequence of words forming a sentence, map it to another in another language. ie Machine Translation.
Or given a waveform of sound map it to a sequence of words. ie Speech Recognition.

I have only touched the surface of problems here. There are more. That said many problems can be reduced to these fundamental ones.