What are the types of artificial intelligence and how are they measured?

I believe you are referring to something that Arend Hintze wrote about in his article "Understanding the four types of AI, from reactive robots to self-aware beings".  Here are the four types from his article:

Type I AI: Reactive machines
The most basic types of AI systems are purely reactive, and have the ability neither to form memories nor to use past experiences to inform current decisions. Deep Blue, IBM’s chess-playing supercomputer, which beat international grandmaster Garry Kasparov in the late 1990s, is the perfect example of this type of machine.

Deep Blue can identify the pieces on a chess board and know how each moves. It can make predictions about what moves might be next for it and its opponent. And it can choose the most optimal moves from among the possibilities.

But it doesn’t have any concept of the past, nor any memory of what has happened before. Apart from a rarely used chess-specific rule against repeating the same move three times, Deep Blue ignores everything before the present moment. All it does is look at the pieces on the chess board as it stands right now, and choose from possible next moves.

This type of intelligence involves the computer perceiving the world directly and acting on what it sees. It doesn’t rely on an internal concept of the world. In a seminal paper, AI researcher Rodney Brooks argued that we should only build machines like this. His main reason was that people are not very good at programming accurate simulated worlds for computers to use, what is called in AI scholarship a “representation” of the world.

The current intelligent machines we marvel at either have no such concept of the world, or have a very limited and specialized one for its particular duties. The innovation in Deep Blue’s design was not to broaden the range of possible movies the computer considered. Rather, the developers found a way to narrow its view, to stop pursuing some potential future moves, based on how it rated their outcome. Without this ability, Deep Blue would have needed to be an even more powerful computer to actually beat Kasparov.

Similarly, Google’s AlphaGo, which has beaten top human Go experts, can’t evaluate all potential future moves either. Its analysis method is more sophisticated than Deep Blue’s, using a neural network to evaluate game developments.

These methods do improve the ability of AI systems to play specific games better, but they can’t be easily changed or applied to other situations. These computerized imaginations have no concept of the wider world – meaning they can’t function beyond the specific tasks they’re assigned and are easily fooled.

They can’t interactively participate in the world, the way we imagine AI systems one day might. Instead, these machines will behave exactly the same way every time they encounter the same situation. This can be very good for ensuring an AI system is trustworthy: You want your autonomous car to be a reliable driver. But it’s bad if we want machines to truly engage with, and respond to, the world. These simplest AI systems won’t ever be bored, or interested, or sad.

Type II AI: Limited memory
This Type II class contains machines can look into the past. Self-driving cars do some of this already. For example, they observe other cars’ speed and direction. That can’t be done in a just one moment, but rather requires identifying specific objects and monitoring them over time.

These observations are added to the self-driving cars’ preprogrammed representations of the world, which also include lane markings, traffic lights and other important elements, like curves in the road. They’re included when the car decides when to change lanes, to avoid cutting off another driver or being hit by a nearby car.

But these simple pieces of information about the past are only transient. They aren’t saved as part of the car’s library of experience it can learn from, the way human drivers compile experience over years behind the wheel.

So how can we build AI systems that build full representations, remember their experiences and learn how to handle new situations? Brooks was right in that it is very difficult to do this. My own research into methods inspired by Darwinian evolution can start to make up for human shortcomings by letting the machines build their own representations.

Type III AI: Theory of mind
We might stop here, and call this point the important divide between the machines we have and the machines we will build in the future. However, it is better to be more specific to discuss the types of representations machines need to form, and what they need to be about.

Machines in the next, more advanced, class not only form representations about the world, but also about other agents or entities in the world. In psychology, this is called “theory of mind” – the understanding that people, creatures and objects in the world can have thoughts and emotions that affect their own behavior.

This is crucial to how we humans formed societies, because they allowed us to have social interactions. Without understanding each other’s motives and intentions, and without taking into account what somebody else knows either about me or the environment, working together is at best difficult, at worst impossible.

If AI systems are indeed ever to walk among us, they’ll have to be able to understand that each of us has thoughts and feelings and expectations for how we’ll be treated. And they’ll have to adjust their behavior accordingly.

Type IV AI: Self-awareness
The final step of AI development is to build systems that can form representations about themselves. Ultimately, we AI researchers will have to not only understand consciousness, but build machines that have it.

This is, in a sense, an extension of the “theory of mind” possessed by Type III artificial intelligences. Consciousness is also called “self-awareness” for a reason. (“I want that item” is a very different statement from “I know I want that item.”) Conscious beings are aware of themselves, know about their internal states, and are able to predict feelings of others. We assume someone honking behind us in traffic is angry or impatient, because that’s how we feel when we honk at others. Without a theory of mind, we could not make those sorts of inferences.

While we are probably far from creating machines that are self-aware, we should focus our efforts toward understanding memory, learning and the ability to base decisions on past experiences. This is an important step to understand human intelligence on its own. And it is crucial if we want to design or evolve machines that are more than exceptional at classifying what they see in front of them.


How do I locate a specific object in an image?

so assuming your not allowed to use transfer methodologies (like take an already exisiting elephant object detector) my recommendation is to train a CNN classifier (labels are binary-- elephant exist, elephant doesnt exist) and then use strategies founded in like grad cam. Note there does exist a gradcam++ but because you can assure theres only one instance, it isnt necessary and is just more complicated.

Note that since you just need the location and not the pixel specificity, you dont even need to do the guided backprop, but just the relation with respect to the last convoluitional map.

A quick description is that its using the gradient of the class loss w.r.t the last feature map to see which locations helped make the classification, and from there you can upscale to the receptive field that those neurons touch

Hope this helped!


As you read, in keras the input dimensions for a LSTM layer are:  (batch_size, timesteps, input_dim). Where:

batch_size: number of samples (sentences in your case) to compute the loss before running the gradient descent. So number of sentences to train on before compute loss and optimize your model.
timesteps: length of the sequence, in your case length of the sentence
input_dim: the features of your data (the words)

The nice thing about Keras is that you can train with an specific batch size, say batch_size=16, that: helps the model convergence (because it averages the loss of the 16 sentences prediction) and boosts the speed (since the weights are updated only once every 16 sentences).

But then you can infer (or do the predictions) with batch_size=1, meaning, one sentence at a time.

So, if you want to train, specifically with one sentence at a time: batch_size=1. But if you want to take the advantage of using mini-batch then use batch_size=16 or higher for training and batch_size=1 for inference.

BONUS: Keras allows for dynamic batch size change in the models. When a dimension is dynamic in Keras has the value of None, that is why when you do model.summary() you can see: (batch_size, timesteps, input_dim) = (None, 40, 100). This None allows for different batch_size in training and inference


Check out this answer, and other answers too. They explained the what the ideal order is and why so. Also, you can just add layer of activation function.
from tensorflow.keras import layers
from tensorflow.keras import activations

model.add(layers.Activation(activations.relu))

They are actually first things when you search on google, so I kindly recommend you to search before asking


Are there independent evaluations of various major AI platforms?

In recent times different data science magazines and institutions have published their reviews of the top AI toolkits. In these reviews they tend to highlight the innovative features possessed by each platform as well as their reliability and ability to scale.

Below are a some evaluations of AI platforms that I recommend you have a look at:

KDnuggets review https://www.kdnuggets.com/2016/04/top-15-frameworks-machine-learning-experts.html

Paperspace review https://blog.paperspace.com/which-ml-framework-should-i-use/

Predictive Analytics review https://www.predictiveanalyticstoday.com/deep-learning-software-libraries/

Infoworld review https://www.infoworld.com/article/3026262/machine-learning/13-frameworks-for-mastering-machine-learning.html#slide4

Wikipedia comparison https://en.wikipedia.org/wiki/Comparison_of_deep_learning_software

However if your main interest is to get an objective and unbiased assessment of the different ML toolkits. Then the best place to get such first hand reviews is at GitHub. Here you can look at the number of forks, stars and downloads, the update frequency, the number of issues, the number of contributors, the documentation, the license and even the code itself. This approach enables you to get an impartial and neutral appraisal directly from the developers themselves.

Below is my personal evaluation of the different ML toolkits:

Tensorflow - TensorFlow is a powerful open source library for numerical computation using data flow graphs. Some of the advantages of tensorflow are: It is portable and consequently can run on GPU's, CPU's, desktops, servers and mobile computing platforms. It comes with tensorboard which is useful in visualising and fine-tuning your network. Tensorflow has a large following with lots of researchers and developers using it. This means that most issues can be dealt with easily since they are usually the same issues that other developers run into.

Microsoft CNTK - Is a unified deep learning toolkit that describes neural networks as a series of computational steps via a directed graph. It delivers state of the art results when it comes to speech recognition, in recent times it has even outperformed humans in transcribing speech to text. The external CNTK toolkit available as open-source is also identical to the internal toolkit. This means you get to access all the features that internal Microsoft developers utilize in production.

Theano - Theano is a symbolic compiler library that builds an optimized Neural Network from mathematical expressions. A major advantage of theano is that it supports Python libraries meaning that a wide range of Python libraries i.e. Numpy, SciPy and Scikit-learn can easily be integrated with theano. Additionally theano has excellent high level API's such as keras and lasagne which greatly simplify the use of the Theano toolkit.

Torch - Is a scientific computing framework and script language that provides a wide range of algorithms for machine learning. The benefits of torch are: It has a very active open source community given that Facebook is standing behind it. Additionally it is based on Lua which is a very easy language to pick up. On top of that a significant number of AI researchers publish their research findings in torch.

Caffe - Is a deep learning framework originally developed at UC Berkeley. Some of the perks of using Caffe are: It has an excellent Matlab and Python interface, it supports functions such as ReLu straight out of the box furthermore Caffe excels at CNN's for images.


What is the appropriate RNN structure to do Sentiment Analysis with multiple dependent ratings?

Inspired by (1) machine translation where source phrases are first encoded to a feature vector and decoded by another RNN and (2) automatic captioning where a picture is also encoded to a feature vector before it's decoded by the RNN followed. A proper structure for the problem above may be the inverse of these, where the review is encoded by RNN to a vector, and a DNN is followed to decode the vector into #1 - #3.


Executing trained image classification model for video

It is possible to deploy models that process video ranging from frame-by-frame analysis up to live streaming, depending on the use case. The reason for deployment will influence video quality and pre-processing as well as the complexity of model that can be used.

As video can result in a lot of data piling up very rapidly, compromises might be made between quality of image resolution and colour, frames might be skipped, processing could be done offline, or it might just cost a ton of money to pay for the compute power! Without going into much detail, different algorithms can be used for the different tasks of detection, recognition/classification, and tracking, and the input to each can be processed in different ways.

This tutorial does a great job of showing how OpenCV can be used for object detection from a laptop webcam stream and a pre-trained CNN, it explicitly mentions dropping frames to improve performance.

There's also this post with a good overview of object detection, recognition and tracking. Depending on the use case in the context of each algorithm described, you can determine how important it is to retain and process all frames or not - for instance, see the section on background subtraction; you might not want to risk something flying through the frame without detecting it, let alone classifying it.


How to automatically detect and correct false information in columnar data?

While I can see that there are some heuristics that can tell you whether an entry is 'weird', I don't see any way that you can correct this. Where would you get a correct value from?

I would perhaps start with a statistical analysis, looking at the distribution of values to get an idea of the state the data is in. From this you can then already see some values that wouldn't make sense (this depends very much on what data it is: census data will include toddlers, but credit card applications wouldn't).

Your best bet is to encode some constraints, such as a minimum/maximum range for each value, and perhaps some correspondences, such as a maximum weight for a certain age range. This you can do with simple conditional statements in Python.

You can then flag up those entries which don't look quite right. What you do with them depends on your application. You can filter them out, or manually change the values to something that is consistent with possible values.

Machine Learning techniques would not be able to do anything better than that.


Cnn for Combination of both digits and letters(small and capital)

As for me, the easiest path to what you are asking for is generating them yourself.
An Example
I usually grab some TTF fonts and put them into a directory so that I have variety for character identification.  Begin by importing dependencies and creating a generator function:
from PIL import Image, ImageDraw, ImageFont
from os import listdir

WIDTH       =   200
HEIGHT      =   100
MINX        =    20
MINY        =    20
MAXX        =    WIDTH-60
MAXY        =    HEIGHT-60
MINSIZE     =    24
MAXSIZE     =    48

fonts = [i for i in filter(lambda i:i[-3:]=="ttf", listdir("../data/fonts"))]

def generate_character_image():
    fonts = [i for i in filter(lambda i:i[-3:]=="ttf", listdir("../data/fonts"))]
    charset=list("0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ")
    while 1:
        img = Image.new('RGB', (WIDTH,HEIGHT), color = (255,255,255))
        font = ImageFont.truetype(f'../data/fonts/{fonts[randrange(len(fonts))]}', randrange(MINSIZE,MAXSIZE))
        canvas = ImageDraw.Draw(img)
        where = (randrange(MINX,MAXX), randrange(MINY,MAXY))
        character = charset[randrange(len(charset))]
        coords = canvas.textbbox(where, character, font)
        # The coordinates are the top left corner
        bounding_box = (coords[0], coords[1], coords[2]-coords[0], coords[3]-coords[1])
        #canvas.rectangle(coords, outline=0, width=1)
        canvas.text(where, character, font=font, fill=(0,0,0), anchor="la")
        yield (character, coords, img)

for i in range(5):
    character,coords,image = next(generate_captcha())
    print(f"{character} is in Bounding Box: {coords}")
    display(image)
    print(image)

You can then use this to generate an infinite number of training samples pretty trivially.
Why This Might Be Good
Personally, I prefer to generate my datasets whenever possible.  It allows me to do the following:

Train without ever repeating a sample
The validation data changes every time (in fact, I'll often not bother with a validation step since every epoch uses different data)
Zero storage required

Hope this helps!


Problem with fitness calculation in NEAT

Seem like time is a good fitness, though you need it to engage into learning side sensor inputs and side movement.

I would consider adding a bit of randomness to the environment. How about adding some random mild forces that might sway it left, right, front and rear a bit so that that bots are forced to use other sensors and inputs to stay in the center.

In cases when drone is not simulated, this task is a little harder but adding randomness to the environment can still be done. For example, tilt your drone a tiny amount in a random direction at random intervals. This will force your drone to learn to correct for being jostled by wind, without you having to actually produce wind.


How can I translate a natural language question to an MDX query?

You can use a component library which can help you to implement Natural language query builder in your application( the question part ) called Open Natural Language Processing Package
 ,  so you can definitely develop a module, by using existing modules of OpenNLP such as entity extraction, chunking and parsing.

According to wikipedia source   ; it points out that as of 2001, Q&A applications typically includes "a question classifier module which determines the type of question and the type of answer," so "a multiagent question and answering architecture has been proposed, where each domain [or variable] is represented by an agent which tries to answer questions, taking into account its specific knowledge."

But it still need some effort to build a NLgenerators inline with databases , for the answer query and also is the link to help you on how you can work with Compositional Semantic Parsing on Semi-Structured Tables

Hope this can give you some insight.


Is it possible to derive meaning from text by providing multiple ways of saying the same thing to a neural network?

No, it will not derive patterns of meaning, as the network has no understanding of language. What will happen, is that it picks up surface features (usually letter sequences) which are common between sentences with the same (or a similar) meaning.

This approach is often used in chatbots for intent recognition. Sometimes it picks up subtle patterns that humans would not notice, but they are also often not reliable: you get wrong classifications without knowing why.

Having said that, it works fine if you want to distinguish between a limited number of different intents (you don't even need a lot of training examples — 4 to 5 examples are often sufficient if they are well-selected). In this fairly limited scenario yes, otherwise no.


Why is the average reward plot for my reinforcement learning agent different than the usual plots?

Well, the way to know that the agent is actually learning is by looking at its behavior while it performs the task, and by comparing against a known optimal performance.

So, does your agent reaches the goal quickly? Does it step out of the grid frequently? What is the maximum possible sum of rewards / minimum number of steps attainable? Is the agent close to that limit? From your graphic, and if I understood correctly your RL problem, the maximum average reward per step should be close to 1 (depending on the specific environment you are using), so I guess you are not so far from the optimal solution.

Also, probably if you keep training for a longer period, your agent will reach a stable solution that might or might not be optimal. If you keep training after that, your curves surely will look like the ones you found online.


Do deep learning algorithms represent ensemble-based methods?

You should think of them as different approaches. A deep neural net is a single independent model, whereas ensemble models are ensembles of many independent models.

The primary connection between the two is dropout, a particular method of training deep neural nets that's inspired by ensemble methods.


Significance of depth of a deep neural network

Note: The following statements are about feed-forward neural networks. If you're interested in something else, please let me know.

There is a paper which proofs that one hidden layer is enough. So theoretically you could represent any network with many layers with a single hidden layer. I don't remember the title, though. Also I don't remember how it scaled (how many additional nodes you need per added layer).

In practice, it is a very different story anyway:

You need to train the network in an acceptable wall-clock time,
able to fit stuff in memory
You have limitations on maximum time for one inference
You need to find the weights by a training algorithm

Anecdotally, I can say wider networks are usually have more problems with overfitting / memory limitation problems while deeper networks have more problems with maximum inference time.

There is a nice paper from Microsoft where they trained the first super deep network. Many papers followed to that one:

  He, Kaiming, et al. "Deep residual learning for image recognition." Proceedings of the IEEE conference on computer vision and pattern recognition. 2016.

One main claim was that deeper networks learn more complex features. I don't remember how much of a problem overfitting was. Another important insight is that skip connections are critical for really deep networks (> 20 learning layers).


Is there a naming convention for network weights for multilayer networks?

When the system grows matrix notation is used, as a=Wx, being a (input to activation function in hidden layer) and x (values from input layer) column vectors, transpose of (a1,a2,...a_m) and (x1,x2,...,x_n), and W a m-by-n matrix of dimensions m rows and n columns. The standard way to denote matrix elements is w(i,j) where "i" is the row number and "j" column number:

(from wiki)

For this reason, the weight that applies to h4 from x3 is element in row 4 column 3 of the matrix W, that is, W(4,3) ( as your teachers advocates but with a sad lack of ability to explain ).

In your example:

Note: things are a few more complex when x1, x2, ... are itself vectors, but final conclusion is the same.

( PS: URGENT to allow latex notation on this stack exchange ! )


How can I determine whether a video's frame is realistic (was recorded by a camera) or contains computer-generated graphics?

As per your requirements, I would suggest that you start with any simple CNN network.

CNNs take advantage of the hierarchical pattern in data and assemble
more complex patterns using smaller and simpler patterns. Therefore,
on the scale of connectedness and complexity, CNNs are on the lower
extreme.

Here is a Keras example:
model = models.Sequential()
model.add(layers.Conv2D(32, kernel_size=(3, 3), activation='relu', input_shape=image_shape))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(64, kernel_size=(3, 3), activation='relu'))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(64, kernel_size=(3, 3), activation='relu'))
model.add(layers.Flatten())
model.add(layers.Dense(64, activation='relu'))
# output layer
model.add(layers.Dense(1))

where image_shape is the resolution and number of channels of images (e.g. 128x128x3 for RGB images). I also suggest you downscaling the images to a lower resolution. You will also have to crop the images as they must all be the same image_shape.
Also take a look at the MaxPooling2D  and BatchNormalization layers.
Since you only have real and CGI images, this becomes a binary classification problem. Therefore you can have a single output (0 - CGI, 1 - real). Such problems can be solved with BinaryCrossentropy loss.
model.compile(loss=losses.BinaryCrossentropy(from_loits=True), optimizer='adam')

Finally, you can fit your model
history = model.fit(train_images, train_labels, epochs=1000, validation_data=(test_images, test_labels))

You can find a complete example here.
Please note that depending on your data, the model can become biased if your dataset is unbalanced. That is, if all of your CGI images have text, and only a small fraction of the real images also have text, they might be misclassified. Therefore, I recommend that you visualize your model to better understand what it has learned. Here is an example of such a problem we faced at our university.
There are also more advanced CNN architectures such as ResNet, VGG or YOLO. You can also extend your model with time series (i.e. video) using LSTM or GRU architecture.


Neural network architecture for comparison

A RBM (restricted Boltzmann machine) can be trained to extract document features. The same resulting machine can extract features of two or more documents.  Because documents can be just as easily processed in series using the same machine parameters and CPU (saving the feature results) as documents could be processed in parallel using separate CPUs, the Siamese idea is less of a characteristic of system architecture and more of a characteristic of the process topology. The architectural decision of how to run the process topology on available hardware should usually remain somewhat decoupled.

The comparison operation could be a MLP (multi-layer perceptron) trained to produce the comparison results based on a requisite number example document comparisons.  In such a case, the MLP should be trained with the previously trained RBMs as a front end.  You will need some human labor to produce or extract from available data sources the example document comparison results.  The example results will be need, along with a reference to the pair of RBMs feature extraction results corresponding to the comparison, to train the MLP.

Even further upstream from the RBMs, it is often useful to producing reliable and accurate document processing to pre-process the documents.  Images and text can be delineated so that features can be extracted from the images in one way and from the text in another and then from the sequence of text and images in a third way.  In such process topologies, CNNs (convolution network) typically pre-process the images and LSTM networks typically pre-process the text.  They may then directly feed the RBM or indirectly through another process component.


Was DeepMind's DQN Atari game learning simultaneous?

Switching required a re-learn.

Also, note that:

  We use the same network architecture, learning
  algorithm and hyperparameters settings across all seven games, showing that our approach is robust
  enough to work on a variety of games without incorporating game-specific information. While we
  evaluated our agents on the real and unmodified games, we made one change to the reward structure
  of the games during training only.

and

  the
  network has outperformed all previous RL algorithms on six of the seven games we have attempted
  and surpassed an expert human player on three of them.


What is the mean in the variational auto-encoder?

Kind of neither, although leaning towards the first definition of the Mean as a simple average of values.

It's a distribution parameter of the Gaussian, so it's the expected average of samples as the number of samples approaches infinity.

The distrinction is that you could draw three samples, -2, 0 and -1, from a Standard Normal - the mean of samples would be -1, but the distribution mean is still 0.

From a network architecture PoV, it's a learnable transformation applied on top of samples from a Standard Normal, so you only need to learn the transformation and get sampling for 'free'.


Why does Alpha Zero's Neural Network flip the board to be oriented towards the current player?

I am not an expert in RL. I have been playing Go for some years.
Let's quote from AlphaZero's paper first:

Aside from
komi, the rules of Go are also invariant to colour transposition; this knowledge is
exploited by representing the board from the perspective of the current player (see
Neural network architecture).

In the Game of Go, the difference between Black and White except the board representation is the komi (the amount of points that Black has to compensate White in the final count for playing first). Except the presence of komi, there should be no difference in strategy under the same position if colours exchanged. In other words, given a state $s$ of black stones and white stones on the board, if the optimal policy of Black playing first is $\pi$, then if colours of stones on the board exchanged and it is White's turn, the optimal policy for White should be the same as $\pi$.
With this in consideration, there are at least 2 advantages of using a network that represents the board in the perspective of Self/Opponent rather than Black/White.
The first is that it prevents the network from the possibility of giving inconsistent strategies under two representations of the same state. Consider a network $f_\theta$ that accepts the board representation in the order of $(B,W)$, and a state $s = (X_t,Y_t)$ in which $X_t$ is a feature map for black stones and $Y_t$ is a feature map for white stones and it is black's turn. Now consider a state $s' = (Y_t,X_t)$ (i.e. colours flipped) and it is white's turn. $s$ and $s'$ are essentially representation of the same state (except Komi which does not affect optimal policy). There could be a possibility that the network $f_\theta$ gives different policies for these two representations. However, if $f_\theta$ accepts the state as $(Self,Opponent)$, the input to the network would be the same (except the komi feature).
Therefore, this representation would significantly reduce number of states represented by the features vector $(X_t,Y_t)$, which would be the second advantage to training the neural network. If we consider that in Go, the same local position could appear in exchanged colour in another position, the network could by this implementation, recognize them as the same position. A decrease in the number of states could mean a significant drop in parameters and power needed from the network.
The same principle of making use of different representations of the same state is followed in AlphaGo's other training implementations as well, such as augmenting its training data to include rotations and reflections of the same board position.
However, in the game of Chess, this would be a different case. For a chess position, if the pieces' colours are exchanged and it becomes opponent's turn, it would be a different state because the positions of the KING and the QUEEN are not the same for the two colours.


Have any AI's been able to decode human vision 'thoughts'

There have been studies in University of Oregon and Kyoto University to be able to visualise thoughts and dreams on a screen using voxel values of an FMRI scan as input and an estimation of an image of the thoughts as the output. Instead of linking you to these studies and papers - you could just watch this episode of mind field where both these studies are demonstrated and linked.

The idea behind this is easier to understand if you have a good understanding of generative networks such as generative adversarial networks or so. Essentially in GAN's you'd map a known latent distribution to images in pixel-space. You would be doing the same thing here, just that the latent distribution would now be the FMRI scan input and the mapping would be made in a supervised setting where they are initially showed images. A very rough understanding of the idea can be drawn on these lines.


How much of a problem is white noise for the real-world usage of a DNN?

The white noise that fools DNNs isn't really white noise. It has been altered in the same way as the synthetic misclassified pictures have been altered. You have to change many input pixels in exactly such a way, that these little changes aren't perceptible, but propagated through the network add up to a misclassification. This is not going to happen by chance.


Weighing connections between two users using machine learning

The question might be improved so that the answer can be more specific by defining two things more rigorously.

Strength of the connection
What the automation must infer from that value

Learning, Correlation, Projection, and Maximizing Profitability

Guessing from the comments, it appears that the goal is to profile the user for business purposes.  Two common applications of user profiling are

To present buying options or
To suggest friends in a social network.

Although it may appear that a learning program would serve both of these applications, they are actually two very different problems.

One is to optimize e-commerce profitability through increased sales per session.  The other is to build a social network where there are some similarities and perhaps a few deliberate differences in the pairing of user profiles from which the friend suggestion comes.

The first of the two has to do with the projecting of the potential of a sale by matching the users and relying on the assumption that two people with like interests are more likely to mirror one another in purchasing patterns to some degree that could be projected with statistical analysis.

But approaching the project by immediately jumping to learning algorithms may not produce the best results after a month or a year of development.

One Reasonable Approach for Some Problems of This Type

Depending on the definitions of STRENGTH and AUTOMATICALLY in the question, it may be a simple application of a probability matrix or hypercube that will create a functional application of user profiling based on interests. To create a correlation between users via an ordered list of interests to place in one of the cells of the matrix or hypercube, the Spearman's Rank Order Correlation is a generalized yet effective method for such applications.

Learning Usually Begins With a Guess

First, you must determine what is (preferably in precise mathematical terms) what is a favorable product presentation or favorable potential friend introduction.  After that is determined, one can develop a probability formula to project what would be favorable based on the correlation matrix or hypercube.  Bayes's Theorem may be quite applicable in the development of a probability formula from the constraints and relationships of the application.

Learning Added

Once improved favorablility gained through the application of the correlations  can be demonstrated, it would then be wise to apply a learning algorithm.  The sequence is thus.

Results gained by random selection of options to present the user
Results gained by using the correlation matrix or hypercube
Results gained by iteratively converging on an optimal presentation
Tuning the iterative convergence track change and yet not oscillate

A multi-tier, back propagating Hopfield neural net could be one option for varying parameters in the use of the correlations (or the choice of correlations, since Spearman's is only one correlation model for ordered discrete interests).  Neural networks are only one type of convergent approach too.  Modelling with as a system of differential equations and using extensions of the Taylor Series to evaluate with a defined independent input vector is another.

To see how simple convergence can be and how primitive what appears to be a learning algorithm can become, think about the divide and average method of iteratively determining a square root.


Which NN would you choose to estimate a continuous function $f:\mathbb R^2 \rightarrow \mathbb R$?

It depends on the complexity of your problem. $\mathbb{R}^2 \rightarrow \mathbb{R}^1$ looks simple, but I can give you some nonsense complicated examples that need a deep network. So, the complexity of the problem sets the number of layers and neurons. The kind of problem will determine the architecture of your network (if it needs memory or not). In most cases, the mean squared error is ok. However, for the activation function, I would go with the ReLU.
If the SVM is good enough for your problem, then go with it. Your question is general and an exact answer needs more information about the problem.


Need examples for the following definitions

The 'vocabulary' used is:

$\mathscr{F}$ - fluents, conditions that change over time. The predicate holds(<fluent>,<state>) defines whether a condition applies in a given state.
$\mathscr{A}$ - actions, describe, erm, actions that happen. The predicate occurs(<action>,<state>) specifies that a given action happens at a particular state (presumably triggering a change to a new state, and changing the value of one or more fluents)
$\mathscr{T}$ - time points, which specify a chronological ordering.

Now we can look at the actual definitions: a discourse combines events (actions) and facts (fluents) and provides a (partial) ordering, ie a sequence in which they occur. This is represented in an acyclic graph. So the discourse describes a sequence of events that happen in a particular order and have an effect on facts. The ordering is relative, as there are no absolute time reference points give. Actions and facts are expressed in predicate calculus by the predicates mentioned above. Actions can involve a character moving from location A to location B; the fluent describing the location of the character will have been changed through that action.

An embedding assigns a given point in time to each state in the discourse. Each state is assigned a time point in a way that is consistent with the relative timings of the states in the discourse. So you have a general story, and you fix each action in time. The absolute time of an action in the embedding has to be before/after other actions according to their order in the discourse.

A domain adds two further predicates to a discourse, namely static($\phi$) and causes($\phi$,<fluent>). These formulas $\phi$ express logical constraints, that add consistency to the discourse. For example, if an action involves killing a character, the consequence has to be that the character is dead afterwards. So that action mandates a particular change, which would be encoded by such a formula. If the character was then to do some action after having been killed, that would invalidate the discourse in that domain. Different domains might have different constraints etc.

A model then works out if the domain is consistent, by evaluating all the actions and events using the domain constraints, and assigning it a truth value depending on whether it is possible or not. I might be wrong there, as I'm not sure I have interpreted the definition correctly — please let me know in a comment if that is the case.

Basically, if a model for a domain exists, it means that the actions and facts are consistent, and hence the narrative works (it is "a discourse compatible with a given domain"): a narrative is a discourse in a domain which, when embedded in a timeline, is consistent.

A default domain is one domain out of a set of other domains which seems preferable over the others.

(I have to stop here for lack of time — I might be able to expand this answer later)


Why don't we decorrelate transitions for policy-based data?

We do decorrelate training experience, even for policy gradient methods. This is because decorrelation helps training data be more like IID data, which helps with the convergence of SGD-like optimizers.

The shuffling is done on line 151 of OpenAI's "baselines" implementation of PPO.

I'm going to guess that there's a bug somewhere in your implementation. If you're using the same truncated advantage estimation as in the PPO paper,

$$\begin{align}
&\hat{A}_t = \delta_t + (\gamma\lambda)\delta_{t+1}+\dots+(\gamma\lambda)^{T-t+1}\delta_{T-1}\\
&\text{where}\quad\delta_t=r_t+\gamma V(s_{t+1})-V(s_t)
\end{align}$$

make sure you're not shuffling experience before computing your advantage estimates $\hat{A}_1,\dots,\hat{A}_T$. Each of these estimates is a function of several unbroken steps of experience. After you compute your advantage estimates, then create the tuples $(s_t, a_t, \hat{A}_t)$ and shuffle and sample from those. I think that's all the information you need  per time step for constructing their objective function.


What is the difference between batch and mini-batch gradient decent?

It is really simple.

In gradient descent not using mini-batches, you feed your entire training set of data into the network and accumulate a cost function based on this full set of data. Then you use gradient descent to adjust the network weights to minimize the cost. Then you repeat this process until you get a satisfactory level of accuracy. For example, if you have a training set consisting of 50,000 samples, you would feed all 50,000 samples along with the 50,000 labels into the network, then perform gradient descent and update the weights. This is a slow process because you have to process 50,000 inputs to do just one step of gradient descent.

To make things go faster instead of running all 50,000 inputs through the network, you split up the training set into "batches". For example, you could break the training set up into 50 batches each containing 1000 samples. You would feed the network the first batch of 1000 samples, accumulate the loss value then perform gradient descent and adjust the weights. Then you feed in the next batch of 1000 samples and repeat the process. So, now, instead of only getting one step of gradient descent for 50,000 samples, you get 50 steps of gradient descent. This method of using batches leads to a much faster convergence of the network.


Binary mode or Multi-label mode is correct when using binary crossentropy and sigmoid output function on multi-label classification

Since you have a multiclass classification problem rather than a binary classification problem (i.e. a two-class problem), I recommend to adjust your architecture and use softmax instead of sigmoid as final activation function and categorcal_crossentropy instead of binary_crossentropy.
Softmax will ensure all your outputs are valid probabilities. This is crucial in calculating the loss of your model. Using sigmoid will only ensure this if you have binary classes. Further categorical crossentropy is used for multiclass problems, again this is to make sure you calculate the correct loss. This is just a general model characteristic:
binary -> sigmoid & binary_crossentropy
multi-class -> softmax & (sparse_)categorical_crossentropy
To answer you original question, I think you should use average = micro, macro or weighted when calculating the F1-score. This depends on what you like your model to put more emphasis on and how severe your class imbalances are.


When training a CNN, what are the hyperparameters to tune first?

Firstly when you say an object detection CNN, there are a huge number of model architectures available. Considering that you have narrowed down on your model architecture a CNN will have a few common layers like the ones below with hyperparameters you can tweak:

Convolution Layer:- number of kernels, kernel size, stride length, padding
MaxPooling Layer:- kernel size, stride length, padding
Dense Layer:- size
Dropout:- Percentage to keep/drop


What is the difference between same convolution and full convolution in terms of feature map size?

What happens to the size of output feature map in case of full convolution?

It increases.

First one is valid padding: the blue square is not padded, so the green square is smaller.  Third one is same padding: the blue square is padded just enough so that the green square is the same size.  Fourth one is full padding: the blue square is padded as much as possible for that size of filter, so the green square is larger.
From here.


How to construct Transformers to predict multidimensional time series?

There is an implementation of the paper ("Adversarial Sparse Transformer for Time Series Forecasting"), in Python using Pytorch, here. Although it has the training and evaluation functionality implemented, it appears to be lacking a function for running a prediction.  Maybe you can fork it and extend it.
UPDATE
There is also a paper, "Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting", by Zhou et al., which does forecasts on univariate and multivariate data.  Their code is here.


Use of machine learning for analyzing companies enlisted in stock market

In one way the answer is NO. You can't incorporate all the necessary details in an AI program to correctly predict the financial market, at least not in currently. A particular event is a result of many actions and events from the past. Everything is like a chain.
You might have heard of the ButterFly Effect. It simply states that small things can have very large effects. Like flap of butterfly's wings leading to a hurricane. You can read more about it here.
Now back to your question. I believe you are somewhat familiar with the financial system and stocks. Close your eyes and try to imagine all the data that is being churned into machines. Apart from that, there are many decisions that lead to changes in the financial market directly. Like policies made by the government deals bagged by the company related to imports and exports, death of the main employee (like CEO, founder), and other many things. Apart from that, there are uncountable things that indirectly affect the market. I want you to think about it for a while. Now consider doing this for all the stocks present. After even stock values of one are dependent on the others in one way or another.
This amount of data, in other ways considering all these things, is not possible for current generation computers to process. You might have a hit with quantum computers coming though.
Now what you can do, is try to make a Deep Network (with Reinforcement Learning) that can predict the behavior and shift in the direction of some stocks. For example, whether it will go up for down or try to keep a constant line on the graph.
Selection of Features is always an important key to the success of Machine Learning algorithms. Another problem that stock market-related AI programs face is that Economics is itself an emerging and novice field of science. It's not as mature as physics, chemistry or biology. Thus finding out and deciding over the choice of features is another difficult job.
I found this paper online. Haven't given a look to it but, I believe this might help you a bit.


Bayes Error Rate
For the general case of K different classes, the probability of classifing x instance correctly is:
\begin{equation} \label{eq1}
\begin{split}
P(correct) & = \sum_{i=1}^{K} p(x \in H_i, C_i) \\
 & = \sum_{i=1}^{K} \int_{x \in H_i} p(x,C_i) \, dx\\
 & = \sum_{i=1}^{K} \int_{x \in H_i} P(C_i|x)p(x)\,dx \\
\end{split}
\end{equation}
where $H_i$ is the region where class $i$ has the highest posterior. So the Bayes Error Rate is:
\begin{equation} \label{eq2}
\begin{split}
P(error) & = 1 - p(correct) \\
 & = 1 - \sum_{i=1}^{K} \int_{x \in H_i} P(C_i|x)p(x)\,dx
\end{split}
\end{equation}
Be careful if we drew $x$ from $H_i$, the region covered/classified by $C_i$, $P(C_i|x) \ne 1$, because that would mean that we surely predict correctly all the time. If $x$ belongs to a decision area, this does not imply that it belongs to the corresponding class.


How is the F1 score calculated in a question-answering system?

In QA, it's computed over the individual words in the prediction against those in the True Answer. The number of shared words between the prediction and the truth is the basis of the F1 score: precision is the ratio of the number of shared words to the total number of words in the prediction, and recall is the ratio of the number of shared words to the total number of words in the ground truth
Source: https://qa.fastforwardlabs.com/no%20answer/null%20threshold/bert/distilbert/exact%20match/f1/robust%20predictions/2020/06/09/Evaluating_BERT_on_SQuAD.html#F1


How can I train a deep learning model to predict a matrix?

You could use a CNN or Fully Connected Network and output a matrix of size 8*2*2. The first 8*2 matrix is the real number and the second is the imaginary number. Example code below uses keras.

'''
input: numpy array of shape(batch_size,input_dim_0,input_dim_1,1)
Y: keras tensor of shape(batch_size,8,2,2)

'''

model = Sequential()
model.add(Conv2d(64,3,input_dim =(input_dim_0,input_dim_1,1))
model.add(Conv2d(64,3)
model.add(Conv2d(2,3)

model.compile(loss='MSE',metrixs=['accuracy'])
model.fit(input,Y)


Why does the transformer do better than RNN and LSTM in long-range context dependencies?

I'll list some bullet points of the main innovations introduced by transformers , followed by bullet points of the main characteristics of the other architectures you mentioned, so we can then compared them.
Transformers
Transformes (Attention is all you need) were introduced in the context of machine translation with the purpose to avoid recursion in order to allow parallel computation (to reduce training time) and also to reduce drop in performances due to long dependencies. The main characteristics are:

Non sequential: sentences are processed as a whole rather than word by word.
Self Attention: this is the newly introduced 'unit' used to compute similarity scores between words in a sentence.
Positional embeddings: another innovation introduced to replace recurrence. The idea is to use fixed or learned weights which encode information related to a specific position of a token in a sentence.

The first point is the main reason why transformer do not suffer from long dependency issues. Original transformers do not relies on past hidden states to capture dependencies with previous words, they process a sentence as a whole, reason why there is no risk to loose (or 'forget') past information. Moreover, also multi-head attention and positional embeddings both provide information about the relationship between different words.
RNN / LSTM
Recurrent neural networks and Long-short term memory models for what concern this question are almost identical in their core properties:

Sequential processing: sentences must be processed words by words.
Past information retained through past hidden states: sequence to sequence models follow the Markov property, each state is assumed to be dependent only on the previously seen state.

The first property is the reason why RNN and LSTM can't be trained in parallel. In order to encode the second word in a sentence I need the previously computed hidden states of the first word, therefore I need to compute that first. The second property is a bit more subdole, but not hard to grasp conceptually. Information in RNN and LSTM are retained thank's to previously computed hidden states. The point is that the encoding of a specific word is retained only for the next time step, which means that the encoding of a word strongly affect only the representation of the next word, its influence is quickly lost after few time steps. LSTM (and also GruRNN) can boost a bit the dependency range they can learn thanks to a deeper processing of the hidden states through specific units (which comes with an increased number of parameters to train) but nevertheless the problem is inherently related to recursion. Another way in which people mitigated this problem is to use Bi-directional models, which encode the same sentence from two direction, from the start to end and from the end to the start, allowing this way words at the end of a sentence to have stronger influence in the creation of the hidden representation, but this is just a workaround rather than a real solution for very long dependencies.
CNN
Also convolutional neural network are widely used in nlp since they are quite fast to train and effective with short texts. The way they tackle dependencies is by applying different kernels to the same sentence, and indeed since they're first application to text (Convolutional Neural Networks for Sentence Classification) they were implement as multichannel CNN. Why do different kernels allow to learn dependencies? Because a kernel of size 2 for example would learn relationships between pairs of words, a kernel of size 3 would capture relationships between triplets of words and so on. The evident problem here is that the number of different kernels required to capture dependencies among all possible combination of words in a sentences would be enormous and unpractical because of the exponential growing number of combinations when increasing the maximum length size of input sentences.
To summarise, Transformers are better than all the other architectures because they totally avoid recursion, by processing sentences as a whole and by learning relationships between words thank's to multi-head attention mechanisms and positional embeddings. Nevertheless, it must be pointed out that also transformers can capture only dependencies within the fixed input size used to train them, i.e. if I use as a maximum sentence size 50, the model will not be able to capture dependencies between the first word of a sentence and words that occur more than 50 words later, like in another paragraph. New transformers like Transformer-XL tries to overcome exactly this issue, by kinda re-introducing recursion by storing hidden states of already encoded sentences to leverage them in the subsequent encoding of the next sentences.


Why is A* optimal if the heuristic function is admissible?

This is well covered in the corresponding chapter of Russell & Norvig (chapter 3.5, pages 93 to 99 (Third Edition)). Check that out for more details.
First, let's review the definitions:

Your definitions of admissible and consistent are correct.
An admissible heuristic is basically just "optimistic". It never overestimates a distance.
A consistent heuristic is one where your prior beliefs about the distances between states are self-consistent. That is, you don't think that it costs 5 from B to the goal, 2 from A to B, and yet 20 from A to the goal. You are allowed to be overly optimistic though. So you could believe that it's 5 from B to the goal, 2 from A to B, and 4 from A to the goal.
A tree search is a general search strategy for searching problems that have a tree structure: that is, it's never possible to "double back" to an earlier state from a later state. This models certain types of games, for instance, like Tic-Tac-Toe. The tree search does not remember which states it has already visited, only the "fringe" of states it hasn't visited yet.
A graph search is a general search strategy for searching graph-structured problems, where it's possible to double back to an earlier state, like in chess (e.g. both players can just move their kings back and forth). To avoid these loops, the graph search also keeps track of the states that it has processed.
For more on tree vs. graph search, see this answer.

Okay, now let's talk through the intuition behind the proofs.
We first want to show that

If $h(n)$ is admissible, A* using tree search is optimal.

The usual proof is by contradiction.

Assume that A* with tree search and an admissible heuristic was not optimal.

Being non-optimal means that the first complete path from the start to the goal discovered by A* (call this $q$) will be longer than some other path $p$, which A* explored up to some state $s$, but no further.

Since the heuristic is admissible, the estimated cost of reaching the goal from $s$ must be smaller than the true cost.

By 3, and the fact that we know how much it costs to reach $s$ along $p$, the estimated total cost of $p$, and thus the cost to expand $s$ must be smaller than the true cost of $p$.

Since the true cost of $p$ is smaller than the cost of $q$ (by 2), the estimated cost to expand $s$ must be smaller than the true cost of $q$.

A* always picks the path with the most promising total cost to expand next, and the cost of expanding the goal state is given by the total path length required to reach it.

5 and 6 form a contradiction, so our assumption in 1 must have been incorrect. Therefore A* must be optimal.

The graph search proof uses a very similar idea, but accounts for the fact that you might loop back around to earlier states.


Why not teach to a NN not only what is true, but also what is not true?

Yes this is done routinely. For example this is how the YOLO object detection and classifier system works, to give a real-world for example. In YOLO, the "non-object" classification is "background" i.e. any image segment that doesn't contain one of the types of object we are interested in.

In general, you can add an "other" class to any classifier, provided you have data examples that fit into "other" class to learn and some sense of how often "other" will occur in the production system you are aiming for. Whether you choose to do so depends on the purpose of the model.

Many toy and test models do not include an "other" category, because they are used in a closed way to assess how each machine learning system works. That includes the famous MNIST handwritten digits data set for instance, so if you read tutorials about that, there is an underlying assumption that the trained network will only be presented with other handwritten digits and its only task is to classify them. However, this is not a general assumption for machine learning classifiers in general, just related to the goal of using the MNIST data set.

Adding a new "I do not know" category does not increase the accuracy or performance of a system when that category is not important in the target production system. When such a category is required due to the nature of a task, then the performance metrics will be different - a system that has been trained with some negative examples will likely perform better in that case.


How does the region proposal method work in Fast R-CNN?

Yes, it is not specified because the region proposal algorithm did not change from R-CNN (the previous version from Fast R-CNN, however, in the next verion, Faster R-CNN, this algorithm is replaced by a CNN).
The region proposal algorithm you are looking for is called selective search. You can find in the R-CNN paper that the algorithm is described in "Selective Search for Object Recognition", I found a copy here.
The algorithm is based on a series of segmentation and aggregation techniques of the input image for generating the proposed regions. Check it out 4 iterations of segmentation & aggregation over the same input image to build the proposed regions.

All the algorithm is doing is just iterating over 4 steps:

Initial regions based on segmentation by pixel light intensity are obtained by applying a segmentation algorithm described in the paper. For example, given a picture of a shepherd with his sheep in the mountain, it is segmented by light intensity, and the image of Figure (a) is obtained.
Different regions are proposed based on the previous segmentation, Figure (e)
The similarity between the proposed regions is calculated using the formula proposed in equation 6
in Section 3.2 in the paper which is nothing more than an aggregate metric of the similarity of two regions based in 4 metrics: similarity in color, texture, size and fill (measures how well a region within another)
Add the regions based on similarity and get Figure (b). Then return to step two.

That is how iteratively you get all the images depicted.


On which data evaluate an object detection model ? (similar or real life data ?)

If you want to evaluate on real thermal image dataset, you can use this one.

Thermal Image dataset

is mAP a relevant metric when I want to show result to a client ? (e.g a client doesn't understand if I tell him "my model has a mAP=0.7")

  Mean Average Precision is the relevant metric but it's more technical. You can start explaining with False Positives and False
  Negatives in the predictions. In turn, it leads to Precision and Recall. It mostly depends upon your use case because there will always be a trade-off between them.


What does "death" intuitively mean in the paper "Death and Suicide in Universal Artificial Intelligence"?

The authors do actually give an English definition in terms of the well-known agent formulation of AI:

  We intend this usage to be intuitive: death means that one sees
  no more percepts, and takes no more actions.

It would seem that this becomes possible for a reinforcement learning agent such as AIXI in a formulation that uses semi-measures of probability (which need not sum up to 1), rather than the more traditional notion.


When and how to use a mix of loss functions for back-propagation?

Mixing loss functions is very possible.  For example, in the case of neural style transfer, there is a style loss and a content loss. Both of them are backpropagated through the network. The final loss used for the backpropagation is a weighted sum of the losses. In the case of style transfer, it ensures that the image generated is not only imitating the style of the style image, but also keeping the original content. Sometimes there is also a third loss called variation loss. This is also applied to a weight and summed. Note the weight is a hyperparameter and is not changed during training. Also, the loss should not measure the same purpose, but rather a different purpose that you want to optimize together. Example code using PyTorch:

loss1 = torch.nn.MSELoss(a,b)
loss2 = torch.nn.MSELoss(b,c)
loss = loss1 * alpha + loss2 * beta


Can ELMO embeddings be used to find the n most similar sentences?

I ended up finding this article which does what I'm looking for.
Below is the portion of code I adapted for my needs

from sklearn.metrics.pairwise import cosine_similarity

import tensorflow_hub as hub
import tensorflow as tf

elmo = hub.Module("https://tfhub.dev/google/elmo/2", trainable=True)

def elmo_vectors(x):
  embeddings=elmo(x, signature="default", as_dict=True)["elmo"]

  with tf.device('/device:GPU:0'):
    with tf.Session() as sess:
      sess.run(tf.global_variables_initializer())
      sess.run(tf.tables_initializer())
      # return average of ELMo features
      return sess.run(tf.reduce_mean(embeddings,1))

corpus=["I'd like an apple juice",
        "An apple a day keeps the doctor away",
         "Eat apple every day",
         "We buy apples every week",
         "We use machine learning for text classification",
         "Text classification is subfield of machine learning"]

elmo_embeddings=[]
print (len(corpus))
for i in range(len(corpus)):
    print (corpus[i])
    elmo_embeddings.append(elmo_vectors([corpus[i]])[0])

print ( elmo_embeddings, len(elmo_embeddings))
print(elmo_embeddings[0].shape)
sims = cosine_similarity(elmo_embeddings, elmo_embeddings)
print(sims)
print(sims.shape)


How can we print weights per iteration in a simple feed forward MLP for an specific class?

A common model used for this kind of classification task is to have one output neuron per class. So, for example, neuron 1 may have a loss function that is related to outputting "1" for examples of class 1, and "0" for examples of other classes. Neuron 2 may be asked to do the same, but for class 2 rather than class 1.

If you use a model of this kind, you can pull the weights for each neuron in the final output layer. It sounds like this is what you are being asked to plot.


Extracting multiple softmax values from image

It is not unheard of to share network weights with multiple output layers. I have seen it on DeepMind's Asynchronous Deep Learning paper, and I have also used it here.

The idea is to share all the layers and just have multiple outputs. However, this might decrease the accuracy of your networks, as is the usual performance VS accuracy trade-off.

To optimize this, just calculate the loss of both outputs and sum them when feeding the optimizer, like this

            self.target_policy_fast_t = tf.placeholder('float32', [None, a_size], name='target_policy_fast_t')
            self.loss_policy_fast = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=self.network.policy_fast_before_softmax, labels=self.loss_policy_fast_t))

            self.target_policy_slow_t = tf.placeholder('float32', [None, a_size], name='target_policy_slow_t')
            self.loss_policy_slow = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=self.network.policy_slow_before_softmax, labels=self.target_policy_slow_t))

            self.loss = tf.reduce_mean(self.loss_policy_fast + self.loss_policy_slow, name='loss')


What are some alternatives to the book "Artificial Intelligence: A Modern Approach"?

The following 2 books helped me understand the basics and guided me through my first AI / CI implementations.

Computational Intelligence: An Introduction by Andries P. Engelbrecht
It includes the most relevant developments in computational intelligence with good discussions on intelligence and artificial intelligence (AI).

Computational Intelligence: Concepts to Implementation by Russell C. Eberhart and Yuhui Shi
It is a comprehensive textbook, supported with lots of practical examples.


How does a single neuron in hidden layer affect training accuracy

The claim that Neural Network with a single hidden layer can model any functions is proven in Cybenko's Approximation by superpositions of a sigmoidal function.
https://link.springer.com/article/10.1007/BF02551274
check also: https://en.wikipedia.org/wiki/Universal_approximation_theorem
The thing is that the neural network using sigmoidal functions, which are non-linear functions can.


Is it beneficial to represent a neural net as a matrix?

It depends on the type of neural networks you are dealing with.

For medium sized neural nets, the matrix approach is a very good way to do quick computations and even backpropogation of errors. One can even exploit sparse matrixes for understanding the sparse architecture of some neural nets.

But, for very large neural nets, using matrix computations would be computationally very intensive. So, relevant methods like graph-based stores, etc are used for them depending on the purpose and the architecture.


How to avoid falling into the "local minima" trap?

There are several elementary techniques to try and move a search out of the basin of attraction of local optima. They include:

Probabalistically accepting worse solutions in the hope that this
will jump out of the current basin (like Metropolis-Hastings acceptance in Simulated Annealing).
Maintaining a list of recently-encountered states (or attributes thereof) and not returning
to a recently-encountered one (like Tabu Search).
Performing a random walk of a length determined by the current state of the search (an explicit 'Diversification strategy', e.g. as used in 'Reactive Tabu Search').

See the excellent (and free online) book 'Essentials of Metaheuristics' by Sean Luke for more details on these kind of techniques and some rules of thumb about when and how to use them.


That's definitely possible with Convolutional Neural Networks. You should follow Google's tutorial at:

  https://www.tensorflow.org/tutorials/deep_cnn#cifar-10_model

The tutorial will teach you how to build a deep learning model for classifying cars, trucks etc. It's an easy tutorial to follow.

Your model will build on images pixels. The classifier will know what a car looks like.


Analogy between reinforcement learning and web servers

To paraphrase: "Is there an analogy between client/server in web development and agent/environment in reinforcement learning?"

The answer is "not really". There is no useful analogy here that allows any insight into RL from web server knowledge or vice versa.

However, you could set up an agent where the goal was to collect information, and the available actions were to make web requests. Clearly in order to to this, you would need to make use of the client/server model for web servers, with the agent having control over the client web requests, and the environment being the network and servers of the world wide web.

There are some very hard challenges to construct an open-ended "web assistant" agent. Here are a couple that I can think of:

How to describe actions? Starting with raw web requests composed as strings would likely be very frustrating. Probably you would simplify and have a first action be a call to a search engine with some variation of the topic description, and then decisions about which links to follow, or perhaps whether to refine the search to better fetch sites related to the topic as it is being built.
How to create a model of reward for collecting information? The first major stumbling block would be to measure the amount of useful information that the agent had found on any request.

I think with current levels of Natural Language Processing, setting an agent free to discover information according to some goal from a text topic description is a very hard task, beyond cutting edge research. It would definitely be unreasonable to expect any such agent to end up with any resemblance to "understanding" subject matter from text. The agent would have very little ability to tell the difference between accurate facts or lies, or just grammatically correct gibberish.

One interesting idea for agents trying to learn unsupervised from exploring an environment is creating a reward signal from data compression. An agent will have learned something useful about its environment, if on processing new information, it is able to compress its existing world model more efficiently. This is basic concept behind ideas of general learning agents from Jürgen Schmidhuber, Marcus Hutter and others. Research into this idea could be a driver towards creating more generic AI learning systems - however, it is one idea amongst many in AI, and so far is research-only, it has not yet led to anything as practical as an AI web-searching assistant.


Why is exact inference in a Bayesian network both NP-hard and P-hard?

It's not completely clear from your question, but it looks like you want to prove that exact inference in a Bayesian Network is both NP-Hard and P-Hard.

It appears that you have proven that it is NP-Hard, but are unsure how to show that it is also P-Hard.

This is more of a TCS question than an AI question, but shouldn't be too difficult. You just need to pick a P-Complete problem and reduce it to BN.


Do good approximations produce good gradients?

In general $|f(x) - f_k(x)| \leq \epsilon$ doesn't ensure $|\nabla f(x) - \nabla f_k(x)| \leq c\epsilon$. And for neural networks there is no reason to believe it will happen either.

You can also look at Sobolev Training Paper (https://arxiv.org/abs/1706.04859). In particular, note that Sobolev training was better than critic training, which indirectly may indicate approximating function may not be the same as approximating gradient and function. In Sobolev training, the network is trained to match gradient and function whereas in critic training network is trained to match function. They produce quite different results which might give us some hints about the above problem.

In general, if two functions are arbitrary close, they might not be close in gradients.

Edit:
(Trying to come up with a negative example)
Consider $f(x) = g(x) + \epsilon \sin (\frac {kx} {\epsilon}) $. $g(x)$ is some neural network. Now, we train some another neural network $h(x)$ to fit $f(x)$ and after training we get $h(x) = g(x)$ ($h(x)$ and $g(x)$ have same weights precisely). However, $\nabla f_x =\nabla g_x + k\cos (\frac {kx} {\epsilon})$ is not arbitarariy close $\nabla g_x$.

I hope this example is enough to prove that a neural network that nicely approximates the function may not nicely approximate the gradients and no such result can be proved mathematically rigorously. However, considering the paper in the discussion, you might think for practical purposes it works. However, if you have both function and grad information available that is expected to work better.


Why do very deep non resnet architectures perform worse compared to shallower ones for the same iteration? Shouldn't they just train slower?

In theory, deeper architectures can encode more information than shallower ones because they can perform more transformations of the input which results in better results at the output. The training is slower because back propagation is quite expensive, as you increase the depth, you increase the number of parameters and gradients that need to be computed.

Another issue that you need to take into account is the effect of the activation function. Saturating functions like sigmoid and hyperbolic tangent result in very small gradients in their edges, other activation functions are just flat, eg. ReLU is flat on the negatives therefore, there is no error to propagate because the gradient is either very small (as in saturating functions) or zero. Batch Norm greatly assists in this operation because it collapses values in better ranges where the gradients aren't close to zero.


How to deal with large (or NaN) neural network's weights?

This problem is called exploding gradients, resulting in an unstable network that at best cannot learn from the training data and at worst results in NaN weight values that can no longer be updated.

One way to assure it is exploding gradients, is if loss is unstable and not improving, or if loss shows NaN value during training.

Apart from the usual gradient clipping and weights regularization that are recommended, I think the problem with your network is the architecture.

30 is an abnormally high number of nodes for 2 layer perceptron model. Try increasing number of layers and reducing nodes per layer. - This is under the assumption that you're experimenting with MLP's, because for the problem above, convolutional neural networks seem like an obvious way to go. If unexplored - definitely check out CNN's  for digit recognition, two layer models will surely work better there.

Hope this helped!


How to detect multiple playing cards of the same class with a neural network?

Although it was not crystal clear, we'll assume that by, "Multiple cards of the same class in one picture," is meant that multiple cards of identical suit and rank will be grouped together in the same example image but each card in the image will be selected from a unique deck.

That arrangement would only be fruitful if the objective of training was to classify, flag, or otherwise analyze the same kind of groupings after training. Otherwise, it would likely be most productive and efficient to first devise a way to divide up the images by deck so that the focus of learning is detection of the three dimensions or perhaps just the first two, depending on the intended use of the trained network.

Rank
Suit
Style


How to identify the face of a certain customer in a grocery store?

You might consider pre-training a CNN on a large dataset. The CNN should be structured such that you input 2 different images and the CNN predicts whether or not they are the same person. Your dataset should include images from multiple angles, with and without occlusions like sunglasses, and with changes in hair. (One dataset useful for this is the AR Face Database.) Then you can just check if the face matches any of the pictures you took as they entered the store.


Why L1/L2 regularization technique did not improve my accuracy?

Regularization is one of the important prerequisites for improving the reliability, speed, and accuracy of convergence, but it is not a solution to every problem.  Irregularity in data is only one of many root causes for slow or otherwise inadequate learning results, and as the results in the question indicates, it can reduce reliability, speed, or accuracy in some cases.

For those who are new to this topic, these are a few other root causes.

Complexity beyond the capacity of the computing machinery to model adequately
Insufficient number of examples for training
Poor distribution alignment between data sets (training, testing, validation, production)
Saturation of back propagation values in floating point
Outliers cause by error in example creation or labeling
Local optimization minima in the loss function combined with insufficient stochastic injection in SGD
Leaning to heavily on artificial network convergence and neglecting using other known algorithms as part of an overall system architecture
Using activation functions or hyper-parameters that are not well tuned to the example set or the model

In the case of regularization, L1 and L2 have known properties that have been proven in theorem form to guarantee convergence in fewer examples or epochs, but they rely on very specific models toward which convergence is targetted.  L1 and L2 are not always beneficial for polynomial models.  Trying to make them so tends to lead to over-fitting.

The resulting trained networks lack the needed generalization to produce reliable behavior upon receiving data outside the training set.  Irregularities might not always be caught during test and verification.  Deployed systems can exhibit the signs of over-fitting over time because the network learned time-specific features.

Overfitting can be prevented using dropout regularization, another technique first proposed in Dropout: A Simple Way to Prevent Neural Networks from Overfitting by Srivastava, et al. 2014.

Another two things to look at ...

Is the logistic function is the best choice and for which layers?  It seems to be down-trending.  More people are using ReLU and its softer varieties because of higher performance shown in a number of problem domains and models.
Is the noise injection clean in that it is a good pseudo noise source and is it configured properly with whatever SGD methods and libraries being used.

I am training a Multilayer Neural Nets with 146 samples (97 for training set, 20 for validation set and 29 for testing set). I am using: automatic differentiation, SGD method, fixed learning rate + momentum term, logistic function, quadratic cost function, L1 and L2 regularization technique, adding some artificial noise 3%.

The question indicated the best results for the current training set and model.

  The best result was obtained using a lambda of 0.001 without regularization.

That's the starting point for improving further.  Since complete automated convergence is not yet something libraries and frameworks provide, the engineer normally has to converge on the combination of techniques and settings that best help the network's convergence.


What is the relationship between robustness and adversarial machine learning?

A robust ML model is one that captures patterns that generalize well in the face of the kinds of small changes that humans expect to see in the real world.

A robust model is one that generalizes well from a training set to a test or validation set, but the term also gets used to refer to models that generalize well to, e.g. changes in the lighting of a photograph, the rotation of objects, or the introduction of small amounts of random noise.

Adversarial machine learning is the process of finding examples that break an otherwise reasonable looking model. A simple example of this is that if I give you  a dataset of cat and dog photos, in which cats are always wearing bright red bow ties, your model may learn to associate bow ties with cats. If I then give it a picture of a dog with a bow tie, your model may label it as a cat. Adversarial machine learning also often includes the ability to identify specific pieces of noise that can be added to inputs to confound a model.

Therefore, if a model is robust, it basically means that it is difficult to find adversarial examples for the model. Usually this is because the model has learned some desirable correlations (e.g. cats have a different muzzle shape than dogs), rather than undesirable ones (cats have bow ties; pictures containing cats are 0.025% more blue than those containing dogs; dog pictures have humans in them more often; etc.).

Approaches like GANs try to directly exploit this idea, by training the model on both true data and data designed by an adversary to resemble the true data. In this sense, GANs are an attempt to create a robust discriminator.


How to best make use of learning rate scheduling in reinforcement learning?

I have not used learning rate schedules, but I do have experience with adjustable learning rates.
The Keras callback ReduceLROnPlateau is useful for adjusting the learning rate. If you use it to monitor the validation loss versus training loss, you will avoid the danger of overfitting. Also, you can use the ModelCheckpoint callback to save the model with the lowest validation loss and use that to make predictions. The documentation is here.
I look at the validation loss as a deep valley in $N$ space, where $N$ is the number of trainable parameters. As you progress down the valley, it becomes increasingly narrower, so it is best to reduce the learning rate to get further down the valley (closer to the minimum). With the adjustable learning rate, you can start with a larger initial rate that converges faster, then reduces as needed to achieve a minimum loss.
I wrote a custom callback that initially monitors the training loss and adjusts the learning rate based on that until the training accuracy achieves 95%, then it switches to adjusting the learning rate based on validation loss.
I also am experimenting with a slightly different approach to training. On a given epoch, assume the quantity you are monitoring does NOT improve. That means that you have moved to a point in $N$ space (value of the weights) that is NOT as "good" as the point you were at in the previous epoch. So, instead of training from the point you are at in the current epoch, I set the weights back to what they were for the previous (better) epoch, reduce the learning rate then continue training from there. This appears to work rather well.


Issue with graphical interpretation of the universal approximation theorem

The classical version of the universal approximation theorem states that, roughly, given a continuous function $f \colon [0, 1]^n \to [0, 1]^n$, there exists a single layer neural network and a set of weights and biases such that this network approximates the given function $f$ arbitrarily well.
It doesn't say anything about how you obtain such weights: the result is entirely independent of the way you train your network, and what it says is that the set of single-layer neural networks has enough capacity, in principle, to approximate any continuous function arbitrarily well.
This kind of result is fairly common in mathematics: it is shown that a suitable object must exist, but the proof is non-constructive, i.e. it doesn't actually show you how to get that object.
You're indeed correct that the backpropagation algorithm might not be able to find suitable weights. In fact, if you fix an architecture in advance, there might not even be any weights that would lead to a neural network that is a good approximation to $f$.
Why do we care?
Universality tells us, at least in principle, that we should be able to approximate the function we want, if we pick an appropriate architecture. The XOR Problem, for example, affects a type of model called a perceptron (essentially a single neuron neural network), and says that the XOR function cannot be approximated by perceptrons. It is always useful to know the kinds of functions that can and cannot be expressed by a certain type of model.


What should we do when the selection step selects a terminal state?

In the basic form, if you encounter a terminal leaf, you add visits and score depending on whether it is a win or loss, and backpropagate accordingly. The same as if you made a simulation step, but in this case the "simulation" is instant.

But you can improve that: If the leaf is losing, you can give it a very large negative score or even $-\infty$, so in the next selection step it surely won't be chosen, unless other moves are as bad. But if it is a winning leaf, you not only can give it a very big positive score or $\infty$, but also add a negative score for the immediate parent, so the parent won't be chosen, as it is obviously a losing state for him. This way we can save some simulations. I had encountered that situation many times in my game and Monte Carlo tree searches.

Suppose the parent has $200$ unexplored children, $10$ of which are immediate wins for the opponent. Your search may explore $100$ non-winning children and have a score of 80/100. But then it encounters the terminate child. After that, normally, it would have $80/101$, so, still, a big chance to be chosen in the next iteration. And it would take many iterations to see that this is not a good move, as it would need to get like $80/150$ or more. But if we cancel out the score or give it a negative one like $-1/101$, then we ensure it won't be chosen.

It seems in literature it's called "MCTS solver", to backpropagate proven wins and losses.


Why do I get small probabilities when implementing a multinomial naive Bayes text classification model?

You could say that the class of the data (e.g. spam vs not spam) is a hidden quality that can be inferred through the observable features (e.g. message subject contains "bitcoin"),
$$P(C \; | \; F)$$
which says that the probability of the class $C$ is conditioned on the visibility of the feature $F$.
Using Bayes' theorem we can write
$$P(C \; | \; F) = \frac{P(F \; | \; C) \, P(C)}{P(F)}$$
Your problem is that you are not taking into account the evidence, $P(F)$, denominator in this expression.

You can have multiple features in your likelihood function
$$P(F \; | \; C) = P(F_1,F_2,F_3,\ldots \; | \; C)$$
and these can be independent
$$\prod_k P(F_k \; | \; C)$$
and then you have your prior
$$P(C) \prod_k P(F_k \; | \; C)$$
but what you are missing is the evidence factor in the denominator
$$P(C \; | \; F) = \frac{P(C) \prod_k P(F_k \; | \; C)}{\sum_C \scriptsize{P(C) \prod_k P(F_k \; | \; C)}}$$
This is the general view of the problem. Specifically, to fix your issue, evaluate the probabilities for all your classes, add them together, then divide each probability by that sum.
For instance, if you have two classes and the probabilities are $p_0$ and $p_1$, then write
$$P(C=0) = \frac{p_0}{p_0 + p_1}$$
and
$$P(C=1) = \frac{p_1}{p_0 + p_1}$$
which guarantees that
$$P(C=0) + P(C=1) = 1$$


In principle, yes, you can also evolve the genetic algorithm (or, in general, evolutionary algorithm), i.e. you can evolve its operations (such as the mutation and cross-over) and hyper-parameters (such as the size of the population or mutation rate). For example, you could use genetic programming to evolve the cross-over operation of a genetic algorithm. However, these genetic operators and hyper-parameters are usually designed and determined by a human and do not change during the evolution process. Nevertheless, there is a framework for evolutionary computation in which reproduction and mutation can also evolve, known as auto-constructive evolution, and there are other examples in the literature of meta-optimization methods applied to evolutionary algorithms, such as

Evolutionary principles in self-referential learning, or on learning how to learn: The meta-meta-... hook (1987, Schmidhuber's thesis)

Parallel optimization of evolutionary algorithms (1994, by T. Bäck)

Adaptive and Self-adaptive Evolutionary Computations (1995, by Peter Angeline)

Meta-Genetic Programming: Co-evolving the Operators of Variation (1998, by B. Edmonds)

More generally, the optimization of an optimization algorithm is known as meta-optimization and/or hyper-parameter optimization/tuning, so the idea of meta-optimization is not just restricted to evolutionary algorithms, but can be applied also to e.g. deep learning.


In a neural network given partial inputs and complete outputs, is it possible to predict remainig inputs

It's important to understand that though neural networks generalize to the whole input space, usually the meaningful input space, from which the training data is taken, is a manifold inside that space.

For example, the image classifier can take any image, including white noise, but it's usually trained on photos, which generally have some nice structure and statistical properties, e.g. smoothness of color, and is expected to work well on photos, not everywhere.

So your question can be reformulated like this: can we learn the data manifold given the neural network and some partial data?

If we ignore deliberately crafted small examples (like ones in which the output just equals to one of features) and consider real world, high-dimensional tasks, we see that the manifold is quite resistant to small perturbations. In high-dimensional spaces a change of one dimension doesn't drastically change the distance to the manifold, it's still very very close. Example: a change of a single pixel is very unlikely to affect the output, hence it would be impossible to predict it. The larger the space is, the less is influence of each individual dimension.

Hope this answers your question.

I'd like to mention here one very promising technique called GAN, which basically does this: it uses one neural network for classification and the other to learn the data manifold. And it works so good that it'd be fair to say that both these networks actually complement each other, not derive from one another.


I will assume that the camera is stable (no change in position, zoom or other settings during the video recording), otherwise the task becomes markedly more complicated.

Let's say that your dataset is an array of rasters (images in array format). You mention that you want to detect events "start line" and "end line".

One way of doing this would be to compute an approximate time derivative of your image series. For instance, take the image raster at index idx and the one right after, at index idx+1 (captured at instants $t$ and $t + \Delta t$, where $\Delta t$ is the sampling interval).

At coordinates $(i,j)$, this derivative could look something like: timeDerivative = (images[idx][j][i] - images[idx+1][j][i])/DeltaT. This is a crude estimate, and there are better ways of computing an approximate discrete derivative, but you get the idea.

Following this, you could declare a state of the recording: drawing line or not drawing line. The states, we assume are always alternating, as a teacher has to take his hand off the blackboard to draw a new line.

When a derivative with large values is detected (a region in the image goes from being black to white suddenly) and the state is "not drawing", the event "start line" is recorded and the state switches to "drawing". While a derivative with large values continues to be detected as time goes on in the vicinity of the previous spot with a large derivative, nothing changes. Once this is no longer true, the state changes to "not drawing" and the event "stop line" is recorded at the last location with a large derivative.

This is the main idea, which can be improved with:

Defining the area of the blackboard, either by hand or automatically
Thresholding the images to better isolate the chalk trace from the blackboard
Using a tracker such as a Kalman filter to know where to look next for the chalk


What is the intuition behind importance sampling for off-policy value evaluation?

Recall that our goal is to be able to accurately estimate the true value of each state by computing a sample average over returns starting from that state:
$$v_{q}(s) \doteq \mathbb{E}_{q}\left[G_{t} | S_{t}=s\right] \approx \frac{1}{n} \sum_{i=1}^{n} Return_i $$
where $Return_i$ is the return obtained from the $i^{th}$ trajectory.

The problem is that the $\approx $ does not hold, since in off-policy learning, we got those returns by following the behavior policy, $p$, and not the target policy, $q$.

To address that, we have to correct each return in the sample average by multiplying by the importance sampling ratio.

$$v_{q}(s) \doteq \mathbb{E}_{q}\left[G_{t} | S_{t}=s\right] \approx \frac{1}{n} \sum_{i=1}^{n} \rho_i Return_i$$

where the importance sampling ratio is : $\rho=\frac{\mathbb{P}(\text { trajectory under } q)}{\mathbb{P}(\text { trajectory under } p)}$

What this multiplication does is that it increases the importance of returns that were more likely to be seen under the target policy $q$ and it decreases those that were less likely. So, at the end, in expectation, it would be as if the returns were averaged following $q$.

(A side note: To avoid the risks of mixing $p$ and $q$, it might be a good idea to denote/think of the behavior policy as $b$ and the target policy as $\pi$, following the convention in Sutton and Barto's RL book.)


What is the space complexity of bidirectional search?

Norvig & Russell's book (section 3.5) states that the space complexity of the bidirectional search (which corresponds to the largest possible number of nodes that you save in the frontier)
$$O(2b^{d/2}) = O(b^{d/2}).$$
The intuition behind this result is that (as opposed to e.g. uniform-cost search or breadth-first search, which have space (and time) complexity of $O(b^{d})$) is that the forward and backward searches only have to go half way, so you will not eventually need to expand all $b^{d}$ leaves, but only half of them.
However, this space complexity is correct if you use a breadth-first search for the forward and backward searches (which is your scenario!), given that breadth-first search, assuming a finite branching factor, expands one level at a time, so it's guaranteed that both the forward and backward searches meet in the middle. This can be seen in figure 3.17 of the same book, where you can see that both searches have the same "radius". Moreover, the only nodes that you need to store in the frontier are the ones on the circumference (not all nodes that you see in the image)

However, if you used another search algorithm to perform the forward and backward searches, the space complexity may be different. This is true if e.g. the searches do not meet and then they end up exploring all the state space.


Is a multilayer perceptron a recursive function?

Inherently, no. The MLP is just a data structure. It represents a function, but a standard MLP is just representing an input-output mapping, and there's no recursive structure to it.

On the other hand, possibly your source is referring to the common algorithms that operate over MLPs, specifically forward propagation for prediction and back propagation for training. Both of these algorithms are easy to think about recursively, with each node performing a sort of recursive call with its children or parents as the target, and some useful information about activations or errors attached. I actually encourage my students to implement it recursively for this reason, even though it's probably not the most efficient solution.


Is it possible to train deep learning agent to pass a multiple-choice exam?

There are programs that do this today, for some values of "curriculum" and "exam".  It does not even require deep learning; a simpler information retrieval algorithm and some rules for composition work and achieve high scores on machine graded essays.

For human graders, there is research on automatically generating essay-length text responses to queries in a certain domain.

Both linked applications are rule-based rather than based in deep-learning.  I'd guess that a deep-learning approach would be much less efficient (in computer resources) in producing comparable results.