You are correct in the question that in RL terms chess a game of chess where the agent is one player, and the other player has an unknown state is a partially observable environment. Chess played like this is not a fully observable environment.
I did not use the term "fully observable game" or "fully observable system" above , because that is not a reinforcement learning term. You may also read "game of perfect information" which is similar - it means there are no important hidden values in the state of the game which may impact optimal play. This is a different concern to understanding the state of your opponent.
Here is a counter-example showing that games of perfect information are not fully observable systems when you have an opponent with an unknown strategy:

Optimal play in tic tac toe leads to a forced draw.

Let's define a reward signal from the agent's perspective of +1 for a win, 0 for a draw, and -1 for a loss.

If the agent's opponent always plays optimally, then a RL agent will learn to counter that optimal play and also play optimally. All action choices will have an expected return of 0 or -1, and the agent will choose the 0 options when acting greedily.

If the agent's opponent can make a mistake that allows the agent to win, then there will be a trajectory through the game with a return of 1, or perhaps some other postive value in cases where the mistake is only made according to random chance.

The value of states in the game therefore depends on the opponent's strategy.

The opponent's strategy is however not observable - it is unknown and not encoded into the board state.

This should match your intuition when asking the question.
Why then, do many two player game-playing reinforcement agents for games like chess perform well, without using POMDPs?
This is because game theory on these environments supports the concept of "perfect play", and agents that assume their opponent will also attempt to play optimally - without mistakes - will usually do well. Game theory analyses choices leading to forms of the minimax theory - making a choice that your opponent is least able to exploit.
That does mean that such an agent may in fact play sub-optimally against any given opponent. For example, they could potentially turn some losing or draw situations into a win, but have little or no capability to do so unless trained against that kind of opponent. Also, playing like this may be a large risk against other opponents, it may involve playing sub-optimally at some earlier stage.
I have observed a related issue in Kaggle's Connect X competition. Connect 4 is a solved game where player one can force a win, and the best agents are all perfect players. However, they are not all equal. The best performers tweak their agent's choices for player two, to force the highest number of wins against other agents who have not coded a perfect player one. Different kinds of learning strategy lead to different imperfections, and the top of the leaderboard is occupied by the current best perfect agent that also manages to exploit the population of near-perfect agents below it in the rankings. This difference in the top-ranking agents is only possible due to the partially-observable nature of the Connect 4 game played against agents with unknown policies.

What exactly are partially observable environments?

They are environments where in at least some states, the agent does not have access to information that affects the distribution of next state or reward.
Chess played against an opponent where you have a model of their behaviour - i.e. their policy - is fully observable to the agent. This is implicitly assumed by self-play agents and systems, and can work well in practice.
Chess played against an opponent without a model of their behaviour is partially observable. In theory, you could attempt to build a system using a partially observable MDP model (POMDP) to account for this, in an attempt to try and force an opponent into states where they are more likely to make a decision that is good for the agent. However, simply playing optimally as possible in response to all plays by the opponent - i.e. assuming their policy is the same near optimal one as yours even after observing their mistake - is more usual in RL.
The original Alpha Go actually had a separate policy network for its own choices and modelling those of humans. This was selected experimentally as performing slightly better than assuming human opponents used the same policy as the self-play agent.


What is meant by a multi-dimensional continuous action space?

Let me rephrase it a little - it's a multidimensional continuous space of actions. So, you assign each action some vector from $R^{n}$. For intuition -- imagine you have a robot arm with four joints. For every joint you could applied a rotation force from [-1, 1] and thus you get a 4-D vector with float numbers for each possible action.


Understanding the perceptron algorithm in the book "A Course in Machine Learning"

Y is the desired output of the perceptron (often referred to as target) , for the given set of input vectors.

Rationale behind Y.a<=0 :

Prerequisite knowledge :

A=A-B : Moves vector A away from direction of vector B
A=A+B : Moves A in the direction of B
A (.) B >0 ; A vector is directed acutely (<90 deg.) towards B vector
A (.) B <0 ; A vector is directed away from (>90 deg.) from B vector [(.) denotes dot (scalar) product and bold letters indicate vectors]
W is augmented vector (includes threshold as another weight along with normal input weights)
X is augmented input (includes -1 as extra input (corresponding to threshold) along with other normal inputs
a(activation) = W (.) X
a >=0 ; Perceptron output 1
a<0 ; Perceptron output -1 (Not zero as implicit in the given algorithm I think)

Now Rationale :

(Y.a<0)

This means
Either of the following :

Y=-1 and a>0 ; in this case the target output is -1 but as a>0 so the Perceptron outputs 1. So we must move the weight vector away from this set of input vector, so that angle between them increases and the dot product (a) becomes < 0 so that we can get the target output.
Hence : W=W-X

Or ,W=W + (-1)*X
Or, W=W+YX

Y=1 and a<0

This means the target output is 1 but as the activation is <0 so Perceptron is outputting -1. So we must move the weight vector close to this set of input vector so that the activation can become >0 (angle decreases) and the Perceptron can output the desired output.
So :
W = W+X
Or W=W + YX

Again , Y.a=0 is a boundary case.
By now I think you can understand the rationale behind Y.a=0. If any doubt , comment to this answer , I will explain it.

Sorry for so much long answer though. :) :)


How to refine K-means clustering on a data set?

The usual parameters to adjust in a k-means:

Number of clusters (recall many clusters can have same label).
Distance definition (euclidean is the most basic, Gauss is an
improvement)
Selection of initial cluster positions.
Data preprocessing (data normalization, ...)


Approach to classify a photo and extract text from it

There are three functional aspects to this project, each of which could be a sub-project, even though they are interrelated. The decoupling of these aspects in the associated R&D will likely improve the rate of development.

Classify images — in this case, the vendor brand
Locate and orient the text
OCR the text — in this case, the total

These are certainly feasible in the particular case of invoices from vendors, so the entire project certainly is.

The goal is to produce two fields of information from input images.

Vendor code
Total in some specific monetary unit

That past invoices, the names of the vendors, and the totals of each invoice are available as training data may be helpful. Whether the volume of training data is sufficient can only be determined through a very complicated application of theory to some of the data metrics or by trial. It is recommended to do both.

  What algorithms would you guys recommend?

It would be irresponsible to define an AI design in terms of what people recommend for algorithm. The algorithms that guys recommend are already covered by the path names for files in the algorithms directory paths, the example's directory paths, and the sections of the documentation of any good AI framework. Let's first talk about the design and then the algorithm options, so there is more of a basis for algorithm selection than the largely random reading selections of the members of a site.

The models involved and which can be parameterized such that they can be tuned through training are partly defined by the three sub-projects above.

Analysis

For this project, there is no reason for the AI to recognize which text on the invoice is the vendor name. If there are a hundred vendor invoice types, the speed in which a single person can identify the key boundaries of each invoice type is orders of magnitude faster than the work necessary to develop a completely general algorithmic approach for automating that work. It would only be resource thrifty to develop such automation if there were thousands of vendors and constantly varying templates for invoices, which is likely in the case of personal finance.

These are the key bounds of the form quadrilaterals (four sided polygons) within which the following objects lie.

Vendor name text box
Total amount text box
Rectangle around the entire form

Digitizing the twelve points through one of the labelling programs is a model much more likely to produce a reliable system than the representation of three rectangular bounding boxes and tilts. This is because tilt angle requires at least two adjacent points anyway and the aspect ratio cannot be held constant in real scanning scenarios when the scanner might be replaced or produce different ratios with wear or the smoothness or wear characteristics of the paper invoice.

Doing all three things using a network would require more than one person's invoices for a decade, unless the person is a billionaire compulsive buyer with a team of secretaries doing scanning and data entry.

The models are then these.

Digitization of a quadrilateral document with varying contrast, brightness, tilt, horizontal pixels per inch, vertical pixels per inch, relative location and size of vendor name, and relative location and size of invoice total
Numeric characters and other monetary characters in a rectangular box
Brand image for the vendor, which may include a logo and type of arbitrary and possibly unique font

Now we can talk about artificial network approaches.

Adjusting for contrast, brightness, tilt, resolution, locations, and sizes will require a customized input, loss function, and layer arrangement based on the geometry involved. A GRU network may gain some advantages if the documents can be ordered chronologically, since the collection then becomes a time series, the trends of which can be exploited.
Monetary values can best be done using OCR libraries.
Recognizing the brand is probably best done with CNN as a categorization machine.

The system must indicate if the vendor and total are not found, in which case a new template is indicated, and the twelve points must be digitized for this new type.

It is possible to do this with one single deep convolutional network, but, again, the data set would need to be augmented. The one other way to do this is to create a filled-out-invoice generator to produce the volume of stochastically variable data across the various variability dimensions listed above to train the deep CNN.


How could emotional intelligence be implemented?

Architectures for recognizing and generating emotion are typically somewhat complex and don't generally have short descriptions, so it's probably better to reference the literature rather than give a misleading soundbite:
Some of the early work in affective computing was done by Rosalind W. Picard. There is a research group at MIT specializing in this area.
Some of the more developed architectural ideas are due to Marvin Minsky.
A pre-publication draft of his book, The Emotion Machine, is available via Wikipedia.
Emotional intelligence would certainly seem to be a necessary component of passing the Turing test - indeed, in the original Turing test essay in Computing Machinery and Intelligence implied some degree of "Theory of Mind" about Mr. Pickwick's preferences:

Yet Christmas is a Winter’s day, and I do not think Mr. Pickwick would mind the comparison.


What is the advantage of using more than one environment with the advantage actor-critic?

What is the advantage of using more than one environment with a single agent?

There are two main advantages to this approach:

The dataset for training is closer to the independent, identically distributed (i.i.d.) ideal, important for theoretical and practical reasons when training a neural network. Samples taken from a single trajectory are not independent, but instead are correlated due to rules of the environment - so using a single trajectory is furthest from i.i.d. This is a similar motivation to use of experience replay tables for DQN variants of Q-learning. However, experience replay is inherently off-policy, so not a good fit to A2C or A3C that need samples taken when acting under the current policy.
Collecting experience is often a major bottleneck in training RL agents. Being able to do so in parallel in a distributed environment can significantly speed up the training process.


Why do ResNets avoid the vanishing gradient problem?

Before proceeding, it's important to note that ResNets, as pointed out here, were not introduced to specifically solve the VGP, but to improve learning in general. In fact, the authors of ResNet, in the original paper, noticed that neural networks without residual connections don't learn as well as ResNets, although they are using batch normalization, which, in theory, ensures that gradients should not vanish (section 4.1). So, in this answer, I'm just giving a potential explanation of why ResNets may also mitigate (or prevent to some extent) the VGP, but the cited research papers below also confirm that ResNets prevent the VGP. Given that I didn't fully read all the papers mentioned in this answer, the information in this answer may not be fully accurate.
The skip connections allow information to skip layers, so, in the forward pass, information from layer $l$ can directly be fed into layer $l+t$ (i.e. the activations of layer $l$ are added to the activations of layer $l+t$), for $t \geq 2$, and, during the forward pass, the gradients can also flow unchanged from layer $l+t$ to layer $l$.
How exactly could this prevent the vanishing gradient problem (VGP)? The VGP occurs when the elements of the gradient (the partial derivatives with respect to the parameters of the NN) become exponentially small, so that the update of the parameters with the gradient becomes almost insignificant (i.e. if you add a very small number $0 < \epsilon \ll 1$ to another number $d$, $d+\epsilon$ is almost the same as $d$) and, consequently, the NN learns very slowly or not at all (considering also numerical errors). Given that these partial derivatives are computed with the chain rule, this can easily occur, because you keep on multiplying small (finite-precision) numbers (please, have a look at how the chain rule works, if you're not familiar with it). For example, $\frac{1}{5}\frac{1}{5} = \frac{1}{25}$ and then $\frac{1}{5}\frac{1}{25} = \frac{1}{125}$, and so on. The deeper the NN, the more likely the VGP can occur. This should be quite intuitive if you are familiar with the chain rule and the back-propagation algorithm (i.e. the chain rule). By allowing information to skip layers, layer $l+t$ receives information from both layer $l+t-1$ and layer $l$ (unchanged, i.e. you do not perform multiplications). For example, to compute the activation of layer $l+t-1$, you perform the usual linear combination followed by the application of the non-linear activation function (e.g. ReLU). In this linear combination, you perform multiplications between numbers that could already be quite small, so the results of these multiplications are even smaller numbers. If you use saturating activation functions (e.g. tanh), this problem can even be aggravated. If the activation of layer $l+t$ are even smaller than the activations of layer $l+t-1$, the addition of the information from layer $l$ will make these activations bigger, thus, to some extent, they will prevent these activations from becoming exponentially small. A similar thing can be said for the back-propagation of the gradient.
Therefore, skip connections can mitigate the VGP, and so they can be used to train deeper NNs.
These explanations are roughly consistent with the findings reported in the paper Residual Networks Behave Like Ensembles of Relatively Shallow Networks, which states

Our results reveal one of the key characteristics that seem to enable the training of very deep networks: Residual networks avoid the vanishing gradient problem by introducing short paths which can carry gradient throughout the extent of very deep networks.

In the paper Norm-Preservation: Why Residual Networks Can Become Extremely Deep?, the authors also discuss another desirable effect of skip connections.

We show theoretically and empirically that each residual block in ResNets is increasingly norm-preserving, as the network becomes deeper


Which fields of AI are actively researching consciousness?

I think we still have a long way to go before any progress is made on artificial consciousness. However, researchers are taking inspiration from traits of human consciousness. One relevant paper is Machine Theory of Mind by DeepMind. They show that their model can (at least to some extent) represent the desires, beliefs, and intentions of agents that it observes. It even passes a form of the Sally-Anne test, showing that it can represent the false beliefs of an agent.


Why do we prefer ReLU over linear activation functions?

The ReLu is a non-linear activation function. Check out this question for the intuition behind using ReLu's (also check out the comments). There is a very simple reason of why we do not use a linear activation function.

Say you have a feature vector $x_0$ and weight vector $W_1$. Passing through a layer in a Neural Net will give the output as

$W_1^T * x_0 = x_1$

(dot product of weights and input vector). Now passing the output through next layer will give you

$W_2^T * x_1 = x_2$

So expanding this we get

$x_2 = W_2^T * W_1^T * x_0 = W_2^T * W_1^T * x_0 = W_{compact}^T * x_0$

Thus as you can see there is a linear relationship between input and output, and the function we want to model is generally non-linear, and so we cannot model it.

You can check out my answer here on non-linear activation.

Parametric ReLu has few advantages over normal ReLu. Here is a great answer by @NeilSlater on the same. It is basically trying to tell us that if we use ReLu's we will end up with a lot of redundant or dead nodes in a Neural Net (those which have a negative output) which do not contribute to the result, and thus do not have a derivative. Thus to approximate a function we will require a larger NN, whereas parametric ReLu's absolve us of this problem,(thus a comparatively smaller NN) as negative output nodes do not die.

NOTE: alpha = 1 will be a special case of parametric ReLu. There must be a balance between the amount of liveliness you want in the negative region vs the linearity of the activation function.


What does it mean for a neuron in a neural network to be activated?

A neuron is said activated when its output is more than a threshold, generally 0.

For examples :
\begin{equation}
y = Relu(a) > 0
\end{equation}
when
\begin{equation}
a = w^Tx+b > 0
\end{equation}

Same goes for sigmoid or other activation functions.


What is the role of convex optimisation in AI systems?

Is there any role of convex optimization in AI?

Yes, of course!

  If so, in what algorithms or problem settings or systems?

The problem of finding the parameters of a support vector machine can be formulated as a convex optimization problem. Another example is linear regression.

See also the paper Convex Optimization: Algorithms and Complexity (2014) by Sébastien Bubeck, which also mentions SVM as a typical example.


The deeplearning.ai Natural Language Processing Specialization, is a great place to start. It'll brush up some basic concepts of NLP and then get into SOTA methods  such as Transformers. There's even a project in one of the courses about building a chatbot using transformers, it also has a ton of papers and other sources mentioned.

Rasa is a great software tool/package to get started with chatbots too. It's a great engaging community so your queries should be quickly answered.


How can a neural network approximate all functions when the weights are not allowed to grow exponentially?

What is Proven
The question references the proof of Approximation by Superpositions of a Sigmoidal Function, G. Cybenko, 1989, Mathematics of Control, Signals, and Systems.
The 1989 proof stated that the network, made of activations that were required to be, "Of continuous sigmoidal non-linearity," could, "Uniformly approximate any continuous function of n real variables," so, as the question stated, the proof doesn't directly apply to 1-bit discrete outputs.  Note that the network is expected to merely approximate the desired circuit behavior.
The question defines the system as an arbitrary mapping from input bit vector

$I: { i_1, \; \dots, \; i_n}$

to output bit vector

$O: { o_1, \; \dots, \; o_n}$

It was further back proven that such a mapping can be accomplished with one Boolean expressions for each output bit.  For all $2^n$ possible input vector permutations, there exists a Boolean expression made up of AND and NOT operations that calculates a result that matches any arbitrary logical truth table.
There are techniques for reducing redundancy in the array of Boolean expressions, which is critical to VLSI chip layout.
Without the retention of state anywhere in the network other than the attenuation matrix (parameters), the system is not Turing complete.  However, with regard to the ability to realize Boolean expressions in describing the mapping, given an arbitrary number of layers, the network is complete.
Estimating Layer Depth Requirements
Only one inner layer is required in the 1989 proof, so how many layers would it take for an accurate n-bit-to-n-bit mapping to be learned?
The question proposes that there are $2^n$ to the power of $2^n$ permutations.  The mapping of each input bit vector to the desired output bit state can be represented by a truth table of $n$ binary dimensions.
Each output is an independent bit, meaning the $2^n$-bit representations of unique Boolean functions that could produce each output bit is not tied to any other output channel.  As would be expected, there are $2n$ freedoms of motion for the mapping of I to O.
For the case where the input is a bit vector of $n$ bits, where $n$ is the number of activations for any one of $L$ layers, the total number of activations in the network $a_t$ and the total number of scalar elements for all attenuation matrices (the parameters that represent training state) $q_t$ for the network is as follows.

$a_t = \sum_{\, 0 = v}^{L - 1} \; n_v$

$= n \, L$

$p_t = \sum_{\, 0 = v}^{L - 2} \; n_v^2$

$= n^2 \, (L-1)$

If IEEE 64 bit floating point numbers are used for each element in the attenuation matrix, we can calculate the number of bits available in the training parametrization.

$b_t = 64 \, (L - 1) \, n^2$

It would be normal today to use ReLU, leaky ReLU, or some other more quickly convergent activation instead of sigmoid for all layers but the last and use a simple binary threshold for the last.
Thus we have a formulation of the information theory comparison inferred by the question, and can reduce it.

$2^{2n} \le 64 \, (L - 1) \, n^2$
$L \ge 1 + \frac {2^{2n-6}} {n^2}$

This is a rough threshold.  For a highly reliable training for the binary inputs to binary outputs, the number of layers should be well above the threshold.
Below the threshold the trainability of the mapping will degrade to an inadequate approximation for most applications because of signal saturation in the back propagation mechanism.


Showing first layer RGB weights similarly to AlexNet

It seems Alex has just used the Matlab function mat2gray, as described here: https://www.mathworks.com/help/vision/ug/image-category-classification-using-deep-learning.html
The visual outcome of the features is very similar. mat2gray will simply scale the weights between 0 and 1 (no clipping).
Leaving the (slightly adapted) example code of Mathworks here for future reference:
layer1weights = mat2gray(layer1weights) ;
layer1weights = imresize(layer1weights,5) ;

figure
montage(layer1weights)
title('First convolutional layer weights')


Why is the generation of deep style images so slow and resource-hungry?

Real time style transfer and neural doodle is very much possible and is an active topic I see users working on to improve upon. The basic idea is to do only feed forward propagation at test time and train with appropriate loss functions at train time.

Perceptual Losses for Real-Time Style Transfer
and Super-Resolution is a good starting point to understand a methodology for this purpose.


Does it make sense to combine classifiers trained on the same dataset?

These are generally known as ensemble methods. Your method is essentially what Scikit-Learn's VotingClassifier does, which is perfectly reasonable and might give you better results. Of course, if you have an ensemble of classifiers and some of them perform quite poorly, the ensemble might not be able to beat the best classifier: you'll need to check this in your cross-validation. Be aware that any confidence or probability estimates may not be well-calibrated, so the predictions from the ensemble may not be particularly meaningful.
There are more elaborate ways of ensembling classifiers. Random forest classifiers, for example, are just ensembles of decision trees; a technique called bagging is also used here to improve performance.
The technique of using a second model to weight the predictions from an ensemble is known as stacked generalisation. It was introduced by Wolpert in 1991 [1], and you can find plenty of interesting examples of using the technique, e.g. on Kaggle.
[1] David H. Wolpert. Stacked generalization. Neural Networks 5.2 (1992), pp. 241-259.


In Deep Q-learning, are the target update frequency and the batch training frequency related?

It is fairly common in DQN to train a minibatch after every observation received after the replay memory has enough data (how much is enough is yet another parameter). This is not necessary, and it is fine to collect more data between training steps, the alogrithm is still DQN. The value higher than 1 for train_freq here might be related to use of prioritised replay memory sampling - I have no real experience with that.
The update to target network generally needs to occur less frequently than training steps, it is intended to stabilise results numerically, so that over or under estimates of value functions do not result in runaway feedback.
The parameters choices will interact each other, most hyperparameters in machine learning do so unfortunately. Which makes searching for ideal values fiddly and time-consuming.
In this case it is safe to say that train_freq is expected to be much lower than target_update_freq, probably by at least an order of magnitude, and more usually 2 or 3 orders of magnitude. However, that's not quite the same as saying there is a strong relationship between choices for those two hyperparameters. The value of batch_size is also relevant here, as it shows the rate that memory is being used (and re-used) by the training process.
The library you are using has these defaults:
    batch_size::Int64 = 32
    train_freq::Int64 = 4
    target_update_freq::Int64 = 500

They seem like sane starting points. You are relatively free to change them as if they were independent, as there is no simple rule like "target_update_freq should be 125 times train_freq". As a very rough guide, you can expect that high values of train_freq, low values of batch_size and low values of target_update_freq are likely to cause instability in the learning process, whilst going too far in the opposite direction may slow learning down. You might be able to set train_freq to 1, but I am not completely certain about that either in combination with the prioritised replay memory sampling which seems to be the default in the library you are using.


Why does the binary cross-entropy work better than categorical cross-entropy in a multi-class single label problem?

https://stats.stackexchange.com/questions/260505/machine-learning-should-i-use-a-categorical-cross-entropy-or-binary-cross-entro
Is relevant.

based on my reading when you have a NN and do Binary crossentropy on what you might call 'Linked category data' the accuracy can tend to be better than in a Categorical crossentropy model. The binary aspect implies the categories can undergo multiple splits before deciding on the exact category when the data is categorically splittable like this in a tree like hierarchy the accuracy can tend to be better.

Think of how difficult it would be to memorize the name of every type of clothing in someones wardrobe if each piece had its own special name. Vs if they had structurally relevant names like Upper/lower for category number one is it warn on your top half or bottom half. followed by inner or outer. It an inner or outer layer of clothing. learning such binary name/feature categories enables a more accurate model. If it was data unrelated in this way it would most likely not be as accurate. The binary model can take advantage learning such features while a muti categorical model I think assumes independence and tries to learn best the features of each group and gives out a prediction of how sure it falls in each category.


Is there any practical application of knowing whether a concept class is PAC-learnable?

If you know that a concept class is PAC-learnable (i.e. its VC dimension is finite), then there's a possibility that you can design an algorithm that can find a function (or concept) that is arbitrarily close to your target (or desired) function.

This is not really an application, but a consequence, which can lead to applications.

However, note that asking if PAC learning is useful in practice is like asking if special relativity is useful in practice. Yes, they are useful, but in the sense that they can be used to predict the outcomes of an experiment or explain the rules in their specific context. In the case of machine learning, PAC learning can be used to explain e.g. the probably required number of data points needed to learn a concept (a target function) approximately.

See also Are PAC learning and VC dimension relevant to machine learning in practice? for more concrete "applications" of PAC-learning and the VC dimension.


Is the choice of the optimiser relevant when doing object detection?

I have experimented with this to a small degree and have not noticed that much of an impact.
To date, Adam appears to give the best results on a variety of image data sets. I have found that "adjusting" the learning rate during training is an effective means of improving model performance and has more impact than the selection of the optimizer.
Keras has two callbacks that are useful for this purpose. Documentation is at https://keras.io/callbacks/. The ModelCheckpoint callback enables you to save the full model or just the model weights based on monitoring a metric. Typically, you monitor validation loss and set the parameter save_best_only=True to save the results for the lowest validation loss. The other useful callback is ReduceLROnPlateau, which allows you to adjust the learning rate based on monitoring a metric. Again, the metric usually monitored is the validation loss. If the loss fails to reduce after a user-set number of epochs (parameter patience), the learning rate will be adjusted by a user-set factor (parameter factor). You can think of the training process as traveling down a valley. As you near the bottom of the valley, it becomes more and more narrow. If your learning rate does not adjust to the "narrowness" there is no way you will get to the bottom of the valley.
You can also write a custom callback to adjust the learning rate. I have done this and created one which first adjusts the learning rate based on monitoring the training loss until the training accuracy reaches 95%. Then it switches to adjust the learning rate based on monitoring the validation loss. It saves the model weights for the lowest validation loss and loads the model with these weights to make predictions. I have found this approach leads to faster training and higher accuracy.
The fact is you can't tell if your model has converged on a global minimum or a local minimum. This is evidenced by the fact that, unless you take special efforts to inhibit randomization, you can get different results each time you run your model. The loss can be envisioned as a surface in $N$ space, where $N$ is the number of trainable parameters. Lord knows what that surface is like and where your initial parameter weights put you on that surface, plus how other random processes cause you to traverse that surface.
As an example, I ran a model at least 20 times and got resultant losses that were very close to each there. Then I ran it again and got far better results for exactly the same data.


What kind of experience do I need to get into an AI master's programme?

Well, mathematics is a very strong field in AI as that is what most AI algorithms revolve around. You only need to learn how to write code (Mostly python or c++, python is easier to learn in my opinion) and how to write mathematical equations in code. Once you have that down it should be enough but I don't how courses work at yours as I know our University require a degree in computer science.

If you want to impress somebody then I can pass you an old University assignment I had about backpropagation in neural networks. It does not involve any code but to determine the partial derivative of an equation that is used to update weights in a neural network. If you do the assignment correctly you should know the basics of how a fully connected neural network works, which is the cornerstone of most neural networks, and also how to train it through backpropagation using only maths. Not only would you know how neural networks work but you would have something to show for it. Witch hopefully could contribute to applying for your Masters.

Lastly, I know of people that wrote scientific papers in the realm of computer science, to prove they would survive in masters, and then applied for a Master's degree in Computer Science. If you are really serious about it you can do this but just double-check with someone as you might do all the work for naught.


Why don't neural networks project the data into higher dimensions first, then reduce the size of each layer thereafter?

To better understand this you should think in terms of capacity. Capacity is a theoretical notion that shows how much information your network can model.
The capacity of a network (given sufficient training) ties in directly with the bias/variance tradeoff:

too little capacity and your network isn't able to learn the complex relationships in the data.
too much capacity and your network has the capability of learning the noise in the dataset, besides the useful relationships.

At some point a network reaches a point where it has a high enough capacity to memorize the whole training set!
Now, by increasing the number of hidden neurons you essentially increase the capacity of your network. If the network already has enough capacity to learn the problem, then by increasing the neurons you are giving the network the capability of overfitting more easily.
Note: all the above assume that the network is trained sufficiently (i.e. no early stopping, etc).


What could possibly replace artificial intelligence?

By definition, artificial intelligence includes all forms of computer systems capable of completing tasks that would ordinarily warrant human intelligence.

A superintelligent AI would have intelligence far superior to that of any human and therefore would be capable of creating systems beyond our capabilities.

As a consequence, if a technology superior to AI were to be created, it would almost certainly be created by an artificial intelligence.

  For the purposes of mankind, however, superintelligent artificial intelligence is the ultimate technology due to the fact that it will be able to surpass humans in every field, and, if anything, replace the need for human intelligence.

In our past experience, intelligence has been the most valuable trait for any entity to manifest - for this reason, in an anthropomorphic context, we can predict that artificial intelligence will be the ultimate achievement.

  The main reason why we will certainly not be able to replace superintelligent AI is that it will surpass us in every respect - if there is ever any replacement, it will be created by the AI similarly to the way we may create an AI that replaces us.


Can neural networks evolve other neural networks?

Yes, this is an active area of research as we speak. Both using classic algorithms (decision trees, random forests, Bayesian ensembles) as well as neural networks. This can also be done via evolutionary algorithms. I have personally used them for hyperparameter tuning in a few cases where squeezing out a couple of extra points of accuracy was key.

This is in fact what Google is doing with their AutoML system. They are using neural networks for architecture search.

Here is a Github repo with some interesting papers and links on the topic you are describing: https://github.com/hibayesian/awesome-automl-papers.


What is the difference between training and testing in reinforcement learning?

What is reinforcement learning?
In reinforcement learning (RL), you typically imagine that there's an agent that interacts, in time steps, with an environment by taking actions. On each time step $t$, the agent takes the action $a_t \in \mathcal{A}$ in the state $s_t \in \mathcal{S}$, receives a reward (or reinforcement) signal $r_t \in \mathbb{R}$ from the environment and the agent and the environment move to another state $s_{t+1} \in \mathcal{S}$, where $\mathcal{A}$ is the action space and $\mathcal{S}$ is the state space of the environment, which is typically assumed to be a Markov decision process (MDP).
What is the goal in RL?
The goal is to find a policy that maximizes the expected return
(i.e. a sum of rewards starting from the current time step). The policy that maximizes the expected return is called the optimal policy.
Policies
A policy is a function that maps states to actions. Intuitively, the policy is the strategy that implements the behavior of the RL agent while interacting with the environment.
A policy can be deterministic or stochastic. A deterministic policy can be denoted as $\pi : \mathcal{S} \rightarrow \mathcal{A}$. So, a deterministic policy maps a state $s$ to an action $a$ with probability $1$. A stochastic policy maps states to a probability distribution over actions. A stochastic policy can thus be denoted as $\pi(a \mid s)$ to indicate that it is a conditional probability distribution of an action $a$ given that the agent is in the state $s$.
Expected return
The expected return can be formally written as
$$\mathbb{E}\left[ G_t \right] = \mathbb{E}\left[ \sum_{i=t+1}^\infty R_i \right]$$
where $t$ is the current time step (so we don't care about the past), $R_i$ is a random variable that represents the probable reward at time step $i$, and $G_t = \sum_{i=t+1}^\infty R_i $ is the so-called return (i.e. a sum of future rewards, in this case, starting from time step $t$), which is also a random variable.
Reward function
In this context, the most important job of the human programmer is to define a function $\mathcal{R}: \mathcal{S} \times \mathcal{A} \rightarrow \mathbb{R}$, the reward function, which provides the reinforcement (or reward) signal to the RL agent while interacting with the environment. $\mathcal{R}$ will deterministically or stochastically determine the reward that the agent receives every time it takes action $a$ in the state $s$. The reward function $R$ is also part of the environment (i.e. the MDP).
Note that $\mathcal{R}$, the reward function, is different from $R_i$, which is a random variable that represents the reward at time step $i$. However, clearly, the two are very related. In fact, the reward function will determine the actual realizations of the random variables $R_i$ and thus of the return $G_i$.
How to estimate the optimal policy?
To estimate the optimal policy, you typically design optimization algorithms.
Q-learning
The most famous RL algorithm is probably Q-learning, which is also a numerical and iterative algorithm. Q-learning implements the interaction between an RL agent and the environment (described above). More concretely, it attempts to estimate a function that is closely related to the policy and from which the policy can be derived. This function is called the value function, and, in the case of Q-learning, it's a function of the form $Q : \mathcal{S} \times \mathcal{A} \rightarrow \mathbb{R}$. The name $Q$-learning derives from this function, which is often denoted as $Q$.
Q-learning doesn't necessarily find the optimal policy, but there are cases where it is guaranteed to find the optimal policy (but I won't dive into the details).
Of course, I cannot describe all the details of Q-learning in this answer. Just keep in mind that, to estimate a policy, in RL, you will typically use a numerical and iterative optimization algorithm (e.g. Q-learning).
What is training in RL?
In RL, training (also known as learning) generally refers to the use of RL algorithms, such as Q-learning, to estimate the optimal policy (or a value function)
Of course, as in any other machine learning problem (such as supervised learning), there are many practical considerations related to the implementation of these RL algorithms, such as

Which RL algorithm to use?
Which programming language, library, or framework to use?

These and other details (which, of course, I cannot list exhaustively) can actually affect the policy that you obtain. However, the basic goal during the learning or training phase in RL is to find a policy (possibly, optimal, but this is almost never the case).
What is evaluation (or testing) in RL?
During learning (or training), you may not be able to find the optimal policy, so how can you be sure that the learned policy to solve the actual real-world problem is good enough? This question needs to be answered, ideally before deploying your RL algorithm.
The evaluation phase of an RL algorithm is the assessment of the quality of the learned policy and how much reward the agent obtains if it follows that policy. So, a typical metric that can be used to assess the quality of the policy is to plot the sum of all rewards received so far (i.e. cumulative reward or return) as a function of the number of steps. One RL algorithm dominates another if its plot is consistently above the other. You should note that the evaluation phase can actually occur during the training phase too. Moreover, you could also assess the generalization of your learned policy by evaluating it (as just described) in different (but similar) environments to the training environment [1].
The section 12.6 Evaluating Reinforcement Learning Algorithms of the book Artificial Intelligence: Foundations of Computational Agents (2017) by Poole and Mackworth provides more details about the evaluation phase in reinforcement learning, so you should probably read it.
Apart from evaluating the learned policy, you can also evaluate your RL algorithm, in terms of

resources used (such as CPU and memory), and/or
experience/data/samples needed to converge to a certain level of performance (i.e. you can evaluate the data/sample efficiency of your RL algorithm)
robustness/sensitivity (i.e., how the RL algorithm behaves if you change certain hyper-parameters); this is also important because RL algorithms can be very sensitive (from my experience)

What is the difference between training and evaluation?
During training, you want to find the policy. During the evaluation, you want to assess the quality of the learned policy (or RL algorithm). You can perform the evaluation even during training.


Explain the difference in graphical patterns between discriminator fake loss and generator loss in GAN

Your statement that  we should be expecting some similarity in graphical patterns is not correct. The GAN loss takes the following form:
$$
\min_G \max_D V(D,G) = \mathbb{E}_{x \sim p_{data}(x)} [\log D(x)] + \\
+ \mathbb{E}_{z \sim p_z(z)} [\log (1 - D(G(z)))]\label{minimax}
$$
We want to maximize this loss w.r.t. D in order to distinguish between real and fake samples ($D(x)\rightarrow 1$ and $D(G(z))\rightarrow 0$), whereas the task of G is exactly the opposite. We want to minimize the function w.r.t. G so that the difference between real and generated data will be minimal. Thus, the problem becomes a minimax non-cooperative game. Here is a good explanation of the loss.
Training GANs requires finding a Nash equilibrium of a non-convex game with continuous, high dimensional parameters. GANs are typically trained using gradient descent techniques that are designed to find a low value of a cost function, rather than to find the Nash equilibrium of a game. Two models are trained simultaneously and each model updates its cost independently with no respect to another player in the game.
In other words, as soon as D becomes better, G becomes also better. Updating the gradient of both models concurrently cannot guarantee a convergence. For more information, see Improved Techniques for Training GANs and Wasserstein GAN.
Moreover, since D and G play a non-cooperative game, it can be shown that there are some cases when it is impossible to find a Nash equilibrium at all:


3D CNN's are used when you want to extract features in 3 Dimensions or establish a relationship between 3 dimensions.

Essentially its the same as 2D convolutions but the kernel movement is now 3-Dimensional causing a better capture of dependencies within the 3 dimensions and a difference in output dimensions post convolution.

The kernel on convolution will move in 3-Dimensions if the kernel depth is lesser then the feature map depth.

On the other hand 2-D convolutions on 3-D data mean that the kernel will traverse in 2-D only. This happens when the feature map depth is the same as the kernel depth (channels)

Some use cases for better understanding are - MRI scans where relationship between a stack of images is to be understood ; and a low level feature extractor for spatio-temporal data like videos for Gesture Recognition, Weather forecast etc. (3-D CNN's are used as low level feature extractors only over multiple short intervals as 3D CNN's fail to capture long term spatio-temporal dependencies - for more on that check out ConvLSTM or an alternate perspective here.)
Most CNN models that learn from video data almost always have 3D CNN as a low level feature extractor.

In the example you have mentioned above regarding the number 5 - 2D convolutions would probably perform better, as you're treating every channel intensity as an aggregate of the information it holds, meaning the learning would almost be the same as it would on a black and white image. Using 3D convolution for this on the other hand would cause learning of relationships between the channels which do not exist in this case! (Also 3D convolutions on an image with depth 3 would require a very uncommon kernel to be used, especially for the use case)

Hope your query has been cleared!


Can RNNs get inputs and produce outputs similar to the inputs and outputs of FFNNs?

Yes, it is possible. What you have shown in case of ANN is what happens in a regression model using NNs. What you have shown in case of RNN is what happens when you are doing sequence-to-sequence translation (like French to English).
If you want to get single values like in case of ANN, suppose you are doing regression, then, in the end, you will flatten the features aggregated by RNN (in case of Tensorflow, use Flatten layer and in case of PyTorch, you can directly do it). It should be then followed by a dense layer of 3 (in case of Tensorflow) or linear layer of 3 (in case of PyTorch), if I am talking about your example.
Since, you have shown values above 1, I presume you are doing some kind of regression. But, it would be good idea to normalize your outputs in case of regression, it makes the optimization easier.
If you want to classification then in the last layer, use Dense layer with softmax (in case of Tensorflow) and softmax followed by a linear layer (in case of PyTorch).


Can ML/AI understand incomplete constructs like humans?

I believe that an AI can understand much more of a movie than we do. As you said, sometimes a detail at the beginning of the movie is the key to understanding the final outcome. The problem is that the storyline distracts us. In the case of an AI, he would just be waiting for the movie to complete because he has all the details at all times.

Imagine that an AI is by your side watching the movie. He will not feel emotion like you. He is only analyzing the images, colors, soundtrack, main characters and is extracting from the lines, whatever he can, to understand the movie. What would it be to understand, in your eyes? To be able to generate a synopsis or to compare the film with some reality or criticizes the life of the human?

An artificial superintelligence would have the task of overcoming us. We could pass a movie and let her explain why we cried watching the movie, because we were surprised, and everything. Since an artificial superintelligence would have the role of not being distracted by the film but analyzing all possible content, try to analyze human reactions and spit out an analysis that could leave you reflecting on life for a few days.

But this is an artificial superintelligence that I imagine may one day exist, but I believe it is far away.

Something real and what we can build is an AI that can extract the genre from the movie and maybe even generate a synopsis with a spoiler like "X character dies."

Another factor that also makes it easier for us to understand movies is our huge film base. We watch so many American cliches movies, that most are already very predictable: "The main character does not usually die," "When the villain is close to killing the main character, something surprising happens and he is prevented," ...


Feature Selection algorithm for a high featured data

Have you tried any possibilities?

You have many possible approaches.

FeatureMiner (http://featureselection.asu.edu/index.php) or SkLearn (https://scikit-learn.org/) are popular options for your purposes.

The documentation from these two possibilities is also interesting.


Which functions can't neural networks learn efficiently?

One of the important qualifications of the Universal approximation theorem is that the neural network approximation may be computationally infeasible.

  "A feedforward network with a single layer is sufficient to represent any function, but the layer may be infeasibly large and may fail to learn and generalize correctly." - Ian Goodfellow, DLB

I can't think of any function that I would definitively declare as unlearnable, but neural networks have many problems. Consider adversarial examples and adversarial patches, which highlight the poor generalization going on under the hood of recent advances in computer vision.

Neural Networks are also inherently limited by the innate priors baked into their architecture and the sample density of their training data. Check out this recent discussion at Stanford's AI Salon between Yann LeCun and Christopher Manning on innate priors if that is the kind of limitation you are talking about.


The are some fundamental learning theories for developing an AI that imitates human behavior

There is an interesting discussion the progress achieved in this field by far in the paper of Francois Chollet - https://arxiv.org/abs/1911.01547. At the present time, many architectures are able to outperform the human in particular tasks, because they have a strong priors coded into them and ability to process a huge amount of data.
However, when it comes to generalization, or even more difficult task of doing the things, that the coder has not put into the model, the present algorithms do no perform well. There is a rather sophisticated and developed mathematical definition of what is required from the system to be called intelligent. And in a nutshell, it is ability to develop a sensible behaviour and enough accuracy for new tasks without putting strong priors and large amount of experience.
In the end of the paper, there is proposed a benchmark to measure the intelligence of AI system.


Can a variational auto-encoder learn images composed of random noise at each pixel (each drawn from the same distribution)?

VAE's try and model the distribution of your data. So it's not going to learn " images composed of random noise at each pixel" per se (though, if overfitting, it could remember them). But it would be very capable of learning the simple noise distribution from which you sampled your random pixels


Is randomness (either true randomness or simulated randomness) necessary for AI

It depends on how you define Artificial Intelligence.  If you regard it strictly as an intentionally created construct which demonstrates utility, then no.  (For instance, Nimatron, potentially the first functioning AI, beat most human competitors at NIM.  But Nimatron was classical AI, entirely rules based with no learning.) That said:

Randomness has proved a useful component in machine learning, and any feasible AGI would likely require ML.

Given sufficient computing power, aka time and space, it would absolutely be possibly to brute force anything, including AGI, but the resulting algorithm would be "brittle", unable to "compute" anything not previously defined. A learning algorithm, presented a problem outside of its domain of knowledge may initially degrade in performance, but it can learn from those outcomes, and gradually improve performance.

IBM brute forced Chess with Deep Blue, but Chess is a strictly narrow problem that turned out not to require general intelligence.  AGI requires human level performance in all tasks engaged in by humans, which, even if they could be broken down to a set of individual narrow problems, it's an ever expanding set of problems.

  Does it mean "intelligence comes from randomness"?

Not if the definition of intelligence is rooted in utility because deterministic processes can demonstrate utility.

In statistical AI, the intelligence arises from the analysis of random search or the fitness of the genetic algorithm, not the randomness per se.

In other words, if you have the randomness without the analysis, every decision is an unqualified guess.

My sense is that it is free will that would arise from randomness—effects unrelated to causes—because without true randomness, the universe and everything in it is purely deterministic.


What is the relevance of AIXI on current artificial intelligence research?

"Current artificial intelligence research" is a pretty broad field. From where I sit, in a mostly CS realm, people are focused on narrow intelligence that can do economically relevant work on narrow tasks. (That is, predicting when components will fail, predicting which ads a user will click on, and so on.)

For those sorts of tools, the generality of a formalism like AIXI is a weakness instead of a strength. You don't need to take an AI that could in theory compute anything, and then slowly train it to focus on what you want, when you could just directly shape a tool that is the mirror of your task.

I'm not as familiar with AGI research itself, but my impression is that AIXI is, to some extent, the simplest idea that could work--it takes all the hard part and pushes it into computation, so it's 'just an engineering challenge.' (This is the bit about 'finding approximations to AIXI.') The question then becomes, is starting at AIXI and trying to approximate down a more or less fruitful research path than starting at something small and functional, and trying to build up?

My impression is the latter is much more common, but again, I only see a small corner of this space.


Why do we use the softmax instead of no activation function?

Short answer: Generally, you don't need to do softmax if you don't need probabilities. And using raw logits leads to more numerically stable code.
Long answer: First of all, the inputs of the softmax layer are called logits.
During evaluation, if you are only interested in the highest-probability class, then you can do argmax(vec) on the logits. If you want probability distribution over classes, then you'll need to exponentiate and normalize to 1 - that's what softmax does.
During training, you'd need to have a loss function to optimize. Your training data contains true classes, so you have your target probability distribution $p_i$, which is 1 at your true class and 0 at all other classes. You train the network to produce a probability distribution $q_i$ as an output. It should be as close to the target distribution $p_i$ as possible. The "distance" measure between two probability distribution is called cross-entropy:
$$ H = - \sum p_i \log q_i $$
As you can see, you only need logs of the output probabilities - so the logits will suffice to compute the loss. For example, the keras standard CategoricalCrossentropy loss can be configured to compute it from_logits and it mentions that:

Using from_logits=True is more numerically stable.


Inception neural network input layer confusion

The padding is not size zero* in the inception CNN layers. In fact it is deliberately chosen to pad so that the convolution by itself would produce an image the same size as the original. I.e. $p=(f−1)/2$, in some libraries this is called "same" padding.

So, $p=3$

The stride is not 2. It is $s=1$ for the convolution. The Inception CNN does not use strided convolutions. Instead the stride of 2 is associated with a later max-pooling layer.

Therefore, using $(((n+2p-f)/s)+1)$ with the correct values $(((224 + 6 - 7)/1)+1 = 224$

Then apply max-pooling, with stride 2. $224/2 = 112$.

* Not to be confused with "zero padding" which means pad using $0$ as the value to insert into the new area. So you can have "zero padding with $p=3$"


How is the latent vector transforming to a feature map in DCGAN (Generator structure)?

Noise vector (batch_size, 100, 1, 1) is deconvoloved with filter_1 (100, 4, 4). Result is feature_map_1 (1, 4, 4). And since there are 512 filters, so there will be 512 feature maps. Output shape will be (batch_size, 512, 4, 4).
I think you need better undersanting for convolutional calculations in general. In this stack it was explained very well.


How to Mask an image using Numpy/OpenCV?

You can use cv2.bitwise_and and pass rectangle as a mask.

im = cv2.imread(filename)
height,width,depth = im.shape
cv2.rectangle(img,(384,0),(510,128),(0,255,0),3)
cv2.rectangle(rectangle,(width/2,height/2),200,1,thickness=-1)

masked_data = cv2.bitwise_and(im, im, mask=rectangle)

cv2.imshow("masked_data", masked_data)
cv2.waitKey(0)


My bad, didn't look into the block diagram first

maxpooling filter/kernel is 3x3 = fxf

Formula for padding to get the same dimension as the input

p = (f - 1)/2

here f=3 ,so padding to be performed before maxpooling is (3-1)/2 = 1


Why is the completeness of UCS guaranteed only if the cost of every step exceeds some small positive constant?

Let's consider a problem where all edge costs are greater than zero, but not above some $\epsilon$:
Image a problem where we have an infinite path where the first edge is cost $\frac{1}{2}$, the next is $\frac{1}{4}$, the following is $\frac{1}{8}$, and so on forever. Every edge is greater than zero, meeting the condition being proposed in the question. However, this path overall has finite cost (1) even through there are an infinite number of states on that path. So, on this problem UCS will never reach paths with cost greater than 1. Thus, if the solution cost is 2, UCS will not find any solution to this problem, and thus it would not be a complete algorithm. So, all edges being greater than zero is not sufficient.
For most search algorithms to be complete, there must be a finite number of states with any given cost. (To be slightly more precise, there must exist some fixed $\epsilon$ such that in each range size $\epsilon$ there are a finite number of states.)


Algorithms for games with very high branching factors (Connect6)

Typically, Monte-Carlo Tree Search (MCTS) actually is the go-to "solution" for such problems with large branching factors. I can understand that "vanilla" MCTS may still have unsatisfactory performance, but there is a plethora of extensions/enhancements available.

I don't have experience with the specific game you mentioned (Connect6), but from a quick look at how the game works, I imagine there will be a huge number of transpositions in the search tree (positions that are the same but can be reached through multiple different paths in the search tree). This will especially be very common if you treat placing one stone as a single ply; every "combined move" (of placing two stones in two positions subsequently) can be reached in two different ways, simply by switching the order in which the player places them. There has been research in using Transposition Tables with MCTS, so that may be a promising direction to look into.

I also suspect there will be great value in using Deep (Reinforcement) Learning approaches. If there is a large board on which to place stones, there will likely be many moves that are "absurd" and can easily be dismissed altogether by Deep Learning approaches (e.g., placing stones far away in a corner of the board where none of the "action" is going on). Vanilla MCTS, without Deep Learning extensions, will not be able to recognize and dismiss such absurd moves, and play them way too often (in Play-Out but also Selection phase due to the high branching factor). The most obvious source of inspiration here would be AlphaGo Zero.

Finally, there's definitely some published research on Connect6 AI (and even MCTS in Connect6). For example: Two-Stage Monte Carlo Tree Search for Connect6. You can likely also find more relevant research by checking that paper's list of References, and checking later papers on google scholar that cite this one.


What are "development test sets" used for?

In machine learning, you normally split your data into 3 parts (80-10-10%).
The first part (80% of your initial data) is for the training of your ML model: this is known as the training dataset.
The second part (10%) is the development set (or dataset), aka validation set. This is used as measuring your performance with various hyperparameters (e.g. in neural networks: layer size).
After you found your best hyperparameters, you learn the model again on the training set, and then test it on your test dataset (10%), which the model has never seen before. Your result on the test data is now a good indicator how your model prediction quality is in the real world (because it was never optimized for this test data).


What are all the different kinds of neural networks used for?

I agree that this is too broad, but here's a 1 sentence answer for most of them. The ones I left out (from the bottom of the chart) are very modern, and very specialized. I don't know much about them, so perhaps someone who does can improve this answer.

Perceptron: Linear or logistic-like regression (and thus, classification).
Feed Forward: Usually non-linear regression or classification with sigmoidal activation. Essentially a multi-layer perceptron.
Radial Basis Network: Feed Forward network with Radial Basis activation functions. Used for classification and some kinds of video/audio filtering
Deep Feed Forward: Feed Forward with more than 1 hidden layer. Used to learn more complex patterns in classification or regression, maybe reinforcement learning.

Recurrent Neural Network: A Deep Feed Forward Network where some nodes connect to past layers. Used in reinforcement learning, and
to learn patterns in sequential data like text or audio.
LSTM: A recurrent neural network with specialized control neurons (sometimes called gates) that allow signals to be remembered for longer periods of time, or selectively forgotten. Used in any RNN application, and often able to learn sequences that have a very long repetition time.
GRU: Much like LSTM, another kind of gated RNN with specialized control neurons.

Auto Encoder: Learns to compress data and then decompress it. After learning this model, it can be split into two useful subparts: a mapping from the input space to a low-dimensional feature space, that may be easier to interpret or understand; and a mapping from a small dimensional subspace of simple numbers into complex patterns, which can be used to generate those complex patterns. Basis of much modern work in vision, language, and audio processing.
VAE, DAE,SAE: Specializations of the Auto Encoder.

Markov Chain: A neural network representation of a markov chain: State is encoded in the set of neurons that are active, and
transition probabilities are thus defined by the weights. Used for
learning transition probabilities and unsupervised feature learning
for other applications.
HN, BM, RBM, DBM: Specialized architectures based on the Markov Chain idea, used to automatically learn useful features for other
applications.

Deep Convolutional Network: Like a feed-forward network, but each node is really  a bank of nodes learning a convolution from the layer
before it. This essentially allows it to learn filters, edge
detectors, and other patterns of interest in video and audio
processing.
Deep Deconvolutional Network: Opposite of a Convolutional Network in some sense. Learn a mapping from features that represent edges or
other high level properties of some unseen image, back to the pixel
space. Generate images from summaries.
DCIGN: Essentially an auto-encoder made of a DCN and a DN stuck together. Used to learn generative models for complex images like
faces.
Generative Adversarial Network: Used to learn generative models for complex images (or other data types) when not enough training data is
available for a DCIGN. One model learns to generate data from random
noise, and the other learns to classify the output of the first
network as distinct from whatever training data is available.


What are the features get from a feature extraction using a CNN?

You get what we call high-level features, which are basically abstract representations of the parts that carry information in the image you want to classify.

Imagine you want to classify a car. The image you feed your network could be a car on a road with a driver and trees and clouds, etc. The network, however, if you've trained it to recognize cars, will try to focus on parts of the image regarding a car. The final layers will have learned to extract an abstract representation of a car from the image (this means a low-resolution car-like shape). Now your final FC layers will try to classify the image from these high-level features. In this example, you would have an FC layer that learns to classify a car if this this abstract car-like figure is present in the image. Likewise, if it isn't present it won't classify it as a car. By accessing these high-level features, you essentially have a more compact and meaningful representation of what the image represents (based always on the classes that the CNN has been trained on).

By visualizing the activations of these layers we can take a look on what these high-level features look like.

The top row here is what you are looking for: the high-level features that a CNN extracts for four different image types.


This is a very interesting question and also an important one for AI. All the current Deep Learning successes are built on the effectiveness of backprop, so what if it doesn't play a role in the only examples of intelligence currently around?

It makes a lot of sense for the brain to employ some form of backprop, because that would allow it to create low-level features in a way most conductive for the high-level features that finally give rise to intelligent behavior. And if the brain employs backprop, dreams are a logical candidate.

Of course the problem here is that the brain doesn't calculate and propagate gradients, so the question is whether the backpropagation algorithm can somehow be implemented with the biological constraints in mind, such as the locality of neural computation and the non-symmetrical feedback connections.

It turns out this is actually possible, as explained in these talks by Hinton and Bengio. Symmetrical feedback connections (and in fact continually updated feedback connections) aren't necessary for backpropagation, however unintuitive this may seem. And backpropagation can be emulated with a form of temporal difference learning, that is plausible within biological constraints.

Of course this doesn't conclusively answer the question, but it certainly makes the possibility much more plausible than a priori assumed.


How to apply hyperparameter optimization on Monte Carlo Tree Search?

Since no one answers, I will say the approach I used.
It was inspired by the successive halving algorithm. The concept is that we find good performing hyperparameters for one agent by playing manually against it. Then, we put the agent we want to tune to play against it. We assign different configurations. All of these configurations are tried for a predefined number of games per configuration. We keep the best performing half, and then go to the next round with the half we kept and repeat this process. The last configuration that remains is the best performing configuration.


How is PDDL used in production AI systems?

I have seen it used in automated story-telling (or game AI to control NPCs), and in NLG systems, where the generation of text is reinterpreted as a planning task.

What these systems have in common is that they're either off-line or in a simple environment (NPC control). I'm not sure they would be suitable for real-time applications, unless you can be sure that a feasible plan exists which can be found within certain time bounds. I wouldn't want to sit in a car going at high speed on the motorway and waiting for the driving unit to work out a plan how to avoid an obstacle that suddenly appears on the road.


Why is the policy loss the mean of $-Q(s, \mu(s))$ in the DDPG algorithm?

This is not quite the loss that is stated in the paper.
For standard policy gradient methods the objective is to maximise $v_{\pi_\theta}(s_0)$ -- note that this is analogous to minimising $-v_{\pi_\theta}(s_0)$. This is for a stochastic policy. In DDPG the policy is now assumed to be deterministic.
In general, we can write
$$v_\pi(s) = \mathbb{E}_{a\sim\pi}[Q(s,a)]\;;$$
to see this note that
$$Q(s,a) = \mathbb{E}[G_t | S_t = s, A_t=a]\;;$$
so if we took expectation over this with respect to the distribution of $a$ we would get
$$\mathbb{E}_{a\sim\pi}[\mathbb{E}[G_t|S_t=s, A_t=a]] = \mathbb{E}[G_t|S_t=s] = v_\pi(s)\;.$$
However, if our policy is deterministic then $\pi(\cdot|s)$ is a point mass (a distribution which has probability 1 for a specific point and 0 everywhere else) for a certain action, so $\mathbb{E}_{a\sim\pi}[ Q(s,a)] = Q(s,a=\pi(s)) = v_\pi(s)$. Thus the objective is still to maximise $v_\pi(s)$ it is just that now we know the policy is deterministic we say we want to maximise $Q(s,a=\pi(s))$.
The policy gradient of this term was shown to be
\begin{align}
\nabla_\theta Q(s,a=\pi_\theta(s)) & \approx \mathbb{E}_{s \sim \mu}[\nabla_\theta Q(s,a=\pi_\theta(s))]\;; \\ & =  \mathbb{E}_{s\sim\mu}[\nabla_aQ(s,a=\pi(s)) \nabla_\theta \pi_\theta(s)]\;;
\end{align}
where if we put a minus at the front of this term then we would arrive at the loss from the paper. Intuitively this makes sense, you want to know how much the action-value function changes with respect to the parameter of the policy, but this would be difficult to directly calculate so you use the chain rule to see how much the action-value function changes with $a$ and in term how much $a$ (i.e. our policy) changes with the parameter of the policy.
I realise I have changed notation from the paper you are reading so here $\pi$ is our policy as opposed to $\mu$ and here where I have used $\mu$ I take this to be the state distribution function.


Understanding proof of lemma 1 (policy improvement bound) of the "Trust Region Policy Optimization" paper

Let's assume $\gamma = 1$ to simplify things
\begin{align}
\mathbb E_{\tau|\pi} [\sum_{t = 0}^{\infty} A_\pi(s_t, a_t)] &= \mathbb E_{\tau|\pi}[A_\pi(s_0, a_0) + \ldots A_\pi(s_i, a_i) + \ldots]\\
&= \mathbb E_{a_0 \sim \pi,s_1 \sim P(s_1|s_0, a_0)}[A_\pi(s_0, a_0)] + \ldots + \mathbb E_{a_i \sim \pi,s_{i+1} \sim P(s_{i+1}|s_i, a_i)}[A_\pi(s_i, a_i)] + \ldots
\end{align}
if we observe only $i$-th timestep
\begin{align}
\mathbb E_{a_i \sim \pi,s_{i+1} \sim P(s_{i+1}|s_i, a_i)}[A_\pi(s_i, a_i)] &= \sum_{a'} (\mathbb E_{s_{i+1} \sim P(s_{i+1}|s_i, a_i)}[A_\pi(s_i, a')]) \pi(a'|s_i)\\
&= \sum_{a'} (\mathbb E_{s_{i+1} \sim P(s_{i+1}|s_i, a_i)}[\mathbb E_{s_{i+1} \sim P(s_{i+1}|s_i, a_i)}[r(s_i) + V_\pi(s_{i+1}) - V_\pi(s)]]) \pi(a'|s_i)
\end{align}

\begin{equation}
\mathbb E[\mathbb E[f]] = \mathbb E[f]
\end{equation}

\begin{align}
\mathbb E_{a_i \sim \pi,s_{i+1} \sim P(s_{i+1}|s_i, a_i)}[A_\pi(s_i, a_i)] &= \sum_{a'} (\mathbb E_{s_{i+1} \sim P(s_{i+1}|s_i, a_i)}[r(s_i) + V_\pi(s_{i+1}) - V_\pi(s)]) \pi(a'|s_i)\\
&= \mathbb E_{a_i \sim \pi,s_{i+1} \sim P(s_{i+1}|s_i, a_i)}[r(s_i) + V_\pi(s_{i+1}) - V_\pi(s)]
\end{align}

now for all timesteps sum everything up.


Would AI be appropriate for converting unstructured text into an XML?

A rule-based approach will guarantee a correct result, and it works perfectly fine. On the other hand, an AI based approach will introduce errors as AI cannot produce result with 100% accuracy, and also decreasing speed. As the document you are processing is a legal document, it would be better to use a parser as AI is only adding unnecessary wasted time and non-accurate results.

Hope I can help you.


Why is depth-first search an artificial intelligence algorithm?

This is a fundamentally a philosophical question. What makes AI AI? But first things, why would DFS be considered an AI algorithm?
In its most basic form, DFS is a very general algorithm that is applied to wildly different categories of problems: topological sorting, finding all the connected components in a graph, etc. It may be also used for searching. For instance, you could use DFS for finding a path in a 2D maze (although not necessarily the shortest one). Or you could use it to navigate through more abstract state spaces (e.g. between configuration of chess or in the towers of Hanoi). And this is where the connection to AI arises. DFS can be used on its own for navigating such spaces, or as a basic subroutine for more complex algorithms. I believe that in the book Artificial Intelligence: A Modern Approach (which you may be reading at the moment) they introduce DFS and Breadth-First Search this way, as a first milestone before reaching more complex algorithms like A*.
Now, you may be wondering why such search algorithms should be considered AI. Here, I'm speculating, but maybe the source of the confusion comes from the fact that DFS does not learn anything. This is a common misconception among new AI practitioners. Not every AI technique has to revolve around learning. In other words, AI != Machine Learning. ML is one of the many subfields within AI. In fact, early AI (around the 50s-60s) was more about logic reasoning than it was about learning.
AI is about making an artificial system behave "intelligently" in a given setting, whatever it takes to reach that intelligent behavior. If what it takes is applying well-known algorithms from computer science like DFS, then so be it. Now, what is it that intelligent means? This is where we enter more philosophical grounds. My interpretation is that "intelligence" is a broad term to define the large set of techniques that we use to approach the immense complexity that reality and certain puzzle-like problems have to offer. Often, "intelligent behavior" revolves around heuristics and proxy methods away from the perfect, provable algorithms that work elsewhere in computer science. While certain algorithms (like DFS or A*) may be proven to give optimal answers if infinitely many resources can be devoted to the task at hand, only in sufficiently constrained settings would such techniques be affordable. Fortunately, we can make them work in many situations (like A* for chess or for robot navigation, or Monte Carlo Tree Search for Go), but only if reasonable assumptions and constraints over the state space are imposed. For all the rest is where learning techniques (like Markov Random Fields for image segmentation, or Neural Nets paired with Reinforcement Learning for situated agents) may come handy.
Funny enough, even if intelligence is often regarded as a good thing, my interpretation can be summed up as imperfect modes of behavior to address immensely complex problems for which no known perfect solution exists (with rare exceptions in sufficiently bounded problems). If we had a huge table that, for each chess position, gives the best possible move you can make, and put that table inside a program, would this program be intelligent? Maybe you'd think so, but in any case it seems more arguable than a program that makes real-time reasoning and spits a decision after some reasonable time, even if it's not the best one. Similarly, do you consider sorting algorithms intelligent? Again, the answer is arguable, but the fact is that algorithms exist with optimal time and memory complexities, we know that we can't do better than what those algorithms do, and we do not have to resort to any heuristic or any learning to do better (disclaimer: I haven't actually checked if there's some madman out in the wild applying learning to solve sorting with better average times).


In the machine learning literature, what does it mean to say that something is "embedded" in some space?

Embedding is the process of representing data (from a source domain) in a new (or target) domain. Usually, the source domain is discrete, and the target domain is continuous. For example, embedding words into the continuous vector space can be done by the word2vec method.
The main reason behind using the embedding is doing meaningful mathematical computations in the target domain, which is not possible or straightforward in the source domain. For example, summing two words "brother" - "man" + "woman" not meaningful in the word and character levels. However, when using word2vec, embedding("brother") - embedding("man") + embedding("woman") can be meaningful and comparable with other embedded vectors; It should be near the embedded vector of "sister".


Is there any programming language designed by deep learning?

There are certainly things like this.

I'd say a strong example is layered learning approaches, descended from Peter Stone's work.

A programming language is essentially a collection of useful shorthands for assembly-level instructions. Ultimately, everything you do in a programming language eventually gets executed in assembly. So making a programming language amounts to learning how to write short, reusable, assembly language programs that you can then use as building blocks to solve harder problems.

An example of this in action is Kelly & Heywood's approach to constructing 'Tangled Program Graphs' for reinforcement learning (IJCAI 2018). Here an evolutionary algorithm is used to learn short assembly programs, that can be combined into a graph to make more complex programs. This is similar to graphical programming languages like J.


What is the fundamental difference between CNN and RNN?

Basically, a CNN saves a set of weights and applies them spatially. For example, in a layer, I could have 32 sets of weights (also called feature maps). Each set of weights is a 3x3 block, meaning I have 3x3x32=288 weights for that layer. If you gave me an input image, for each 3x3 map, I slide it across all the pixels in the image, multiplying the regions together. I repeat this for all 32 feature maps, and pass the outputs on. So, I am learning a few weights that I can apply at a lot of locations.

For an RNN, it is a set of weights applied temporally (through time). An input comes in, and is multiplied by the weight. The networks saves an internal state and puts out some sort of output. Then, the next piece of data comes in, and is multiplied by the weight. However, the internal state that was created from the last piece of data also comes in and is multiplied by a different weight. Those are added and the output comes from an activation applied to the sum, times another weight. The internal state is updated, and the process repeats.

CNN's work really well for computer vision. At the low levels, you often want to find things like vertical and horizontal lines. Those kinds of things are going to be all over the images, so it makes sense to have weights that you can apply anywhere in the images.

RNN's are really good for natural language processing. You can imagine that the next word in a sentence will be highly influenced by the ones that came before it, so it makes sense to carry that internal state forward and have a small set of weights that can apply to any input.

However, there are many more applications. In addition, CNN's have performed well on NLP tasks. There are also more advanced versions of RNN's called LSTM's that you could check out.

For an explanation of CNN's, go to the Stanford CS231n course. Especially check out lecture 5. There are full class videos on YouTube.

For an explanation of RNN's, go here.


Not really. Unity physics is just an approximation of an approximation. It has to look more or less real but at the same time the performances are very important, so it has not the realistic level you would hope to "bring things to the real world". There are some physics engine you can install that usually work a bit better. Still, don't expect "real-world level".
Based on 1, yes. If, once you got it to the real world you still give it the chance to optimize, it can only make things better.
This is difficult to say. You should try to use the best possible approximation in Unity, maybe testing multiple scenarios and seeing that they perform more or less in the same way. Then train. Then bring this "approximated model" to the real world and let it train more to adjust to the real physics.
I don't know this. Unity seems the most versatile around (together with other game engines). This versatility is its strength and weakness, as it cannot focus on solving perfectly a single problem. It rather has to aim at generalizing as much as possible. There might be "more specialized" programs around, but I doubt.
Just try different algorithms/methods. Usually with complex problems neuroevolutionary techniques perform better, but the amount of code is hugely higher.


How do I classify whether a document is legal or not given a set of keywords that appear only in legal documents?

Maybe this is what you are looking for:
https://en.wikipedia.org/wiki/Aho%E2%80%93Corasick_algorithm
Basically you would

build and store a finite-state machine that resembles a trie with additional links between the various internal nodes using the given keywords.
for the candidate document, go through it with the above finite-state machine.

Then based on the result

If there is any match, then the document is valid.
Otherwise it is possible that the document is invalid. However, in terms of the probability, I do not have any good idea yet.


How is Bayes' Theorem used in artificial intelligence and machine learning?

Bayes theorem states the probability of some event B occurring provided the prior knowledge of another event(s) A, given that B is dependent on event A (even partially).
A real-world application example will be weather forecasting. Naive Bayes is a powerful algorithm for predictive modelling weather forecast. The temperature of a place is dependent on the pressure at that place, percentage of the humidity, speed and direction of the wind, previous records on temperature, turbulence on different atmospheric layers, and many other things. So when you have certain kind of data, you process them certain kind of algorithms to predict one particular result (or the future). The algorithms employed rely heavily on Bayesian network and the theorem.
The given paragraph is introduction to Bayesian networks, given in the book, Artificial Intelligence – A Modern Approach:

  Bayesian network formalism was invented to allow efficient representation of, and rigorous reasoning with, uncertain knowledge. This approach largely overcomes many problems of the probabilistic reasoning systems to the 1960s and 70s; it now dominates AI research on uncertain reasoning and expert systems. The approach allows for learning from experience, and it combines the best of classical AI and neural nets.

There are many other applications, especially in medical science. Like predicting a particular disease based on the symptoms and physical condition of the patient. There are many algorithms currently in use that are based on this theorem, like binary and multi-class classifier, for example, email spam filters.
There are many things in this topic.I have added some links below that might help, and let me know if you need any kind of other help.

Helpful Links
 1. First
 2. Second


What's the difference between architectures and backbones?

The vocabulary is definitely non-standard and a bit confusing, but Feature Pyramid Networks is used as a feature extractor, and its output is then fed into EfficientNet-B2 to be used to classify the image. One neural network model is concatenated at the end of the other.
So it seems like "architecture" is the front half of the neural network model which takes as input the satellite image and extracts image featuers, and then is directly connected to the back half of the model (hence "backbone"), which takes the features extracted from the "architecture" and makes a classification.
This terminology is definitely non-standard here, at least in the AI community, and if you ask anyone here I think it will be uncommon for them to naturally think about the words "architecture" vs "backbone" in this way unless they specialize in a similar field to the authors.


What is the relation between a policy which is the solution to a MDP and a policy like $\epsilon$-greedy?

for example, the "greedy policy" always chooses the action with the highest expected return, no matter which state we are in

The "no matter which state we are in" there is generally not true; in general, the expected return depends on the state we are in and the action we choose, not just the action.

In general, I wouldn't say that a policy is a mapping from states to actions, but a mapping from states to probability distributions over actions. That would only be equivalent to a mapping from states to actions for deterministic policies, not for stochastic policies.

Assuming that our agent has access to (estimates of) value functions $Q(s, a)$ for state-action pairs, the greedy and $\epsilon$-greedy policies can be described in precisely the same way.

Let $\pi_g (s, a)$ denote the probability assigned to an action $a$ in a state $s$ by the greedy policy. For simplicity, I'll assume there are no ties (otherwise it would in practice be best to randomize uniformly across the actions leading to the highest values). This probability is given by:

$$
\pi_g (s, a) =
\begin{cases}
1, & \text{if } a = \arg\max_{a'} Q(s, a') \\
0, & \text{otherwise}
\end{cases}
$$

Similarly, $\pi_{\epsilon} (s, a)$ could denote the probability assigned by an $\epsilon$-greedy strategy, with probabilities given by:

$$
\pi_{\epsilon} (s, a) =
\begin{cases}
(1 - \epsilon) + \frac{\epsilon}{\vert \mathcal{A}(s) \vert}, & \text{if } a = \arg\max_{a'} Q(s, a') \\
\frac{\epsilon}{\vert \mathcal{A}(s) \vert}, & \text{otherwise}
\end{cases}
$$
where $\vert \mathcal{A}(s) \vert$ denotes the size of the set of legal actions in state $s$.


What are the limitations of the hill climbing algorithm and how to overcome them?

As @nbro has already said that Hill Climbing is a family of local search algorithms. So, when you said Hill Climbing in the question I have assumed you are talking about the standard hill climbing. The standard version of hill climb has some limitations and often gets stuck in the following scenario:

Local Maxima: Hill-climbing algorithm reaching on the vicinity a local maximum value, gets drawn towards the peak and gets stuck there, having no other place to go.
Ridges: These are sequences of local maxima, making it difficult for the algorithm to navigate.
Plateaux: This is a flat state-space region. As there is no uphill to go, algorithm often gets lost in the plateau.

To resolve these issues many variants of hill climb algorithms have been developed. These are most commonly used:

Stochastic Hill Climbing selects at random from the uphill moves. The probability of selection varies with the steepness of the uphill move.
First-Choice Climbing implements the above one by generating successors randomly until a better one is found.
Random-restart hill climbing searches from randomly generated initial moves until the goal state is reached.

The success of hill climb algorithms depends on the architecture of the state-space landscape. Whenever there are few maxima and plateaux the variants of hill climb searching algorithms work very fine. But in real-world problems have a landscape that looks more like a widely scattered family of balding porcupines on a flat floor, with miniature porcupines living on the tip of each porcupine needle (as described in the 4th Chapter of the book Artificial Intelligence: A Modern Approach). NP-Hard problems typically have an exponential number of local maxima to get stuck on.
Given algorithms have been developed to overcome these kinds of issues:

Stimulated Annealing
Local Beam Search
Genetic Algorithms

Reference Book - Artificial Intelligence: A Modern Approach


Does the Chinese Room argument hold against AI?

There are two broad types of responses to philosophical queries like this.

The first is to make analogies and refer to intuition; one could, for example, actually calculate the necessary size for such a Chinese room, and suggest that it exists outside the realm of intuition and thus any analogies using it are suspect.

The second is to try to define the terms more precisely. If by "intelligence" we mean not "the magic thing that humans do" but "information processing," then we can say "yes, obviously the Chinese Room involves successful information processing."

I tend to prefer the second because it forces conversations towards observable outcomes, and puts the difficulty of defining a term like "intelligence" on the person who wants to make claims about it. If "understanding" is allowed to have an amorphous definition, then any system could be said to have or not have understanding. But if "understand" is itself understood in terms of observable behavior, then it becomes increasingly difficult to construct an example of a system that "is not intelligent" and yet shares all the observable consequences of intelligence.


Confusing Matlab Artificial Neural Toolbox script

I don't see anything wrong in the generated code.
Let me explain:

In the following block, we define that 70% of the data will be used for training the network, 15% for the validation, and 15% for the testing:

net.divideParam.trainRatio = 70/100;
net.divideParam.valRatio = 15/100;
net.divideParam.testRatio = 15/100;

Then the network is trained from the inputs/targets passed as arguments. Note that here, all inputs are given at once for efficiency. The separation training/validation/testing is done inside the net function from the proportions passed in the net input argument (through the net.divideParam.XRatio properties).

% Train the Network
[net,tr] = train(net,inputs,targets);

Then we test the results of the network on the data without discrimination between training/validation/testing. This performance score should be higher than the "real" performance score of the net as the performance over the training data should be higher than the one over the validation/testing data.

% Test the Network
outputs = net(inputs);  % Compute predictions from the inputs
errors = gsubtract(targets,outputs);  % Compare the predictions to the targets (true values)
performance = perform(net,targets,outputs)  % Built-in Matlab function giving an overall performance score

Now that we know how the net performs overall, we can look at how it performs on the training, validation, and testing sets, respectively. The masks are created as an output of the net function, from the proportions given in the first block of code. The output have been already all computed when the net function has been called.

% Recalculate Training, Validation and Test Performance
trainTargets = targets .* tr.trainMask{1}; % Apply a mask (0's and 1's to select the proper targets)
valTargets = targets .* tr.valMask{1}; % idem
testTargets = targets .* tr.testMask{1}; % idem

trainPerformance = perform(net,trainTargets,outputs); % Compute a score for the training data
valPerformance = perform(net,valTargets,outputs); % same for the validation data
testPerformance = perform(net,testTargets,outputs); % same for the test data

I think your code does the exact same thing but in more steps, with multiple calls to the net function and without using its native way of working (you define the proportions of training/validation/testing by hand for instance without making use of the net.divideParam.Xratio property). So it is probably not optimal from the computational point of view.


Is it possible to classify the subject of a conversation?

This is a difficult problem.
First, how do you define 'subject'? Do you have a (closed) lists of labels you want to assign? What about subjects that overlap, or don't occur in your list? What even is a subject? This is a non-trivial issue.
Second, and this is even harder, how do you want to recognise subjects? A simple solution could be using a list of associated keywords, but this is problematic as many words have multiple meanings, and words are not really a good indicator of a conversation topic in the first place.
Instead of jumping to an implementation method, be clear about how you want to tackle these two items first. Start by annotating a conversation transcript by hand. You will then get a feeling for the problems and possible solutions. After you have done this, you can think about how to get a machine to do it efficiently.
UPDATE: For a scheme to annotate the functions of lines within a conversation have a look at Francis & Hunston (1992) Analysing Everyday Conversation.  In Coulthard, M. (ed.) "Advances in Spoken Discourse Analysis". London: Routledge. pp.123-161.  This is more oriented towards linguistics, but might give you some ideas on how to proceed.


Are humans intelligent according to the definition of an intelligent agent?

"the human mind is a battleground of higher level goals and lower level goals "— Marvin Minsky paraphrasing Sigmund Freud

I argue that in general human agents try to maximise a hierarchy of performance measures.

performance measures of humans

Survival of genetic data

Energy supply and Water
Sex

myriad subgoals....

Mysterious mental mechanisms which neuroscientists do not understand yet force the average human agent to maximise various evaluation metrics.
With the overarching goal of survival of genetic information. Successful genes are immortal. We are still under the yoke of an ancient genetic algorithm.

These measures are optimised throughout a humans life time. A 30 year old agent is better at survival than a 10 year old agent. A 30 year old agent makes fewer mistakes.

We remember our mistakes. Mistakes are burned into our memory by high levels of neurotransmitters (and reinforcing of synapses) so we don't make them again.

We attempt to optimise a swarm of subgoals that are all connected in one way or another to the main goal gene survival.

status
money
education
happiness


Can NEAT produce neural networks where inputs are directly connected to outputs?

Yes, it is possible (depending on the nature of your problem), using the four types of standard NEAT mutation, but it is improbable.

When the NEAT algorithm begins, it operates on a blank canvas. After each generation, the algorithm will either:

Construct a new axon
Construct a new node on an existing axon
Update existing weights/bias
Remove a node or axon from the network

(However, in general, NEAT does not produce neural networks where two input (or output) nodes are connected.)


What is the dimensionality of the output map, given the dimensionality of the input map, number of filters, stride and padding?

where, $p_{x}$ and $p_{y}$ are padding values. (equal on both sides).
You can have different padding on left and right side. (similarly top and bottom). If same padding, equation will have $2*p_{x}$ or $2*p_{y}$, else you can just add values of both padding and replace in the equations. (For example $p_{xLeft} + p_{xRight}$)

Padding is optional. Generally thumb rule is add padding such that the equations above produce integers and also add padding such that if stride is 1, output is same as input and if stride is 2, output is half of input.