Transformers - is the self attention matrix softmax output (layer 1) symmetric?

I compared my results visually to a second implementation known to be working - "The annotated transformer".  I compared the pytorch calculation results of the attention-method to my implementation results.
The answe is - the softmax is applied row by row. Therefore the resulting matrix p-attn is not equal to its transposed version.


What are some common heuristics that might be innate?

Perhaps Occam's razor counts.

Occam's razor is the meta-heuristic that "the simplest explanation is the most likely to be correct". I consider it a meta-heuristic because itself doesn't provide explanations, only means of comparing them among another. It is a heuristic because like all heuristics, it can lead to false conclusions, but is usually right.


Applying Artificial neural network into kaggle's house prices data set gave bad predicted values

Is it because the number of hidden layers with its input? Or because there is few data to train (1460 rows of train set) and the neural network needs more than that? Or is it because of the number of neurons in each layer?

I think you are onto something here. You have over a thousand nodes, almost as many as training samples you have. In my experience, the network will have no trouble overfitting, given you train long enough. If you dont train long enough on the other hand, your network will probably not learn anything.

I would try with a smaller network, so you "force" it to learn something, instead of memorizing.


How to detect any native language when written in Latin characters?

The most distinct words of a language are usually the function words (the, and, of, with,...); other lexical items are often (at least partly) shared between languages that had come in contact with each other. So looking for function words is usually the best way to identify the language in a given text.

This can be done by having a list of function words for each language you recognise, look those words up in your text, and then calculate the probability that the text belongs to each language given the frequencies of the words in your list. There will be overlaps, for example Dutch of is equivalent to the conjunction or in English, but also looks like the English of.

Another approach would be to take texts in your languages, and split them into n-grams, typically trigrams. Then you use a list of trigrams and compare the frequency distributions in your known and unknown texts to find which is most likely to match. This has the advantage that you don't need to know anything about the language structure (because you don't need to identify function words), and it also captures the morphology (eg the common English suffix -ing).

You can simply implement this in a fairly basic program, or you could train a machine learning classifier with it. The former seems easier to me, but is not as exciting as the latter.

However, for single words (as per your example) this might not work properly. Generally, the longer the text, the more precise the recognition.

UPDATE: On re-reading your question I think I have misunderstood it, and that you're asking about transliteration into Latin characters. In a way the same applies, that you need to first identify which language it is, and then choose the correct mapping from the Latin transliteration to the writing system that language natively uses.

Since the transliteration is purely symbolic, I don't think that an ML approach would be better than a simple lookup-table of equivalences. And of course there might be difficulties if the writing system has more distinct characters than the  Latin alphabet provides.

UPDATE 2: If the input is transliterated text, then your reference texts (where you know the language) also have to be transliterated in the same character set (eg Latin in your example).


Building AI from chess - data shape from simulation

Not an expert here by any means. I think the simplest way to approach the chess problem is to have three components a) a game logic component, that can generate all the possible legal moves from a given board position b) an evaluation function that can score a given game position and c) your minimax/alphabeta algorithm that searches through the tree of possible moves generated by a) looking for the branch that returns the best score using b). Here, the machine learning part would be the evaluation function. You would just transform each board position in your data into a feature vector, and train some kind of classifier to return the probability of win/lose at that given board position. I think it's highly doubtful that this would work with such a small data set though.


Is automated feature engineering a path to general AI?

Automated feature engineering, if it is part of any aproach towards general intelligence, cannot be the whole solution. The search for features that are meaningful, as opposed to those that simply exist with no utility, needs some guidance.
In machine learning, feature engineering is typically a search for features that improve performance at a specific task, such as classification or regression. The "intelligence" is partially in the manner of searching, and in setting the goals in the first place. Automated feature engineering typically uses fairly crude search algorithms to look for good features, such as random combination, and follows up with raw processing power in order to cover large numbers of options. Automated feature engineering also does not set the goals for any task, or feed back active behaviour from the output of ML to direct the search. An intelligent agent might actively search for data in its environment in order to test an idea (e.g. repeat an action in order to discover whether the same thing happens, or move in order to observe an interesting event better). Feature engineering is a separate issue to this.
There are some theories of general intelligence that are focused on smart pattern matching as a key component. For instance, Jürgen Schmidhuber has long been a proponent of the reward system for intelligence being compression of observations and predictions. In such a system, better pattern matching discovered by an agent is an intrinsic reward signal as it allows for better compression of world models used by the agent.
Marcus Hutter is another well-known AI researcher, who has proposed a framework called AIXI which incorporates similar ideas. An agent operating using AIXI will benefit from discovering features in its observations that improve its own predictions of what will happen next. Some form of automated feature engineering could well be core to such an agent.


How UCT in MCTS selection phase avoids starvation?

First explore the nodes A,B,C once.
For reference see this paper by David Silver and Sylvain Gelly, Combining Online and Offline Knowledge in UCT

If any action from the current state $s$ is not represented in the tree, $\exists a \in \mathcal{A}(s),(s, a) \notin \mathcal{T},$ then the uniform random policy $\pi_{\text {random }}$ is used to select an action from all unrepresented actions, $\tilde{\mathcal{A}}(s)=\{a \mid(s, a) \notin \mathcal{T}\}$.


How can I automate the choice of the topology of a neural network for an arbitrary problem?

I think in this case, you'll probably want to use a genetic algorithm to generate a topology rather than working on your own. I personally like NEAT (NeuroEvolution of Augmenting Topologies).

The original NEAT paper involves evolving weights for connections, but if you only want a topology, you can use a weighting algorithm instead. You can also mix activation functions if you aren't sure which to use. Here is an example of using backpropagation and multiple neuron types.


Why is the evidence equal to the KL divergence plus the loss?

In variational inference, the original objective is to minimize the Kullback-Leibler divergence between the variational distribution, $q(z \mid x)$, and the posterior, $p(z \mid x) = \frac{p(x, z)}{\int_z p(x, z)}$, given that the posterior can be difficult to directly infer with the Bayes rule, due to the denominator term, which can contain an intractable integral.

Therefore, more formally, the optimization objective can be written as

\begin{align}
q^*(z \mid x) = \operatorname{argmin}_{q(z \mid x)} D_{\text{KL}}(q(z \mid x) \| p(z \mid x))\tag{1} \label{1}
\end{align}

However, solving this optimization problem can be as difficult as the original inference one of computing the posterior $p(z \mid x)$ using the Bayes rule, given that it still involves the possibly intractable term $p(z \mid x)$.

If you use the definition of the KL divergence, you can derive the following equation

\begin{align}
D_{\text{KL}}(q(z \mid x) \| p(z \mid x)) = \mathbb{E}_{q(z \mid x)} \left[ \log q(z \mid x) \right] - \mathbb{E}_{q(z \mid x)} \left[ \log q(z, x) \right] + \log p(x) \tag{2} \label{2}
\end{align}

First, note that the expectations are with respect to the variational distribution, which means that, if you want to approximate these expectations with Monte Carlo estimates, you can do it with respect to the variational distribution, and, given that it is assumed that one can easily sample from the variational distribution (which can e.g. be a Gaussian), this is a nice feature.

Second, the KL divergence contains the term $p(x) = \int_z p(x, z)$, the denominator term in the Bayes rule to compute the posterior $p(z \mid x)$, which (as I said) can be intractable. $p(x)$ is often called the evidence.

The solution is then to optimize an objective that does not contain this annoying intractable term $p(x)$. The objective that is optimized is the so-called ELBO objective

\begin{align}
\text{ELBO}(q) = \mathbb{E}_{q(z \mid x)} \left[ \log q(z, x) \right] - \mathbb{E}_{q(z \mid x)} \left[ \log q(z \mid x) \right]\tag{3} \label{3}
\end{align}

The KL divergence \ref{2} and the ELBO objective \ref{3} are similar. In fact, ELBO is an abbreviation for Evidence Lower BOund, because the ELBO is a lower bound on the evidence $p(x)$, i.e. it is a number that is smaller than $p(x)$ or, more formally, $\text{ELBO}(q) \leq \log p(x)$. Therefore, if we maximize $\text{ELBO}(q)$, we also maximize the evidence $p(x)$ of the data (where $x$ is the data in your dataset).

So, the objective in variational inference is

\begin{align}
q^*(z \mid x)
&= \operatorname{argmax}_{q(z \mid x)} \operatorname{ELBO}({q}) \\
&= \operatorname{argmax}_{q(z \mid x)}  \mathbb{E}_{q(z \mid x)} \left[ \log q(z, x) \right] - \mathbb{E}_{q(z \mid x)} \left[ \log q(z \mid x) \right]
\tag{4} \label{4}
\end{align}

First, note that \ref{4} only contains terms that depend on the variational distribution, so we got rid of intractable terms, which was our goal.

Second, note that, as opposed to \ref{1}, we are maximizing (or finding the parameters that maximize the objective).

The ELBO objective is actually the negative of \ref{2} plus the logarithm of the evidence term, $\log p(x)$ (and you can easily verify it), that is

\begin{align}
\text{ELBO}(q)  = -D_{\text{KL}}(q(z \mid x) \| p(z \mid x)) + \log p(x)
\end{align}

which can also be rearranged as

\begin{align}
\log p(x) = D_{\text{KL}}(q(z \mid x) \| p(z \mid x)) + \text{ELBO}(q) \tag{5}\label{5}
\end{align}

which is your equation (where $\text{ELBO}(q)$ is your $\mathcal{L}$). Therefore, your equation is true by definition, i.e. we define the ELBO such that \ref{5} is true. However, note that we haven't defined the ELBO in the way we dit only for the sake of it, but because it is a lower bound on the log evidence (and this follows from the fact that the KL divergence is never negative).


Lets mock some data up.

  "100 numbers, each one is a parameter, they together define a number X(also given)"

# i.e. size of X_train -> [n x d]
# i.e. size of X_train -> [??? x 100]  , when d = 100

# "I have 20000 instances for training"
# i.e. size of X_train -> [20000 x 100], when n = 20000

import torch
import numpy as np

X_train = torch.rand((20000, 100))
X_train = np.random.rand(20000, 100) # Or using numpy

But what is your Y?

# Since the definition of a regression task,
# loosely means to predict an output real number
# given an input of d dimension

# So the appropriate Y_train would
# be of dimension [n x 1]
# and look like this:

y_train = torch.rand((20000, 1))

y_train = np.random.rand(20000, 1) # Or using numpy

What is a linear perceptron?

Taking definition from this tutorial

Thus, in picture:

Next we need to define training routine,

For now take it as biblical truth that this is an okay routine to train a neural net model (this isn't the only way but easiest or supervised learning):

In code:

import math
import numpy as np
np.random.seed(0)

def sigmoid(x): # Returns values that sums to one.
    return 1 / (1 + np.exp(-x))

def sigmoid_derivative(sx):
    # See https://math.stackexchange.com/a/1225116
    # Hint: let sx = sigmoid(x)
    return sx * (1 - sx)

def cost(predicted, truth):
    return np.abs(truth - predicted)

num_epochs = 10000 # No. of times to iterate.
learning_rate = 0.03 # How large a step to take per iteration.

# Lets standardize and call our inputs X and outputs Y
X = np.array(torch.rand((20000, 100)))
Y = or_output

for _ in range(num_epochs):
    layer0 = X

    # Step 2a: Multiply the weights vector with the inputs, sum the products, i.e. s
    # Step 2b: Put the sum through the sigmoid, i.e. f()
    # Inside the perceptron, Step 2.
    layer1 = sigmoid(np.dot(X, W))

    # Back propagation.
    # Step 3a: Compute the errors, i.e. difference between expected output and predictions
    # How much did we miss?
    layer1_error = cost(layer1, Y)

    # Step 3b: Multiply the error with the derivatives to get the delta
    # multiply how much we missed by the slope of the sigmoid at the values in layer1
    layer1_delta = layer1_error * sigmoid_derivative(layer1)

    # Step 3c: Multiply the delta vector with the inputs, sum the product (use np.dot)
    # Step 4: Multiply the learning rate with the output of Step 3c.
    W +=  learning_rate * np.dot(layer0.T, layer1_delta)

Now that we learn the model, i.e. the W.

When we see the data points that we need to use the model on, we apply the same forward propagation step, i.e. layer1 = sigmoid(np.dot(X, W))

Since we have:

  I have 5000 lines given, each containing the 100 numbers as parameters.My task is to predict the number X for these 5000 instances.

And in code:

# If we mock up the data,
# it should be the same internal dimension.
X_test = np.random.rand(5000, 100)

# The desired output just needs to pass through the W and the activation:
# the shape of `output` -> [5000 x 1] ,
# where there's 1 output value for each input.
output = sigmoid(np.dot(X_test, W))


Let me re-frame your question before answering:

  Can augmented reality be a training system for computer vision?

Yes, why not! Also, I'm pretty sure the big companies who have their aug. reality products out there are secretly optimizing their CV algorithms(although we'll have no proof).

A very similar(but a different) example would be how Comma.ai's app pays users for recording their driving, and then uses it to train their own self-driving cars.

So, if that can be done, I don't see why augmented reality cannot be used to train CV algos. In fact, it's a much more richer, localized dataset.


When doing transfer learning, which initial layers do we need to freeze, and how should I change the last layer for my task?

After a lot of browsing online for answers to these questions, this is what I came up with.

What are the initial layers in this case? How exactly can I freeze them?

The initial few layers are said to extract the most general features of any kind of image, like edges or corners of objects. So, I guess it actually would depend on the kind of backbone architecture you are selecting.
How to freeze the layers depends on the framework we use.
(I have selected PyTorch as the framework. I found this tutorial Some important Pytorch tasks - A concise summary from a vision researcher, which seems to be useful.)

What are the changes I need to make while training the NN to classify one or more new classes?

I just need to change just the number of neurons (or units) in the final layer to the number of classes/objects your new dataset contains.


How can a probability density value be used for the likelihood calculation?

The probability density is used to 'measure how good' the parameters are because it is a natural way of quantifying if these parameters are good for the observed data.
Also, as the notation often causes some confusion, $L(\theta | x)$ denotes the probability of all of your observed data, not just one value. Also the "$|$" may cause confusion as it looks like we are conditioning on $x$ but this is not the case - it may be better practice to use $L(\theta; x)$ which is the notation used when I was learning likelihood. Further, as you have written $L(\theta | x) = p_\theta(x)$ I would like to clarify that if we are being precise in our definitions then this is only correct if you have one observed data point, assuming that you meant $p_\theta(x)$ is the density of $X$.
In my example below I use $p_\mu(x)$ to denote the density of the normal distribution with mean parameter $\mu$, but the density of the likelihood is the product of all the densities (because we assumed iid data). It is crucial that you understand that in general the likelihood is the probability of your observed data and not just the density or the product of densities as you may not always have iid and so it will not always boil down to taking the product of some densities.
The idea of Maximum Likelihood is to maximising the (log-)likelihood for a given set of data. This means that we need to choose a probability distribution that is parameterised by some parameters $\boldsymbol{\theta}$ and then optimise the parameters such that the likelihood is maximum.
Assuming we don't remove any constants then this makes intuitive sense as maximising the likelihood would be maximising the probability that the data came from this distribution -- i.e. the data we observed is most likely to have come from the given distribution with the given parameters. This is important as it means we then have the most likely model of our data which allows us to use this distribution to make inference about our data.
As an example, imagine if I had some iid data $x_1, x_2, ..., x_n \sim \mathcal{N}(0, 1)$. Now I could try to fit a $\mathcal{N}(\mu, 1)$ to this data and optimise for $\mu$. If I chose e.g. $\mu = -1000000$ then the likelihood (again assuming we don't remove any constants) would be $\approx 0$. If I chose a value of $\mu = 0.1$ then the likelihood would be much higher because this parameter is closer to the true parameter value.
To see why it is higher, recall that the likelihood for iid data is given by
$$\prod_{i=1}^n p_\mu(x_i)$$
and if we evaluated our likelihood at $\mu=-10000000$ then you're going to be taking the product of lots of numbers that are $\approx 0$ - if you think about the bell curve shape of a Normal distribution that is centred at $-10000000$ with variance 1 then the density at the true $x_i$ values (recalling they are simulated from a unit Normal) would be approx $0$ - whereas if we evaluated the $x_i$ at the density of a Normal distribution centred at $0.1$ then the density will be non-zero and so your likelihood will have a higher value.
To summarise, the density value can be used to measure how good parameters are for a set of data as maximising wrt the parameters is analogous to maximising the probability that your data arose from said distribution.
As an aside, note that the definition of likelihood is the probability of observing your data under some assumed distribution. For discrete random variables this is fine, but for continuous distributions we have to be a more subtle. For any continuous random variable $X$ we have $\mathbb{P}(X=x) = 0$. However, for a very small $\delta$ we can say that
$$\mathbb{P}(x - \frac{\delta}{2} < X \leq x + \frac{\delta}{2}) = \int_{x - \frac{\delta}{2}}^{x + \frac{\delta}{2}} f_X(x) dx \approx \delta p_X(x) \; ;$$
you can think of this approximation by visualising an integral and recalling that an integral represents the area under the curve, and so for small $\delta$ this integral can be approximated by taking the area of a rectangle which is width $\times$ height, where the width is $\delta$ and the height is $f_X(x)$. This justifies our use of the density function for continuous random variables. Note that typically in maximum likelihood we omit any multiplicative constants as they don't depend on the parameters which is what happens with the $\delta$ from this justification.


How to make a bot to learn from previous games

Reinforcement learning

The problem that you describe, namely, choosing a good sequence of actions based on a reward/score received based on the whole sequence (and possibly significantly delayed), is pretty much the textbook definition of Reinforcement learning.

As with quite a few other topics, deep neural networks currently seem to be a promising way for solving this type of problems. This may be a beginner-friendly description of this approach.


Why do all nodes in a GP tree need to be the same type?

In tree-based genetic programming (TGP), you have a tree that represents a program or a function. The nodes in this tree are functions, while the edges represent the interactions between these functions. The leaves of this tree are the inputs (or random numbers) that you pass to this function. The incoming edges into a node represent the inputs, while the outgoing edges represent the output of the associated function.
So, for example, consider the following function
\begin{align}
f(x)
&= \sin(x^2) \\
&= \sin(y)
\tag{1}\label{1},
\end{align}
where $y = g(x)$ (note that this is just a change of variable in order to illustrate the corresponding tree more clearly below!).
The function $f$ will be represented by the following tree
sin(y)
  |
 g(x)
  |
  x

So, how do we read this diagram? Essentially, we read it from the bottom-up. So, $x$ is first passed to $y = g(x) = x^2$, which then passed to $\sin(y)$. In this case, given that we are dealing with mathematical operations, you expect $x$ to be a number, because, otherwise, what would $x^2$ or $\sin(x^2)$ mean?
Let's now consider a function of 2 inputs.
\begin{align}
f(x, y)
&= x + y
\tag{2}\label{2},
\end{align}
The corresponding tree would be
  +
 / \
x   y

In this case, you naturally expect that $x$ and $y$ are also numbers. However, this may not be actually the case. Let's say that you're evolving Python functions, then x + y is well defined even if x and y are strings, i.e. that would be a concatenation operation.
So, in this sense, we naturally expected the functions (the nodes in the tree) to get parameters/arguments with the right types, but, in some cases, more types are possible for the apparently same function.
So, in general, types don't necessarily need to be enforced! It depends on the implementation of TGP.
There's a specific approach to TGP where the idea is really to ensure type-safety, i.e. strongly-typed (tree-based) genetic programming, where functions can have a different number and type for their parameters and return values, but, in that case, only functions that are consistent with the signature of the function can be "connected" with that function in the tree.
In weakly-typed GP, the types are not checked, so you could end up evolving functions of the form $f(x) = \sin(x)$, where $x$ is a string. Of course, when you execute these programs/functions, the program may crash, but this is a different story, which you need to take care of (i.e. you need to ensure that your individuals can be evaluated), if you need this flexibility. This weakly-typed approach can also be viewed as a strongly-typed approach where all functions' parameters and return values have the same type.
DEAP, a well-known Python library for GAs and GP, provides both weakly and strongly-typed GP, so, if you are familiar with Python, you may want to start from it.
To conclude and answer your question more directly, it's not true that in TGP you need all functions to have the same type for all parameters and return values.


Scrabble was solved using reinforcement learning achieving super-human performance. If you are interested, you can start reading about it here: https://en.wikipedia.org/wiki/Maven_(Scrabble)#cite_note-1

You can also read about it here: http://www0.cs.ucl.ac.uk/staff/d.silver/web/Teaching_files/games.pdf (there're several slides about Maven)


I want to reframe your question.

Don't think about switching, think about adding.

In data science you'll be able to go very far with either python or r but you'll go farthest with both.

Python and r integrate very well, thanks to the reticulate package. I often tidy data in r because it is easier for me, train a model in python to benefit from superior speed and visualize the outcomes in r in beautiful ggplot all in one notebook!

If you already know r there is no sense in abandoning it, use it where sensible and easy to you. But it is 100% a good idea to add python for many uses.

Once you feel comfortable in both you'll have  a workflow that fits you best dominated by your favorite language.


How can I make meaningful English sentences from given set of words?

Two Approaches:

Naive Bayes
LSTM

Train Naive Bayes on a whole dataset learning the probability of the next word given a word.

You can even go with any LSTM approaches, but I'd bet on Naive Bayes.

Eg:

text: hello how are you hello how are you hello No how

to get the suggestion of next word depending on current word - hello

p(how | hello) = 3/4

p(No | hello) = 1/4

take argmax of probabilities.

Also remember to smooth, and train on huge dataset. Training is just finding the probabilities before hand.

Hope it helps ;)


How do I design a fitness function that weighs the importance of eating food?

A human analogy can help you here (a variance).
Initialize all the agents with an initial value $x$; we will call this energyUnits. I Will talk later more about this.
Now, add some value, as an incentive, whenever the agent eats good food, to the energyUnits. You need to add a function that will keep decrementing the value of the agent's energyUnits, as humans degrade energy (calories) with time. We will call this function normalDegrade. This is the core part of the solution for your problem.
Now, for the bad (or poisonous) food you can be more creative with. You can simply subtract a given value whenever an agent eats poisonous food. Or you can extend your normalDegrade function with a very high downward slope. In this case, the energy units (value) of the agent will fall very rapidly. This will force the agent to look for good food to survive.
Since the ratio of food is 9:1 with poisonous, you need to initialize the value of $x$ (energyUnits) very high. You need to do some trial and error to find the right fit for you here.
Also, I am assuming that the agent is being removed from the population whenever the value of $x$ is zero or some negative value (which depends). This is important, as it makes sure that the algorithm is not wasting time in processing bad agents.
Because of this, another problem arises of the population coming to extinction. For this, you need to keep generating new agents for which any of the genetic algorithms will do. A new population with better parents of the already present generation will keep the population fit and efficient.
A good fitness function is a core to solving any problem of this kind, and sometimes it is hard to find. You might need to do some trial and error with different values to look for the right fit.


How should I update the weights of a neural network, given the gradient?

Consider that you have a loss function, and you want to tune your model (network) to decrease the loss. The main concept is to tune parameters in a direction which decreases the loss and gives you a better model. You can imagine a mountain where you should reach to the lower grounds.

There are 2 questions here.

In which direction to move?
How much should we move in that direction?

1. In which direction to move?

By "move" I mean tuning the parameters and therefore changing the model. If you are familiar with the concept of slope or gradient in mathematics, you should move in the direction where the slope is downward the most. The gradient shows the direction with the most slope upward. So, we should move in the opposite of gradient direction. So, we should minus the gradient from the original value, hence the negative sign in the formula below.

2. How much should we move in that direction?

This is defined by the learning rate, which is a number that is multiplied by the gradient. You can imagine that, if the learning rate is big, you are taking bigger footsteps in that direction when coming down the mountain. Similarly, when the learning rate is low, you are taking small footsteps coming down the mountain.

So, consider you have $\nabla_w$ (the gradient of the parameters $w$, with respect to the loss function $L$). Let the learning rate be $\gamma$ (for example, $\gamma= 0.1$). The formula to update the parameters would be

$$
w_{\text{new}} = w_{\text{old}} - \gamma * \nabla_{w_{\text{old}}}
$$

Note that gradient descent is not the only way to optimize your model. Many other gradient-based approaches exist, for example, stochastic gradient descent, Adam, RMSprop, conjugate gradients, etc. There other methods, like evolutionary methods (genetic algorithm, for example), that use different concepts.

Also, note that the learning rate does not necessarily have to be fixed, so it can be tuned during the training, if you want or need.


How to separate image recognition from logic?

The underlying abstraction (which is essentially what you'd be using the first network for) is that of reducing the state-space of the raw input via feature extraction/synthesis and/or dimensionality reduction.

At present, there are few definite rules for doing this: practice is more a question of 'informed trial and error'.

If you add some information to your question regarding what has been previously attempted in this area (e.g. on the
ALE platform), this it might be possible to offer some more specific advice.


How long should the state-dependent baseline for policy gradient methods be trained at each iteration?

I went into the pytorch code for the spinning up implementation of vanilla policy gradient and from what I could understand, found that they use a learning rate of 1e-3 for training the baseline and do a gradient descent 80 times on the same dataset by default with no termination criteria.

Also it is usually impossible to fit the value function completely as we are using a function approximator. The main point is not to get too carried away to get the lowest loss as the main improvement in the agent's performance will happen by doing policy gradient descents rather than trying to aggressively get the accurate value function for the baseline.

Link for the implementation:
spinning up implementation of vpg


If your interest is positional information, encode it!

This could include learning an embedding for each position and leveraging that in your model. You could also use an approach to hard-encode rather than learn it (kinda like adding sinusoids in the transformer paper Attention is All You Need

an example of a paper that encodes the 2D positional info: Attention Augmented Convolutional Networks


How to exclude sections of bad data from time-series data before training an LSTM network

Your idea is a good one.
Another idea is to upsample or aggregate your data.  For example, average by week if you generally have a couple of missing days in every week.
A similar question on Stack Exchange:
https://stats.stackexchange.com/questions/374935/how-to-deal-with-really-sparse-time-series-data-for-a-binary-classification-task


Understanding TD(0) algorithm implementation

The $TD(0)$ algorithm learns from incomplete episodes, but in the earlier algorithm we can see that the loop repeats until $s$ is terminal which mean completion of episode.

In the pseudocode, you have two loops: one for each episode and one (nested) for each step of the episode. The until $S$ is terminal means that you perform the updates until you end the episode (that is, you end up in a terminal state, e.g. checkmate in the game of chess). For each step of the episode, you perform the TD(0) update.

Apparently, you're confusing two things: the fact that each episode ends in a terminal state and the fact that TD learns from incomplete information. Each episode ends in a terminal state (otherwise it would not be called an episode), but this does not mean that it collects a full rollout before updating $V$. In fact, at each step of the episode, it updates $V$.

The information in the David Slider's slide is consistent with the pseudocode. TD learns from experience because it uses the given policy $\pi$ to behave.

  learning from incomplete episodes, mean learning of $V(s)$ even when the episode is not completed

Yes, essentially you're updating the value function during each step of each episode.


Is it feasible using today's technology to use an AI training algorithm to custom teach a robot to do common household cores?

I would suggest using a neural network with back propagation. From what I know, they can be applied to many different circumstances and work well. For your more simpler and repetitive tasks like moving an object, you can just use simpler regression methods.


What is meant by gene, chromosome, population in genetic algorithm in terms of feature selection?

A chromosome in this case could be a set of filters, each extracting a different feature (analogous to Convolutional Neural Network). Your question doesn't say what you want to do with these features, so this solution is made under the assumption that there is a fitness function which would take these features as an input and output a score. Then, each gene is a parameter for a filter, each chromosome defines a set of such filters, which makes up an individual. A population is a set of such individuals.


Huge variations in epoch count for highest generalized accuracy in CNN

Fixed. Was an issue with the random generator. In my class for the Neuron layer where I initialize the weights I get new doubles from the generator for each of the initial weight values, but I found a bug where I was re-initializing the random generator, which was of course causing different values.


What is the definition of the "cost" function in the SVM's objective function?

That is the hinge loss, a type of loss most notably used for SVM classification. The hinge loss is typically defined as:
$$
\ell(y)=\max (0,1-t \cdot y),
$$
which, in your use case, is something like this:
$$
\operatorname{cost}\left(h_{\theta}(x), y\right)=\left\{\begin{array}{ll}
\max \left(0,1-\theta^{T} x\right) & \text { if } y=1 \\
\max \left(0,1+\theta^{T} x\right) & \text { if } y=0
\end{array}\right.
$$
Check this article on it.


How do autonomous robotic vacuum cleaners perceive the environment for navigation?

An agent perceives the environment through sensors and act according to the incoming percepts (agent's perceptual input at any instant). An autonomous vacuum cleaner can be as simple as

  (blocki, clean) --> Move to blocki+1
  (blocki, dirty) --> Clean

This is just a general description, actual one is more complicated. Or the bot can have a memory where it stores all its previous decision and incorporate those while taking new ones.
This can be helpful if the bot wants to remember where an obstacle (like wall, in this case bot don't want to go and check the presence of wall each and every single time it is turned on) is, or where it is more probable to find dirt. If the bot is not remembering its history then it will be scanning the whole house over and over again, sensing the same obstacle every time and going across them.
Bot which keeps no log of its history will take the same procedure again and again, making the same mistakes again and again. This is not an efficient way and a waste of its energy (or battery).
Normally today bots have ordinary sensors which can only sense the dirt and obstacle. This limits the number of tasks a bot can perform. If a bot has decent camera as a sensor, and some algorithms of Image Processing are dumped into it, then it increases the tasks it can perform. Like detecting the stairs and cleaning different floors. Normally stairs will be considered obstacle and bot will just go around them. In case, when camera sensor is provided, stairs are potentially a path to be taken.
A* algorithm is not necessarily used in case when the bot is not remembering the map of the house (or room). A normal robot which just scans the room and cleans it, will not be needing, as it don't know it's destination. Its only goal is to clean if it finds something dirty. But a bot which knows the map of the room and where there is a high probability of finding dirt, the A* algorithm can be used.


An intuitive NN playground can be found in TensorFlow Playground

Also, check the Google ML crash course for coders as they promised to add more practicals.


How to model categorical variables / enums?

Authors use many different approaches.

One approach is to have a different input neuron for each possible category, and then use a "1-hot" encoding. So if you have 10 categories, then you can encode this as 10 binary features.

Another is to use some sort of binary encoding. If you have 10 categories, it is sufficient to use 4 neurons to represent all possible categories by using binary numbers.

A third approach is to convert your categories to cardinal values, and then normalize them. This may be more effective if your categories really are cardinal (i.e. orderable). If there isn't a natural ordering to them though, this might lead to strange results or make the problem difficult to learn (since it ends up embedding non-linear relationships in the learning problem that don't need to exist).


What is a trap function in the context of a genetic algorithm?

"Trap" functions were introduced as a way to discuss how GAs behave on functions where sampling most of the search space would provide pressure for the algorithm to move in the wrong direction (wrong in the sense of away from the global optimum).

For example, consider a four-bit function f(x) such that

f(0000) = 5
f(0001) = 1
f(0010) = 1
f(0011) = 2
f(0100) = 1
f(0101) = 2
f(0110) = 2
f(0111) = 3
f(1000) = 1
f(1001) = 2
f(1010) = 2
f(1011) = 3
f(1100) = 2
f(1101) = 3
f(1110) = 3
f(1111) = 4

That is, the fitness of a string is equal to the number of 1s in the string, except f(0000) is 5, the optimal solution. This function can be thought of as consisting of two disjoint pieces: one that contains the global optimum (0000) and another that contains the local optimum at its complement (1111). All points other than these have fitness values such that standard evolutionary algorithm dynamics would lead the algorithms to tend towards the local optimum at 1111 rather than the global optimum at 0000.

That's basically what is meant by a trap function. You can consider variations on this theme, but that's the gist of it.


How can I derive the rotation matrix from the axis-angle rotation vector?

The rotation matrix $R_k(\theta)$ associated with a given unit-length vector $k$ and angle $\theta$ is given by the following formula

$$\small{R_k(\theta) = \begin{bmatrix}
\cos \theta +k_x^2 \left(1-\cos \theta\right) & k_x k_y \left(1-\cos \theta\right) - k_z \sin \theta & k_x k_z \left(1-\cos \theta\right) + k_y \sin \theta \\
k_y k_x \left(1-\cos \theta\right) + k_z \sin \theta & \cos \theta + k_y^2\left(1-\cos \theta\right) & k_y k_z \left(1-\cos \theta\right) - k_x \sin \theta \\
k_z k_x \left(1-\cos \theta\right) - k_y \sin \theta & k_z k_y \left(1-\cos \theta\right) + k_x \sin \theta & \cos \theta + k_z^2\left(1-\cos \theta\right)
\end{bmatrix}}$$

So, to find your specific rotation matrix, you just need to substitute the values of your $k$ and $\theta$ in the above matrix.

The derivation of this matrix can be found in section 9.2 Rotation Matrix Derivation of the PhD thesis Modelling CPV (2015), by Ian R. Cole. The basic idea of the derivation follows the following steps

Rotate the given axis $k$ and the point $p$ (that you want to rotate) such that the axis $k$ lies in one of the coordinate planes: xy, yz or zx
Rotate the given axis $k$ and the point $p$ (that you want to rotate) such that the axis $k$ is aligned with one of the two coordinate axes for that particular coordinate plane: $x$, $y$ or $z$
Use one of the fundamental rotation matrix to rotate the point $p$ depending on the coordinate axis with which the rotation axis is aligned
Reverse rotate the axis-point pair such that it attains the final configuration as that was in step 2 (that is, you have to undo step 2)
Reverse rotate the axis-point pair which was done in step 1 (that is, you have to undo step 1)

To show that a matrix is orthonormal, you need to show that it is orthogonal (each row is independent of any other row and each column is independent of any other column) and that the length of each row (and column) is 1. Equivalently, a square matrix $Q$ is orthogonal if and only if

$$
Q^TQ = QQ^T = I
$$

where $Q^T$ is the transpose of $Q$ and $I$ is the identity matrix. If you are really stuck, have a look at this proof https://math.stackexchange.com/a/537248/168764 (and this other answer https://math.stackexchange.com/a/156742/168764), but, at this point, it should just be a matter of replacing $R_k(\theta)$ in the equation above and check that the equation holds.


How do we find the length (depth) of the game tic-tac-toe in adversarial search?

If you mean minimax search as adversarial, you can take a number of free spaces on the board as an upper bound of the tree depth.


Can AutoKeras be used for neural networks of PyTorch

"Keras is a high-level neural networks API, written in Python and capable of running on top of TensorFlow, CNTK, or Theano."

But I found a useful tool which collects different deep learning model convertors.

Hope this will be useful for you :D


How can Siamese Networks be viewed as RNNs?

Well, here in the picture we have the unrolled or unfold RNN on the right side. Siamese network is formed when it is said to be "unrolled over two time-steps". So, take part where there is two first iterations of RNN and yes, you have kind of Siamese network.

One take from the source of the image:

  Unlike a traditional deep neural network, which uses different parameters at each layer, a RNN shares the same parameters (U, V, W above) across all steps. This reflects the fact that we are performing the same task at each step, just with different inputs. This greatly reduces the total number of parameters we need to learn.

Sounds familiar to siamese network used on single-object tracking: there we take two signals (image and the tracked object), drive it through identical paths and make some maths to get results. Just something the RNN makes to time separated values!

For proof of similarity, a take from a site where siamese networks are nicely explained:

Side note: I don't know then, how closely those relate in real world (could a Siamese network in anyway be a RNN or vice versa), but supposedly so, because the comparison is made by researcher to say so. At diagrammatic level at least there would be no problem on that.


Why are neural networks always trained "by themselves"?

The way children learn is in many ways supervised. It is true that certain abilities are there genetically (visual system, object recognition, to large extent voice recognition), but a lot of human experience is gained as a result of response, either from the environment or from the mentor (parent, teacher, etc).

Social interaction is possible only when a human already learned a lot (things we consider trivial, but in fact pretty complex), in this sense this is equivalent to AlphaGo learning to play Go by playing with a copy network. NNs that can only perform object or voice recognition can't interact, they are too primitive for this. And a nearly all of todays AI is on this stage right now.

However, it is true that NN learning is much less efficient than human learning and current models are far from AGI. I think RL is the closest area that studies real multi-agent environments.


How should I model all available actions of a chess game in deep Q-learning?

To model chess as a Markov decision problem (MDP) you can refer to the AlphaZero paper (Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm). The exact details can be found starting from the bottom of page 13.

Briefly, an action is described by picking a piece and then picking a move with it. The size of the board is 8 by 8 so there can be 8x8 possibilities for picking a piece. Then we can either pick linear movements (in 8 directions) and then pick the number of steps in that direction (maximum 7 steps) or we can make a knight movement (maximum 8 possibilities). So far that is 8x7 + 8. Furthermore, we also need to consider underpromotions (promoting a pawn into a non-queen piece). In this scenario we can have 3 types of pawn movements (forward, left diagonal or right diagonal capture) and 3 types of promotions (rook, knight, bishop) so that makes it 9. So the total dimension of the action space is 8x8x(8x7+8+9) and this will be the number of neuron outputs you will need to use.

Note that this action space representation covers every possible scenario and for example at the start of the game the action of picking the tile E4 and promoting it to a bishop doesn't make sense (there are no pieces on tile E4 at the beginning of the game). Or if we pick a tile where there is a rook we cannot make a knight movement with it. Therefore you will also need to implement a function that can return you the set of possible actions in a given state and ignore all neural network outputs that is not contained in this set.

Obviously this action representation is not set into stone so if you can come up with something better or more compact you can use that one too. You can also make restrictions to your game for example by not allowing underpromotions.


How to use DQN to handle an imperfect but complete information game?

Would it be helpful to use a LSTM and reduce the input state?

I'd bet, no. LSTM is more complicated and harder to learn, while the input is 4 * 9 * 36 bits is still rather limited.

However, you may want to aggregate the information somehow, e.g., add additional bits informing about what cards were already played (no matter when). This information is redundant, but by providing it, you may save the network quite some learning.

At the same time, you may want to use symmetries (all colors but trumps are equivalent and the therefore the weights should be the same).

  How to handle invalid moves?

That's simple: There are no invalid moves. The network provides 36 outputs of how much it wants to play a given card. You simply take the one valid card having the greatest output value. You don't try to make the network learn what moves are valid as this is neither needed nor helpful.

  Do you have some good ideas for improvements? (like Neural-Fictitious Self-Play or something similar)

I can't tell. But it should matter at the moment. First make you network clearly beat the random players, then you can look for more. Or start with Self-Play, as you want probably have both for comparison.

  Or is this the whole approach absolute nonsense?

I don't think so, but ... (see below)

  I design the input state as a vector (one hot encoded) with 36 "bits" for every player

This doesn't sound good. Every player has 9 of 36 cards and so should be the encoding. A player doesn't know the cards of other players.

  The reward is +1 for winning, -1 for losing,

In most card games I know, it matters by how much you win (unlike e.g., in Go). Even when it doesn't matter, using this information at the early learning stages is IMHO useful.

  -0.1 for a invalid action and

Drop the invalid action. Just transform anything the network produces to a valid action, add no penalty (as written above).

  ... 0 for an action which doesn't lead to a terminal state

All action but the last one lead to a non-terminal state. You can use some Temporal Difference Learning or use the fact that the game has a small fixed number of moves and reward/punish all actions takes in a the whole game.


Are there methods that allow deep networks to learn object categorization in a self-supervised way?

I did an experiment, took a trained densenet121 and kept the bottom layers. I trained the FC layer to a softmax and then to a lambda layer that normalized the vector. I trained the network with imagenet to make the outputs the most far a away from (1,1,1,1,1...1) as possible, so I would get one hot vectors. I did, but the network trained to a single category (put all in the same hot vector). Then I added a penalty that encourages to make vectors different is their features are also different. it improved a little, a dozen categories instead of one, bout noting close to a thousand, the available number.

I am posting this so nobody wastes his time in a silly idea like this one.


How is the gradient of the loss function in DQN derived?

In general, if you have a composite function $h(x) = g(f(x))$, then $\frac{dh}{dx} = \frac{d g}{df} \frac{d f}{dx}$. In your case, the function to differentiate is
$$L_{i}(\theta_{i}) = \mathbb{E}_{(s,a,r,s') \sim  U(D)} \left[ \left(r+\gamma \max_{a '} Q(s',a',\theta_{i}^{-}) - Q(s,a;\theta_{i}) \right)^2 \right]$$
So, we want to calculate $\nabla_{\theta_i} L_{i}(\theta_{i})$, which is equal to
$$\nabla_{\theta_i}  \mathbb{E}_{(s,a,r,s') \sim  U(D)} \left[ \left(r+\gamma \max_{a '} Q(s',a',\theta_{i}^{-}) - Q(s,a;\theta_{i}) \right)^2 \right] \label{2} \tag{2} $$
For clarity, let's ignore the iteration number $i$ in \ref{2}, which can thus more simply be written as
$$\nabla_{\theta}  \mathbb{E}_{(s,a,r,s') \sim  U(D)} \left[ \left(r + \gamma \max_{a'} Q(s',a',\theta^{-}) - Q(s,a; \theta ) \right)^2 \right] \label{3} \tag{3} $$
The subscript of the expected value operator $\mathbf{e}=(s,a,r,s') \sim  U(D)$ means that the expected value is being taken with respect to the multivariate random variable, $\mathbf{E}$ (for experience), whose values (or realizations) are $\mathbf{e}=(s,a,r,s')$, and that follows the distribution $U(D)$ (a uniform distribution), that is, $\mathbf{e}=(s,a,r,s')$ are uniformly drawn from the experience replay buffer, $D$ (or $\mathbf{E} \sim U(D)$). However, let's ignore or omit this subscript for now (because there are no other random variables in \ref{3}, given that $\gamma$ and $a'$ should not be random variables, thus it should not be ambiguous with respect to which random variable the expectation is being calculated), so \ref{3} can be written as
$$\nabla_{\theta}  \mathbb{E} \left[ \left(r + \gamma \max_{a'} Q(s',a',\theta^{-}) - Q(s,a; \theta ) \right)^2 \right] \label{4} \tag{4} $$
Now, recall that, in the case of a discrete random variable, the expected value is a weighted sum. In the case of a continuous random variable, it is an integral. So, if $\mathbf{E}$ is a continuous random variable, then the expectation \ref{4} can be expanded to
$$
\int_{\mathbb{D}} {\left(r + \gamma \max_{a'} Q(s',a',\theta^{-}) - Q(s,a; \theta )\right)}^2 f(\mathbf{e}) d\mathbf{e}
$$
where $f$ is the density function associated with $\mathbf{E}$ and $\mathbb{D}$ is the domain of the random variable $\mathbf{E}$.
The derivative of an integral can be calculated with the Leibniz integral rule. In the case the bounds of the integration are constants ($a$ and $b$), then the Leibniz integral rule reduces to
$${\displaystyle {\frac {d}{dx}}\left(\int _{a}^{b}f(x,t)\,dt\right)=\int _{a}^{b}{\frac {\partial }{\partial x}}f(x,t)\,dt.}$$
Observe that the derivative is taken with respect to the variable $x$, while the integration is taken with respect to variable $t$.
In our case, the domain of integration, $\mathbb{D}$, is constant because it only represents all experience in the dataset, $\mathcal{D}$. Therefore, the gradient in \ref{4} can be written as
\begin{align}
\nabla_{\theta} \int_{\mathbb{D}} \left(r + \gamma \max_{a'} Q(s',a',\theta^{-}) - Q(s,a; \theta )\right)^2 f(\mathbf{e}) d\mathbf{e}
&= \\
\int_{\mathbb{D}} \nabla_{\theta} \left( \left( r + \gamma \max_{a'} Q(s',a',\theta^{-}) - Q(s,a; \theta )\right)^2  f(\mathbf{e}) \right) d\mathbf{e}
&=\\
\mathbb{E} \left[ \nabla_{\theta} { \left(r + \gamma \max_{a'} Q(s',a',\theta^{-}) - Q(s,a; \theta) \right)}^2 \right] \label{5}\tag{5}
\end{align}
Recall that the derivative of $f(x)=x^2$ is $f'(x)=2x$ and that the derivative of a constant is zero. Note now that the only term in \ref{5} that contains $\theta$ is $Q(s,a; \theta)$, so all other terms are constant with respect to $\theta$. Hence, \ref{5} can be written as
\begin{align}
\mathbb{E} \left[ \nabla_{\theta} {(r + \gamma \max_{a'} Q(s',a',\theta^{-}) - Q(s,a; \theta))}^2 \right]
&=\\
\mathbb{E} \left[ 2 {(r + \gamma \max_{a'} Q(s',a',\theta^{-}) - Q(s,a; \theta))} \nabla_{\theta} \left( {r + \gamma \max_{a'} Q(s',a',\theta^{-}) - Q(s,a; \theta)} \right) \right]
&=\\
\mathbb{E} \left[ 2 {(r + \gamma \max_{a'} Q(s',a',\theta^{-}) - Q(s,a; \theta))}  \left( {\nabla_{\theta} r + \nabla_{\theta} \gamma \max_{a'} Q(s',a',\theta^{-}) - \nabla_{\theta} Q(s,a; \theta)} \right) \right]
&=\\
\mathbb{E} \left[ - 2 {(r + \gamma \max_{a'} Q(s',a',\theta^{-}) - Q(s,a; \theta))} {\nabla_{\theta} Q(s,a; \theta)} \right]
&=\\
-2 \mathbb{E} \left[ {(r + \gamma \max_{a'} Q(s',a',\theta^{-}) - Q(s,a; \theta))}{\nabla_{\theta} Q(s,a; \theta)} \right]
\end{align}
The $-2$ disappears because it can be absorbed by the learning rate.


Can we use ML to do anything else other than predicting (in the case of mathematical problems)?

There are quite a few examples of papers where they try and 'teach' neural networks to 'learn' how to solve math problems. Most of the time, sadly, it comes down to training on a large dataset after which the network can 'solve' the sort of basic problems, but is unable to generalize this to larger problems. That is, if you train a neural network to solve addition, it will be inherently constrained by the dataset. It might be able to semi-sufficiently solve addition with 3 or even 4 digits, depending on how big your dataset is, but throw in an addition question containing 2 10 digit numbers, and it will almost always fail.
The latest example that I can remember where they tried this is in the General Language Model GPT-3, which was not made to solve equations per se, but does 'a decent job' on the stuff that was in the dataset. Facebook AI made an 'advanced math solver' with a specific architecture that i have not looked into which might disproof my point, but you can look into that.
In the end, this comes down to 'what is learning' and 'what do you want to accomplish'. Most agree that these network are not able to generalize beyond their datasets. Some might say that not being able to generalize does not mean that it is not learning. It might just be learning slower. I believe that these models are inherently limited to what is presented in the dataset. Given a good dataset, it might be able to generalize to cases 'near and in-between', but I have yet to see a case where this sort of stuff generalizes to cases 'far outside' the dataset.


What research has been done in the domain of "Identifying sarcasm in text"?

The following survey article by researchers from IIT Bombay summarizes recent advances in sarcasm detection: Arxiv link.

In reference to your question, I do not think it is considered either extraordinarily difficult or open-ended. While it does introduce ambiguity that computers cannot yet handle, Humans are easily able to understand sarcasm, and are thus able to label datasets for sarcasm detection.


Is it possible to restore scrambled/damaged text?

You can rephrase your question as a statistical problem, where you for each word or n-gram try to predict the most likely correctly spelled version of each word or n-gram based on your knowledge of the distributions of these across a corpus of text in the target language.

This is how spell-checkers work.

Peter Norvig (who wrote Artificial Intelligence: A Moderne Approach with Stuart Russell) has an article on the this with a short, easy-to-read code example and a description of the statistics behind it on his web page: How to write a Spelling Corrector.


How can I develop an object detection system that counts the number of objects and determines their position in an image?

If you want to get experience, you should probably start with some easier task. Object detection and localization are relatively hard and writing a neural network and image processing pipeline from scratch will take you a long time.
If you want to build up an intuition about how NN's work, you might want to code some simple task from scratch. This is an example.
When you got some intuition about how NN's work, then you should proceed to your task. In here, you have a similar question and answer provided. The current state-of-the-art approach for your task would probably be point 3, that is object detection network, like YOLO or Faster-RCNN.


Using neural network to recognise patterns in matrices

Would I be right in saying that this becomes a sort of 'pattern recognition' problem?

Technically, yes. In practice: no.

I think you might be interpreting the term "pattern recognition" a bit too literal. Even though wikipedia defines Pattern recognition as "a branch of machine learning that focuses on the recognition of patterns and regularities in data", it's not about solving problems that can "easily" be deduced by logical reasoning.

E.g. you say that

  A '1' in the top right triangle in the matrix represents a convex relationship between two faces and a '1' in the bottom left triangle represents a concave relationship

This is true always. In a typical machine learning situation, you wouldn't (usually) have this prior knowledge. At least not to the extent that it would b be tractable to “solve by hand”.

Pattern recognition is conventionally a statistical approach to solving problems when they get too complex to analyze with conventional logical reasoning and simpler regression models. Wikipedia also states (with a source) that pattern recognition "in some cases considered to be nearly synonymous with machine learning".

That being said: you could use pattern recognition on this problem. However, it seems like overkill in this case. Your problem, as far as I can understand, has an actual "analytical" solution. That is: you can, by logic, get a 100% correct result all the time. Machine learning algorithms could, in theory, also do this, and in that case, and this branch of ML is referred to as Meta Modelling[1].

  For example, if I supply the network with a number of training models - along with labels which describe the design feature which exists in the model, would the network learn to recognise specific adjacency patterns represented in the matrix which relate to certain design features?

In a word: Probably. Best way to go? Probably not. Why not, you ask?

There is always the possibility that your model doesn't learn exactly what you want. In addition you have many challenges like overfitting that you'd need to concern yourself about. It's a statistical approach, as I said. Even if it classifies all your test data as 100% correct, there is no way (unless you check the insanely intractable maths) to be 100% sure that it will always classify correctly. I further suspect that you're also likely to end up spending more time working on your model then the time it would take to just deduce the logic.

I also disagree with @Bitzel: I would not do a CNN (convolutional neural network) on this. CNNs are used when you want to look at specific parts of the matrix, and the relation and connectedness between the pixels are important — for example on images. Since you only have 1s and 0s, I strongly suspect that a CNN would be vastly overkill. And with all the sparsity (many zeros) you’d end up with a lot of zeros in the convolutions.

I'd actually suggest a plain vanilla (feed forward) neural network, which, despite the sparsity, I think will be able to do this classification pretty easily.


What is the best programming language to learn to implement genetic algorithms?

There is no "best language" for any problem. There are too many variables to consider, even when advising a single person with a single project plan.

If the choice is between Python and C++, I would generally advise:

If you want to implement from scratch and learn how the algorithm works, use Python with numeric/accelerated libraries such as NumPy or PyTorch. Python script is quicker to prototype and try different ideas, due to loose variable typing and built-in high-level structures such as dict and list.
If you want to write core, efficient libraries, then C++ will out-perform Python, but writing these will take longer. There are plenty of C++ libraries available (e.g. TensorFlow is available with a C++ API), but the community around them is less focused than with Python toolkits.

You can also combine both approaches and write specific libraries in C or C++ to improve performance at any time.

With Genetic Algorithms, the speed bottleneck is most often population assessment - e.g. running the environment simulation to get a fitness measure for each individual - and not the GA itself. So if you have a specific problem or problem domain in mind, may want to orient your choice around a language that already has support for the kind of environments where you want to run your GA. GAs usually benefit greatly from parallelisation, so if you are aiming for something ambitious you will want to look into GPU support and/or distributed computing toolkits.

Most researchers/hobbyists working in AI-related fields end up using multiple languages over time. You may end up with a favourite language environment, which might be Python, Julia, Java, C++, C, C#, Lua, LISP, Prolog, Matlab/Octave, R . . . but you will end up needing a smattering of other languages, and usually skills with specific toolkits such as TensorFlow, Scikit-Learn, Hadoop etc in order to complete projects.

Don't be afraid that you will "waste time" learning in one language initially then needing to transfer to another. Your learning of algorithms will be transferable, and your first attempts will most likely not be that re-usable as library code anyway, so you are going to re-implement your ideas, perhaps many times. My first simulated annealing project was written in Fortran 77 . . . 20 years later I dug that knowledge up again and implemented in Ruby/C - nowadays I work in Python for the AI/hobby stuff even though my professional career sees me working mostly in Ruby.


Why are traditional ML models still used over deep neural networks?

This question is very broad, so let me attempt to answer it using my own background in time series analysis.

As an example, why would I continue using ARIMA to forecast a time series? Why not simply use an LSTM model by default, since this is a type of recurrent neural network that takes time-related dependencies into account?

Well, an LSTM model is not good at modelling all time series. It is effective when it comes to modelling volatile data, but ARIMA still outperforms when it comes to forecasting trend data - LSTM tends to overemphasise volatile patterns in future predictions.

Let's take an example of forecasting weekly hotel cancellations by potential customers. The second time series shows much more variability in the number of weekly hotel cancellations than the first:

H1 Time Series

H2 Time Series

Based on MDA (mean directional accuracy), RMSE (root mean squared error), and MFE (mean forecast error) - ARIMA demonstrates superior performance overall for the first time series, while LSTM shows better performance for the second:

On the basis of this example - which is quite specific given the broadness of your question - deep learning techniques are not always used because simpler models can perform better under certain circumstances. It is all about understanding the data you are working with and then choosing the model - not the other way around.


Why is Sanskrit the best language for AI?

Rick Briggs refers to the difficulty an artificial intelligence would have in detecting the true meaning of words spoken or written in one of our natural languages. Take for example an artificial intelligence attempting to determine the meaning of a sarcastic sentence.

Naturally spoken, the sentence "That's just what I needed today!" can be the expression of very different feelings. In one instance, a happy individual finding an item that had been lost for some time could be excited or cheered up from the event, and exclaim that this moment of triumph was exactly what their day needed to continue to be happy. On the other hand, a disgruntled office employee having a rough day could accidentally worsen his situation by spilling hot coffee on himself, and sarcastically exclaim that this further annoyance was exactly what he needed today. This sentence should in this situation be interpreted as the man expressing that spilling coffee on himself made his bad day worse.

This is one small example explaining the reason linguistic analysis is difficult for artificial intelligence. When this example is spoken, small tonal fluctuations and indicators are extremely difficult for an AI with a microphone to detect accurately; and if the sentence was simply read, without context how would one example be discernible from the other?

Rick Briggs suggests that Sanskrit, an ancient form of communication, is a naturally spoken language with mechanics and grammatical rules that would allow an artificial intelligence to more accurately interpret sentences during linguistic analysis. More accurate linguistic analysis would result in an artificial intelligence being able to respond more accurately. You can read more about Rick Brigg's thoughts on the language here.


Different useful approaches of implementing real-time AI?

About 15 years ago, John Laird's group at Michigan used the Soar rule-based architecture to play several FPS games effectively (Quake II, Descent III):

http://ai.eecs.umich.edu/people/laird/games_research.html

Here's Laird's overview article from 'Computer':

https://www.researchgate.net/profile/John_Laird/publication/2955463_Laird_JE_Using_a_computer_game_to_develop_advanced_AI_Computer_34_70-75/links/54d0f59a0cf20323c21a1bd7/Laird-JE-Using-a-computer-game-to-develop-advanced-AI-Computer-34-70-75.pdf


What type of AI would you recommend for this complex problem?

You are trying to solve a variant of The Halting Problem, which is the problem of detecting whether a computer program is going to stop, or run forever.

The Halting Problem is incomputable, which means it is not possible to write a computer program that solves it. It is easy to see that your problem is also incomputable. If you could predict whether a program would generate an error, then for any program X that someone wants to solve the halting problem for, we could write a new program:

run(X)
Error("X finished running").

and use your algorithm to determine whether X would finish running or not. Since the halting problem is known to be incomputable, this means your problem must be incomputable too.

That's not to say all is lost though. Formal verification is a field that uses some AI techniques (mostly reasoning-based, but I think there's some machine learning now too) to try to solve this problem for some programs. It can't work for every program though.


Using word embedding to extend words for searching POI names

I think word embeddings are overkill in this particular case.

My suggestion would be to go for a simple dictionary based approach: compose sets of semantically related words, and then use those to expand your query terms. This might take a bit longer to set up, but has several advantages:

simplicity: you can't make many mistakes with this
transparency: you know exactly why a certain term matches, and another one doesn't
accuracy: you have tight control over the whole process; if some term is wrong, you remove it from the set. You cannot do that with embeddings
resources: a dictionary-based approach is far simpler and needs less storage

'Old tech' doesn't sound as sexy as the latest deep learning stuff, but unless you want to have this as a toy project to learn about how to do things with embeddings I would say the latter are the wrong tool for the job. At least you can be sure that it works, and if it doesn't you can easily fix it.


Which neural network is appropriate for measuring object dimensions from stereo images?

Is the image taken from a constant distance?

If yes, you'd need to scale the images to the same dimensions first of all. For few images say 100-500 images (more the better) you'd need to label the dataset by proper scaling.

Once labeled, use it to train a CNN (Although best would be training a ResNet). Once trained with decent accuracy, test it for the rest of your dataset.

I did something similar for one of my projects, check it out if you want to here.


How can I solve part b of exercise 3.6 from the book "Artificial Intelligence: A Modern Approach"?

Initial state: initial position of the monkey.
Possible actions

climb on the crate,
get down the crate,
move the crate from one spot to another,
stack one crate on another,
walk from one spot to another,
grab bananas (if standing on the crate)

Goal test: did the monkey get the bananas?
Cost function: the number of actions completed


Are perfect and imperfect information games modelled as fully and partially observable environments, respectively?

There is indeed a close parallel here, but the concepts are distinct. Every perfect information game is fully observable, but not every fully observable game is a game of perfect information.

A game of imperfect information is one in which you lack knowledge of any of the following:

The state of the game (e.g. current market prices).
The rewards you will receive from various states (i.e. utility and cost functions).

In contrast, in partially observable process (specifically, a POMDP), the requirement is that you must not know which state you are in.

This is a subtle distinction, so here are some examples:

A multi-armed bandit game with stationary distributions. Here, you know which state you are in (in fact, if the distributions are stationary, you know that the state doesn't change, except for the value of your winnings). You are not in a POMDP (the game is fully observable), but you are operating with imperfect information, because you don't know the utility function associated with different actions. You are operating in a regular MDP.
The game of chess has perfect information, and is also thus fully observable.
The game of poker has imperfect information because you cannot observe the current state of the game (you can't see the cards in your opponent's hand). It is thus a POMDP.


Is an artificial intelligence software or hardware?

AI or Artificial Intelligence is nothing but intelligence but in its artificial form. Intelligence comes from formation of rules and patterns in the data which is seen or on which it is trained.

For us, we programme or formulate these rules in Mathematics on which an Intelligence is created.
Ex. A neural network shows a lower level of Intelligence. It is created using basic rules of mathematics like gradient descent, derivatives and other arithmetic stuff.
These rules need a base to form themselves. Here's where computers come in. They run these rules on which intelligence thrives in binary ( or quantum sometimes ).
But, every bit of information acquires physical space for processing and storage. Here's where hardware comes. Hardware provides the container in which  information will be processed.

Hence, for an complete AI system, computer programmes provide a soul whereas hardware provides a body/physical form for the soul.

Since, only AI is concerned, it may run on any hardware as it is a software or program or a set of mathematical rules in binary state.
Hence AI is a program.


Bayesian hyperparameter optimization, is it worth it?

Efficiently integrating HPO frameworks into an existing project is non-trivial. Most common datasets/tasks already have established architectures/hyperparameters/etc. and require only a few additional tuning parameters. In this case, the benefits (assuming they exist) brought by Bayesian HPO techniques lack behind development time (simplicity), and this is one dominant reason that users prefer grid search or random search over some more sophisticated Bayesian optimization techniques.
For a very large scale problem and an entirely new task (or dataset) for which the user has no intuition on "good hyperparameters", Bayesian HPO may be a good option to consider over random or grid search. This is because there may be a very large number of hyperparameters (due to insufficient domain knowledge), and integrating HPO into the project may result in far less time than grid search over all possible hyperparameters.


How exactly is the value of each node determined? Is it the same formula throughout the network?

I will overly simplify ANNs in order to point how they work. Examples might not be 100% accurate.

In the simplest form, network is trained using the apriori information extracted from the ground truth. This basically means that ANN uses the relation between the input and output.

For instance, if you are to classify shrubs and trees, one of the input could be height and the other could be the width of the tree. Now, if you have only input and output layers, increasing height means increasing chance for the object to be a tree. Thus, input height would have a positive weight connecting to tree output and a negative weight to shrub output. However, as the plant gets wider, the chance of it being a shrub increases. Taller shrubs are wider than shorter ones. Thus input weight would have positive weight connecting to the shrub output. Finally, the chance of being a tree is not affected by the width and thus will have close to 0 weight between this input and output. This network will effectively work like a linear discriminant classifier.

Now instead of assigning weights by hand, you may use a learning algorithm that tries to adjust weights so that the output is correct when the series of input is supplied. Ideally this training algorithm should reach to the conclusion that we have made in the previous example. Most training algorithms are recursive. They supply the inputs multiple times, and in a simple sense, they reward pathways that are correct by increasing their weights and punishes pathways that are causing incorrect answer.

When hidden layers are used in a system, they would be able to correlate input on higher degrees. Thus, as the number of layers get higher, ANN learns the input set much better. However, this does not mean it gets better. If the ANN over fits the input set, it would be affected from the random noise that is in the dataset. This problem is generally referred as memorization. There are learning algorithms that try to minimize memorization and maximize generalization ability. But ultimately, the number of training samples should be high enough so that ANN cannot overfit to the data.


Is the working of RNNs, LSTM and GRU sequential or parallel?

Yes, you are correct and it was one of original motivations, which inspired the invention of the Attention mechanism in seq2seq problems https://arxiv.org/pdf/1706.03762.pdf.
There is a quote from this paper:

Recurrent models typically factor computation along the symbol
positions of the input and output sequences. Aligning the positions to
steps in computation time, they generate a sequence of hidden states $h_t$,
as a function of the previous hidden state $h_{t−1}$ and the input for
position $t$. This inherently sequential nature precludes parallelization
within training examples, which becomes critical at longer sequence
lengths, as memory constraints limit batching across examples.

On the other hand, Transformer architectures have a loot loom for parallelization, because they take the whole sequence at once, and multiple heads can be executed in parallel.


Why are the initial weights of neural networks randomly initialised?

You shouldn't assign all to 0.5 because you'd have the "break symmetry" issue.

http://www.deeplearningbook.org/contents/optimization.html
Perhaps the only property known with complete certainty is that the initial
parameters need to “break symmetry” between different units. If two hidden
units with the same activation function are connected to the same inputs, then
these units must have different initial parameters. If they have the same initial
parameters, then a deterministic learning algorithm applied to a deterministic cost
and model will constantly update both of these units in the same way. Even if the
model or training algorithm is capable of using stochasticity to compute different
updates for different units (for example, if one trains with dropout), it is usually
best to initialize each unit to compute a different function from all of the other
units. This may help to make sure that no input patterns are lost in the null
space of forward propagation and no gradient patterns are lost in the null space
of back-propagation.


How can I use autoencoders to analyze patterns and classify them?

When using an autoencoder, I believe the data u feed in has to be correlated in one way or another. For example, If i want to learn a latent representation of an image of a cat, The training data that I feed into the autoencoder should constitute only cat images.

Similar to other neural networks, you feed the autoencoder with a set of training data and hope that the network learns a set of weights that is able to output from the latent representation the exact image. To see whether the weights learnt by the autoencoder is able to generalise to other unseen cat images, you would have to use a test set for this.

Here is a paper about autoencoding. https://web.stanford.edu/class/cs294a/sparseAutoencoder_2011new.pdf

I hope this helps you somewhat in deciding whether you should use an autoencoder.


The relation between the human face perception and the keyword spotting in speech recognition?

Specialised neural circuitry to recognise faces is common in all our closest animal relatives. This means that it is likely an evolutionary adaptation that is many millions of years old. Babies can pay attention to faces basically from the moment they are born. There are certain types of brain damage that make it impossible to recognise faces.

Being addressed by an individual name (as opposed to broadcasting an individual signature) is likely a much more recent phenomenon. I have never heard of babies having any special propensity to pick up on names and I have never heard of brain damage that makes it selectively impossible to hear your name.

Also, face recognition can be hardwired because faces always look roughly the same. Names are always different. If you have to learn them anyway, it is more likely that it just happens with you general auditory brain stuff.

And it strikes me as likely that the speed that comes from special circuitry (recognising something early in the cortical hierarchy) is a lot more essential when it comes to faces (and snakes and spiders) then your own name.

So, while humans pay close attention to every sound that might be their name, I doubt there is specialised neural circuitry for that.


How to represent integer values in sequence to sequence prediction task in encoder-decoder LSTM?

One hot encoding helps a lot. You can actually one hot encode each batch just before the training to reduce memory usage. One hot encoding makes the input data more intuitive for the network as a numeric value requires the network to do multiple comparisons to understand the value. For examples please tell me which deep learning framework you are using so I can give you some links or code. Hope I can help you.


Nonbinary and binary values in input tensor

If the two inputs for the binary values are held to the domain set $\{0, 1\}$ during training, testing, and use, it will not break the network functionality, although the question is a good one. Why use so many bits to hold one?

Theory of Holors: A Generalization of Tensors, by Parry Moon and Domina Spencer (ISBNs 978-0521019002 and 0521019001), propose holors precisely because of this kind of limitation on the natural heterogeneity of numerical structure. Gibbs, Ricci, Einstein, and others used holors in their mathematical expressions but without the name holor. Holors are are conceptually related to objects in ontology and classes in object oriented design, but they are numbers, so they fit into mathematical expressions as scalars, vectors, matrices, vector fields, and other tensors do.

This is the basis for Coplien's operator overload in C++ and one of the reasons the language is too mutable to be mastered by average applications programmers. Support for what Moon and Spenser called holors is also the basis of decoupled type safety in some of the early LISP object oriented frameworks.

The current programming paradigm we seen in Java, Python, and other popular languages was set in FORTRAN, which was more attainable for average programmers, so we are stuck with homogeneous structure, even in AI libraries and GPUs. Therefore, once one number requires an IEEE 64 float, bytes and bits will use all 64.

In artificial networks, this only matters in the very first multiplication with a parameter.


What types of AI agents are Djikstra's algorithm and Prim's Minimum Spanning Tree algorithm?

Why do you want to think of these algorithms as agents?

An agent is an abstract and higher-level concept than the concept of an algorithm, which is just a set of instructions.

You could have two agents, one that is supposed to find the minimum spanning tree and another that is supposed to find the shortest path between a source and goal nodes.  In both cases, these two agents have a goal, so they are both goal-oriented agents, but the algorithms they use to reach the goal are irrelevant, as long as they have one or more goals. If you have an agent that performs optimally with respect to some metric, you may want to call that agent an optimal agent.

A similar logic applies to utility-based agents.

So, in short, algorithms are used by agents to act, but the algorithms themselves are not the agents (unless you want to have a philosophical debate or use different definitions of agents). See this answer for more details about the definition of agents.


Should Q values be changing within an epoch/episode or should they change after one episode/epoch?

I was attaining negative values out of my convolutional layers and then using relu on them resulted in the gradient of the activation being 0. Hence, my Q values were not being updated. I've since updated my activations to be ELU. Thanks for the help.


Metrics for evaluating models that output probabilities

For a binary classifier, the cross-entropy loss is a natural measure of probability accuracy, if you care about relative probabilities. By that I mean if you care that the estimate $\hat{p}$ is within some ratio of the true value. So an estimate of $\hat{p} = 0.1$ is a better estimate if the true value is $p = 0.2$ than if the true value is $p = 0.01$ (even though the latter value is closer and would score better under MSE). It also applies that you care that 0.9 is by the same logic "closer" to 0.8 than it is to 0.97. With cross-entropy loss, extreme confidence (predicting close to $0$ or close to $1$) is penalised more heavily when it is wrong.

For completeness, the loss function (per data point) is:

$$\mathcal{L}(\hat{y},y) = -(y\text{log}(\hat{y})+ (1-y)\text{log}(1-\hat{y}))$$

This is likely to be the same loss function as you are using for your objective (or at least it should be), so for tests, simply also use it as your metric*.

$\hat{y}$ is your predicted probability of being in class A, and can be in range $[0,1]$. Ideally you have ground truth probabilities and $y$ is also in that range. In which case the only problem is that the "perfect" score is no longer $0$ but some positive number. If that bothers you, then you could offset by the perfect score, pre-calculating it on each data set (just set $\hat{y} = y$ for each item and you will find the minimum possible score).

If you don't have ground truth probabilities, but you do have classes, then $y$ will either be 0 or 1, and the metric still works. To get an accurate metric, you will need enough samples that the relative frequencies of each class depending on input has a significant effect. That is, you need more data, both training and test, in order to train for accurate probabilities instead of targeting simpler classification accuracy metrics.

Similar logic also works for multi-class probabilities. However, many off-the-shelf libraries use an optimisation in the loss function - assuming only one true class - which makes using probabilities as ground truth impossible. You might therefore need to write your own loss function and gradient functions based on multi-class cross-entropy loss in that case.

* I am making the assumption here that you use standard conventions for noting loss functions (typically per item), cost functions (typically aggregated across a data set and possibly multiple loss functions) and metric functions which don't have to be differentiable or usable as either of the former. The cost function is usually also fed into optimisers as the objective function - i.e. it has the important job of driving parameters to reach a maximum or minimum value. For gradient based solvers, that means it must be differentiable.


Shouldn't Gödel's incompleteness theorems disprove the physical symbol system hypothesis?

The PSSH is often attacked via either Godel's theorems or Turing's incomputability theorem.
However, both attacks have an implicit assumption: that to be intelligent is to be able to decide undecidable questions. It's really not clear that this is so.
Consider what Godel's theorems say, in essence:

"powerful" formal systems cannot prove, using only techniques from within the system, that they are self-consistent.
There are statements that are true that cannot be proven within a given "powerful" formal system.

Suppose that we allow both of those facts. The missing step in the argument is the following statements:

You need to be able to prove the consistency of your own reasoning system to be considered intelligent.
You need to be able to correct reason out a proof of all true statements to be considered intelligent.

The main problem is, under this definition, humans are probably not considered intelligent! I certainly have no way to prove that my reasoning is sound and self-consistent. Moreover, it is objectively not so! I frequently believe contradictory things at the same time.
I also am not able to reason out proofs of all the statements that appear to be true, and it seems entirely plausible that I cannot do so because of the inherent limitations of the logical systems I'm reasoning with.
This is a contradiction. The overall argument was that one of these 4 statements is false:

Godel's theorems say symbol systems lack some important properties.
Intelligent things have the properties that Godel says symbol systems lack.
Humans are intelligent.
Humans can't do the things Godel says symbol systems can't do.

Some authors (like John Searle) might argue the false premise is 4. Most modern AI researchers would argue that the false premise is 2. Since intelligence is a bit nebulous, which view is correct may rely on metaphysical assumptions, but most people agree on premises 1 & 3.


If your game agent performs any kind of advance learning from self play or database of moves, that will generate parameters for some kind of model (e.g. a table of expected values, or neural network weights to select a preferred action). This is unavoidable, and if you want to re-use the results of that machine learning, you absolutely have to store the parameters somewhere.

  if there are other RL methods that produce similar results (consistently beating the human player) without the need to store the policy and comes the correct decision 'on the fly' - would this be this off-policy?

If your agent can access a model that accurately predicts (or accurately samples) outcomes from actions, it can look ahead and plan from the current state. Typically board games and the like do allow you to have such a model, based on the game rules. Some look-ahead techniques are essentially RL methods applied with a focus on solving a decision "just in time", others are more related to search. Common techniques used in game playing are A* search, minimax search (with alpha-beta pruning for performance improvement), Monte Carlo Tree Search. Used purely with just some pre-coded heuristics and a game model, these search techniques do not require you to store model parameters. The downside is that you must spend more computing resources per game move in order to run the search/planning and drive the policy. However if your game is simple, or your heuristics good, or you don't mind a significant wait per computer move, then this approach can be very effective.

This difference between learning and planning is not directly about on-policy vs off-policy, although planning may be considered off-policy as it assesses many actions, most of which the agent does not take.

What a planning method typically looks like in terms of data is a memory-based representation of a game tree starting from the current position, with internal "scores" very similar to RL value functions used to track consequences of action choices. Search and planning methods each have different ways to prune the tree down to a reasonable size and select which game states and actions to look ahead further in. Using "pure" search allows you to work exclusively with this temporary in-memory representation, discarding it after it selects a next action.

You have probably heard of AlphaGo, AlphaGo Zero etc, which are state-of-the-art game playing agents that learn through self play. These, and similar agents, use a combination of a learned policy (from self-play), plus a look-ahead search which refines it for the current game position, getting the best of both learning and planning. The parameters for any learned policy do have to be stored of course. But the benefit of the combined approach is that you can balance resources placed into general learning of the game (stored as parameters to a value function or policy) and specific choice of action for a current game position, which can be much more focussed.


Is it possible to make a neural network to solve this "reaction time test"?

Basically, what you want to do is an anomaly detection at the fastest speed possible. You need to sample the image at two timesteps and if there is a difference you click. The smaller the difference between your timesteps the better it is.
But, you would not need a neural network for it. Since, it is a single colour that will change. A color is represented by a 3-tuple (RGB) so you just need to compare them and if they are different then you click. A single if condition will suffice for it.


How is the "right to explanation" reasonable?

How is the right to explanation reasonable, given the current standards at which we hold each other accountable?

In short, it is quite reasonable.

More specifically, making AI accountable and responsible for explaining the decision seems reasonable because

Humans (DARPA in this case) has chosen to create, raise and evolve AI with the tax-payers money. In our society as well, whenever humans have come together for some purpose (some formal arrangement like government or otherwise), accountability and responsibility is assumed and assigned. Thus, expecting AI (who will take decisions on our behalf) to explain their decisions seems only natural extension of how humans currently operate.
Humans (generally at least) don't have super-power or resources to control and manipulate the rest of the population. But when they do (like our political leaders), we want them to be accountable.
In rare cases, powerful humans become powerful due to things which are accessible to everyone, while AI's super-power won't be. So, our civilization would feel safer and thus less insecure with an AI which doesn't shut the door on face when questions are asked.

Other benefits of AI that offers an explanation on how a decision is reached

Far easier to debug and improve in its early stage.
Can be customized further (and along the way) to amplify certain aspects like social-welfare over profitability, etc.


Convolutional Neural Networks for different-sized Source and Target

Yes actually. There have been quite a few different adaptations to convolutional neural networks to do precisely what you are describing.

Here is an earlier one. See section 3.2.5

Here He et al. create a method known as Spatial Pyramid Pooling(SPP). In this method, you are able construct a fixed length representation regardless of input size by pooling them into "spatial bins" which are proportional to the input size, and thusly do not need to modify input dimensions.

There are a few newer methods that improve on this in various capacities, as usual, your solution will be dependent on the problem and other situation-dependent constraints. I suggest you dig deeper into the literature to find the optimal solution in your case.


If the i.i.d. assumption holds, shouldn't the training and validation trends be exactly the same?

If the i.i.d (independent and identically distributed) assumption holds, shouldn't the training and validation trends be exactly the same?

No, not necessarily. Let me explain why.

If you assume your samples (aka examples, observations, data points, etc.) are i.i.d., this means

that they come from the same distribution, e.g. a Gaussian $\mathcal{N}(0, 1)$ (the identically distributed part), and
that they are independently drawn from it, i.e., intuitively, each sample provides the same kind of information independently of the others

However, even if samples are independently drawn from a certain distribution, they can be different. For example, if you draw a sample $x$ from $\mathcal{N}(0, 1)$, an operation often denoted as $x \sim \mathcal{N}(0, 1)$, $x$ could have the value $0$, $1$, $13$ or $50$ (or any other number), so they could be variable, although your samples will tend to be mainly around $0$, because that's where your Gaussian puts more density (and your standard deviation is just $1$). If your standard deviation was higher, then there would be even more variability in the sampling process.

So, if you assume that your samples are independently drawn from a certain distribution, it doesn't mean that you will always get the same pattern of samples. In other words, you can still have variability in your samples, and this also depends on the distribution you sample from.

To answer your question more directly, there's a chance that your training data and your validation data don't necessarily have the same patterns, even if the independence assumption holds. Therefore, the training and validation trends (and I assume you mean e.g. the performance) are not necessarily the same, but, although this may also depend on the training method, I would say that they shouldn't be very different (if the assumption holds) because, intuitively, each sample should be as informative as any other sample (independence assumption).

  Do we work with the fact that there is a certain degree of wrongness to the assumption or am I interpreting it wrongly?

It is often convenient to make the i.i.d. assumption even, if it doesn't hold, for several reasons:

your training procedure may converge faster (because, intuitively, each sample will be as informative as any other sample)
your models may be simpler (e.g., in naive Bayes, you make the i.i.d. assumption only to simplify the model and, in general, the mathematical formulations)

Sometimes, if it doesn't hold, your training procedure can be highly affected. In those cases, you can find workarounds and try to make it hold. For example, the usage of the experience replay in deep Q-learning is an example of a trick used to overcome the dependence of successive samples, which causes learning to be highly variable. See this question Why exactly do neural networks require i.i.d. data?.

The answers to the question On the importance of the i.i.d. assumption in statistical learning on CrossValidated provide more information and details, so you may want to have a look at it too. Here's another answer, which is related to shuffling and how it can or not make the independence assumption hold, that I highly recommend that you read.


What is the derivative of a specific output with respect to a specific weight?

Formally speaking $x_6$ is a function of $w_{16},\ w_{26}$ and $w_{36}$, that is $$x_6 =f(w_{16}, w_{26}, w_{36})=w_{16}y_1 + w_{26}y_2 + w_{36}y_3.$$
The derivative w.r.t. $w_{26}$ is
$$\frac{\partial x_6}{\partial w_{26}}= \frac{\partial w_{16}y_1}{\partial w_{26}} +\frac{\partial w_{26}y_2}{\partial w_{26}}  +\frac{\partial w_{36}y_3}{\partial w_{26}} = 0 +y_2 \frac{\partial w_{26}}{\partial w_{26}} + 0= y_2.$$
The first equality is obtained using the fact that the partial derivative is linear (so the derivative of the sum is the sum of the derivatives); the second equality comes from again from the linearity and from the fact that $w_{16}y_1$ and $w_{36}y_3$ are constants with respect to $w_{26}$, so their partial derivative w.r.t. this variable is $0$.
Bonus
Not really asked in the original question, but since I'm here let me have a bit of fun ;).
Let's say $x_6$ is the output of the sixth node after you apply an activation function, that is $$x_6 =\sigma(f(w_{16}, w_{26}, w_{36}))=\sigma(w_{16}y_1 + w_{26}y_2 + w_{36}y_3).$$
You can compute the partial derivative applying the properties illustrated above, with the additional help of the chain rule
$$\frac{\partial x_6}{\partial w_{26}}=\frac{\partial \sigma(w_{16}y_1 + w_{26}y_2 + w_{36}y_3)}{\partial w_{26}}=\sigma'\frac{\partial w_{16}y_1}{\partial w_{26}} +\sigma'\frac{\partial w_{26}y_2}{\partial w_{26}}  +\sigma'\frac{\partial w_{36}y_3}{\partial w_{26}}=y_2\sigma'$$
$\sigma'$ denotes the derivative of sigma with respect to its argument.