Could error surface shape be useful to detect which local minima is better for generalization?

In general I agree with @nbro answer, nevertheless sticking strictly to this specific question I'd like to share some speculations:

what the author of the question provides us with is the Loss Function Shape so I'll try to use the full information here to compare the 2 minima
looking at the LF steepness we observe the Left LM is in a steeper region than the Right LM: how can this be interpreted?
I'd interpret the LF Steepness as a Measure of Parametrization Stability: in fact perturbating the Parametrization slightly has a bigger effect on Left LM observed performance rather than the Right LM one
when the NN is running "in production" typically the parametrization is fixed (in most typical application, weights are not changed out of the training phase) so you should not be concerned about parametrization stability,
however I'm one of those believing the Flat Minima - Schmidhuber 1997 idea that Local Minima Flatness is connected to generalization property

However it is important to observe this is still a very open and interesting question as in Sharp Minima Can Generalize For Deep Nets Dinh 2017 et al. demonstrated
it is not just about flatness, because reparametrization of the NN model, though preserving minima locations, change its shape so basically sharp minima might be transformed into flat ones without affecting network performance


How to add external training in chatterbot?

I'm not sure if this is the right one, but according to the documentation you need to create a trainer for it:

from chatterbot.trainers import ChatterBotCorpusTrainer

# Create a new trainer for the chatbot
trainer = ChatterBotCorpusTrainer(chatbot)

# Train based on the english corpus
trainer.train("chatterbot.corpus.english")

Instead of "chatterbot.corpus.english" you obviously choose your own training data.


Did you retrain the network after pruning the weights?

If not, 1) the bubbles are probably appearing because of that, and 2) you definitely should retrain your network.

Did you prune output features only or weights inside the network also?

The standard procedure is to retrain the network after pruning, and the best results can be achieved with iterative pruning: you prune small part of weights (the loss of accyracy is inevitable here), then retrain to get the accuracy back, and repeat until you won't be able to get the accuracy back to high level.

Here Pruning Convolutional Neural Networks for Resource Efficient Inference is an article about (among others) iterative pruning.


Tagging parts of speech when proper noun is a composite

I guess your problem is a form of NER (Named-Entity Recognition) tagging. NER tags consist of PER(person), LOC(location), ORG(organization) and MISC(miscellaneous) and O(other). with the help of a NER-tagger algorithm, you probably would have:

My(O) favorite(O) book(O) is(O) harry(B-MISC) potter(I-MISC) and(I-MISC) the(I-MISC) prisoner(I-MISC) of(I-MISC) azkaban(I-MISC).

now you can tokenize your text with identifying different NER tags, and join "B-" and "I-" prefixes with the same NER tag. e.g. you have "harry potter and ..." as a single token, which is a MISC and starts from harry (since harry is B-MISC) and ends to azkaban (since azkaban has the last consecutive I-MISC tag). Now you can let your POS tagger act the "harry potter ..." as a single token, and it must tag it with "NN"

Another solution for this problem is called "chunking". It works based on a set of rules and detect Noun Phrases (NPs). Here, you define a rule and create the regexp statement for it. e.g. you define all consecutive NNs as a single NP (harry potter): <NN>+ or a DT followed by a string of NNs (the brown fox): <DT>?<NN>+. Now you tokenize your text based on a regexp matching method. but in your case, chunking such a long NP is almost impossible. because a rule that finds this NP as a NP candidate, would by mistake find a lot of other string of words as a NP candidate too, while they're not a NP at all. (see nltk: information extraction)


Where do 'random seeds' get used in deep neural networks?

I suppose the most common part where it will be used is in the initialization of weights before training; the best ways currently known to do that involve randomness.

If you use Dropout during training (randomly setting some activation levels to zero to combat overfitting), that also involves randomness, so your seed could also have influence there. Dropout should not be used anymore after training, although it could if you accidentally implement it to be used there. If you don't make that mistake though, your seed here should also only matter during training.

Depending on implementation, I suppose the seed could also have influence on random ordering of input data between epochs / random selection of minibatches from the training dataset during training. Of course, this is very much implementation-dependent. If you end up implement that kind of data processing yourself and don't do it through some framework, you'll also be the one who determines what random seed has influence on that process.

In general (barring any special cases that I'm unaware of), a Neural Network should behave deterministically after training; if you give it the same input, it should provide the same output, your random seed should no longer have influence after training.


Are feature maps merged or are they passed on as they are?

tl;dr
It helps to think that the channels dimension of a convolutional layer works like a fully connected layer (i.e. the layer computes the weighted sum over all channels).
For a single pixel...
Let's consider a single pixel (e.g. the top left pixel). This pixel has $C$ different values, where $C$ are the number of channels. In order to produce the result of a single filter, the layer takes the weighted sum of these $C$ pixels. It does this by actually having $C$ weights, multiplying the pixel values with their corresponding weights and summing them together.
Example
Let's say you have an $n \times m \times 10$ tensor as an input to a convolutional layer with $1$ filter and a $3 \times 3$ kernel. For creating its output the layer has $3 \times 3 \times 10 = 90$ weights, i.e. a different $3 \times 3$ kernel for each of the $10$ input channels. To create its output, the layer performs the convolution operation separately on each of the input channels (each with its corresponding weight matrix) and creates this way $10$ feature maps which are summed together.
Now imagine the layer has $20$ filters instead of $1$. Nothing changes, just that the same procedure is done $20$ times with $20$ different sets of weights So the total number of weights in the layer in this case is $3 \times 3 \times 10 \times 20 = 1800$.
To answer your questions...
(1) You have a grayscale image $n \times m \times 1$ and it passes through the first convolution layer (which has $10$ filters). This layer will perform the convolution operation on the input image $10$ times independently and produce $10$ feature maps, i.e. an output tensor of $n \times m \times 10$. Now this in fed into the second convolution layer, which again has $10$ filters. For each filter the layer will perform the convolution operation on the $10$ input feature maps independently and will sum the corresponding pixels together to form a $n \times m \times 1$ feature map. It will perform this procedure $10$ times and generate an output tensor of $n \times m \times 10$.
(2) Yes, each layer actually has $30$ filters, a total of $10$ for each of the R, G, B channels. The layer just sums the results of the R, G, B channels to produce a single feature map. For the second part, if I got it right it's pretty much what you said but it sums the maps instead of averaging them
Notes
I'd recommend checking Stanford's CS231 notes on convolution layers, which explain it in much detail and also have numerical examples to confirm if you've got it right.
You could also check this answer for more details.


How to handle the final state in experience replay?

You do not store a terminal state as $s$ in the replay table because by definition its value is always $0$, and there is no action, reward or next state. There is literally nothing to learn.
However, you may find it useful to store information that $s'$ is actually a terminal state, in case this is not obvious. That is typically achieved by storing an additional done boolean component. This is useful, because it allows you to branch when calculating the TD target g:
s, a, r, next_s, done = replay_memory_sample()
if done:
  g = r
else:
  g = r + gamma * max( q(s') )


Convolutional layer to Fully Connected Layer implementation

Yes, you are correct (I think it is quite easily implementable in C++ with pointers). The arbitrary order is to be maintained though, since Fully Connected Neural Nets are "Translationally Invariant" i.e. you have to make sure if pixel $(1,5,6)$ is being supplied to node $38$ or being indexed as $37$ as a single datapoint to be input to a Fully Connected Neural Network, then from then on it must be fixed (cannot put say pixel $(1,6,5 )$ in node $38$.

Backpropagation works the same as it always works, it is tough to give a verbal explanation so I will give you this picture:

So, basically if you visualise like this you understand you have to differentiation will propagate, the "flattening" is only reshaping the value lookup table it is not changing the way the values affect final loss, so if you take gradient w.r.t each values and then convert it back to a $3D$ map same way as before and then propagate the gradients as you were doing in previous layers.


Fourier Transform inputs (Frequency) for RNN

Yes you can apply RNN to any sequence of same data type. The sequence can be in space, time, or any arbitrary ordered list. The items in the sequence can have any data at all, the only requirement is that each represents that same kind of thing (if you have multiple types of thing to process as a sequence, you just need to expand the definition so that the input features can represent all types unambiguously - essentially creating a "base class" that can represent them all).

The RNN will consume the sequence as a time-based sequence, one item per time step of the RNN. However, you can think of that as the same as a processor clock for a computer . . . an RNN is essentially a trainable Turing machine, and in principle can learn to accumulate any data about the sequence it has seen, and output any function of that accumulated data. Although in practice this learning process might be too hard for our current systems, require immense amounts of data etc . . .

In your case, STFT does create a time-based sequence. Each item in the sequence is a frequency analysis for a short period of time, and each time step of the sequence represents a fixed time difference between STFT frames (the windows usually overlap a little), where frequencies in the signal may change. Typically each STFT frame is a single time step input to a RNN. You could input the frequency-domain values in fixed order (e.g. low to high frequency) one at a time into a RNN too, but that would be unusual and would make most learning tasks harder.


How do I detect similar objects in an image?

Disclaimer: I have never used Siamese networks

I would approach this problem in two steps:

First: train object detector and train it for eligible classes of object, for example using Yolo architecture.

You could use pretrained object detector and finetune it for your classes of objects.

Second: extract a lot of bounding boxes of eligible objects of the same classes from your dataset and train Siamese network on their subimages.

Siamese network output similarity measure between two objects.

Your pipleline would look like:

Run object detetctor
For each two pair of objects of the same class rescale bounding boxes and run Siamese network.
Check if the similarity distance between two objects is less then threshold (tunable hyperparameter)


Time Series: LSTM or Augmented Vector Space?

LSTM is a neural network which learns for an input x an output y. In additional to CNNs or MLPs it considers a hiddenstate h (which is influenced by prvious inputs) when your next input x is feed into the network.

Augmenting the feature Space is a technique which you do previous training your LSTM (to augment your data set in order to generatre more data and let the LSTM better generalize to new data). In the field of image recognition you can rotate your images by 40 degree to generate a new one. This process is known as data augmentation. Such methods also appliable to time series.

In summary: first, you start with augmenting your input feature space in order to improve prediction accuracy and then training your LSTM with the augmeneted training data set.


Can an AI be made to maintain a train of thought?

First, for almost every question of the form "Can AI be made to X", the most obvious and straightforward answer is something like "We don't know. Probably, but if it hasn't been done yet, we're really not sure."

It's also important to understand that, from a technology standpoint, there isn't one "thing" called "AI".  There are many, many different technologies, which are loosely related (at best) and are generally lumped together under the overall rubric of "Artificial Intelligence".

All of that said, yes, there has been work on adding memory, even long-term memory, to various kinds of "AI".  The most notable recent example is the advent of LSTM in recurrent neural networks.

Additionally, some of the work done on "cognitive architectures" has focused on the use of memory.  For more info on that, look up ACT-R and/or SOAR and read some of those papers.

What isn't clear to me offhand, is whether or not anybody has tried applying any of these techniques to chat-bots in particular.  I wouldn't be surprised if somebody had, but I can't cite any such research off the top of my head.


How does RL based neural architecture search work?

(I will repeat a few details that you're already aware of, so that other users can also understand the context).

In the Neural Architecture Search (NAS) paper (that I mention in my answer to the question you link to in your question), the agent is the controller (see also this question Is there any difference between a control and an action in reinforcement learning?), which is implemented as a recurrent neural network (RNN). This controller produces actions (or controls), which are strings that represent the hyper-parameters of a neural network (see e.g. section 3.2), based on the reward that it receives, which, in this case, is the accuracy on the validation dataset that the designed (by the controller) and trained neural network obtains.

In this context, the controller is thus the reinforcement learning agent (or policy). The controller is an RNN that is represented by a vector of parameters, $\boldsymbol{\theta}_\text{c}$. This controller is trained using a policy gradient method to maximize the expected accuracy on the validation dataset (see e.g. section 3 of the paper). So $\boldsymbol{\theta}_\text{c}$ are adjusted using this policy gradient method so that the controller generates NN architectures that, after trained on some task, produce higher accuracy on a validation dataset.

The objective function of the controller is thus

$$
J(\boldsymbol{\theta}_\text{c}) = \mathbb{E}[R]
$$

where $R$ is the accuracy on the validation dataset. Why expected? You will be performing this operation multiple times, so, intuitively, you want an average of the accuracy that you obtain on the validation dataset using multiple neural network architectures that the controller might produce.

The accuracy on the validation dataset, represented by $R$, is actually non-differentiable, so the authors use the famous REINFORCE algorithm (which I will not explain here). The authors actually use an approximation of the REINFORCE algorithm (see section 3.2).

$$
\frac{1}{m} \sum_{k=1}^m \sum_{t=1}^T \nabla_{\boldsymbol{\theta}_\text{c}} \log P (a_t \mid a_{{t-1}:1}; \boldsymbol{\theta}_\text{c}) R_k
$$

So, given this formulation and taking into account the formulation of the REINFORCE algorithm, then the state of the agent seems to be $a_{{t-1}:1}$, that is, the previous actions of the agent. (Recall that the agent is a controller that is implemented as a recurrent neural network).


Is it possible to classify resistors using ResNet50?

Yes it should be possible. You may have a bug in your code, or the wrong hyperparameters. Training ResNet-50 will take a long time. Try training on other sets of images and see what accuracy you get to check if your approach is correct. Or, try loading a pretrained model, and training from that.


How to rescale data to its original range after MinMaxScaler?

You can use the function inverse_transform of the created MinMaxScaler object.

See also this Stack Overflow question for other answers and examples.


Why are RNNs better than MLPs at predicting time series data?

In an RNN, the output of the previous state is passed as an input to the current state. Intuitively, there is a temporal (time-based) relationship in the way in which input is processed in an RNN. It can understand how the current state was achieved on the basis of the previous values, i.e value at time-step $t$ is a result of value at time-steps $t-1, t-2$, and so on.
In a DNN, there is no temporal relationship in the way input is processed. Values at time-steps $t, t-1, t-2, \dots $ are all treated distinctly and not as a continuation of the previous time-step values.


Meaning of Actor Output in Actor Critic Reinforcement Learning

When using the loss function for the critic described in your question, the Actor-Critic is an on-policy approach (as are most Actor-Critic methods). Your intuition as to what it is learning seems to be quite close, but the notation/terminology is not quite on point.

First it's important to realize that the $Q(s, a)$ critic is an estimator, we're training it to estimate state-action values. You could say that we are training it such that it can hopefully provide accurate estimates of:

$$Q_w^{\pi} (s_t, a_t) \approx \mathbb{E}_{\pi} \left[ r_t + \gamma V^{\pi}(s_{t+1}) \right].$$

You'll notice I've added quite a number of symbols there in comparison to your $Q(s_t, a_t) = r_t + V(s_{t+1})$:

I have added the $\pi$ superscript to $Q$ and $V$; this denotes the behaviour policy, which is the policy that we're using to generate experience. In on-policy methods, this is equal to the target policy (the policy for which we're learning to predict values). Adding this superscript makes explicit the fact that we're learning expected returns for states and state-action pairs that are only accurate under the assumption that we continue following the $\pi$ policy from state $s_{t+1}$ onwards.
I added the discount factor $\gamma$, which is probably just a tiny detail you forgot.
I added $\mathbb{E}_{\pi}$ to indicate that we're trying to estimate an expectation under $\pi$ (and the environment's dynamics).

So, the critic is trained to estimate $Q^{\pi}(s, a)$, which can intuitively be interpreted as the long-term discounted rewards that we expect to collect when executing $a$ in $s$, and selecting actions according to the distribution $\pi$ subsequently. It definitely still is trying to estimate $Q(s, a)$ values for state-action pairs, not just $V(s)$ values for states alone.

  What In actor critic outputs $Q(s_{t+1},a_{t+1})$?

In practice, when using the loss function described in your question, $a_{t+1}$ really simply is a single action selected in an actual trajectory of experience by the policy $\pi$. The trained network simply takes $s_{t+1}$ as input, and the output corresponding to a single action $a_{t+1}$ as selected by the policy is used as the value for $Q(s_{t+1},a_{t+1})$ in the update rule.

The update rule does not involve any sum over all actions, multiplied with their probabilities. The "trick" is that we do not just run the update rule a single time, but we expect to generate lots (sometimes millions) of trajectories as experience, and we repeatedly run the update rule. In different trajectories, we'll experience the different actions $a_t$ as samples with approximately the correct frequencies, and in expectation we'll have proper update targets (except for potential bias resulting from function approximation).

  I want to use actor critic with experience replay in an environment with a large action space

The Actor-Critic method in your question is, as I mentioned above, on-policy. This means that the experience used in update rules has to be generated according to exactly the same policy for which you are also learning value estimates. This is incompatible with the idea of experience replay, because old trajectories stored in a replay buffer were generated by older versions of your policy.

There are off-policy Actor-Critic methods which can correct for the mismatch in distributions and use experience replay, but these are going to be quite a bit more complicated. Examples are ACER and IMPALA.


If uniform cost search is used for bidirectional search, is it guaranteed the solution is optimal?

UCS is optimal (but not necessarily complete)
Let's first recall that the uniform-cost search (UCS) is optimal (i.e. if it finds a solution, which is not guaranteed unless the costs on the edges are big enough, that solution is optimal) and it expands nodes with the smallest value of the evaluation function $f(n) = g(n)$, where $g(n)$ is the length/cost of the path from the goal/start node to $n$.
Is bidirectional search with UCS optimal?
The problem of bidirectional search with UCS for the forward and backward searches is that UCS does not proceed layer-by-layer (as breadth-first search does, which ensures that when the forward and backward searches meet, the optimal path has been found, assuming they both expand one level at each iteration), so the forward search may explore one part of the search space while the backward search may explore a different part, and it could happen (although I don't have the proof: I need to think about it a little bit more!), that these searches do not meet. So, I will consider both cases:

when the forward and backward searches do not "meet" (the worst case, in terms of time and space complexity)

when they meet (the non-degenerate case)

Degenerate case
Let's consider the case when the forward search does not meet the backward search (the worst/degenarate case).
If we assume that the costs on the edges are big enough and the start node $s$ is reachable from $g$ (or vice-versa), then bidirectional search eventually degenerates to two independent uniform-cost searches, which are optimal, which makes BS optimal too.
Non-generate case
Let's consider the case when the forward search meets the backward search.
To ensure optimality, we cannot just stop searching when we take off both the frontiers the same $n$. To see why, consider this example. We take off the first frontier node $n_1$ with cost $N$, then we take off the same frontier node $n_2$ with cost $N+10$. Meanwhile, we take off the other frontier node $n_2$ with cost $K$ and the node $n_1$ with cost $K + 1$. So, we have two paths: one with cost $N+(K + 1)$ and one with cost $(N+10)+K$, which is bigger than $N+(K + 1)$, but we took off both frontiers $n_2$ first.
See the other answer for more details and resources that could be helpful to understand the appropriate stopping condition for the BS.


How does Monte Carlo have high variance?

When using terms like "high" for high variance, this is in comparison to other methods, mainly in comparison to TD learning, which bootstraps between single time steps.

It is worth spelling out what the variance applies to and where it comes from: Namely the Monte Carlo return $G_t$ distribution, which can be calculated as follows:

$$G_t = \sum_{k=0}^{T-t-1}\gamma^k R_{t+k+1}$$

I am using a common convention here by the way that capital letters such as $G_t$ stand for random distributions of variables and lower case letters such as $g_t$ stand for examples/samples from those distributions. It is not always strictly used, especially in pseudo-code, but is important to make clear for this answer.

So a value function, such as action value function $q_{\pi}(s,a)$ can be written:

$$q_{\pi}(s,a) = \mathbb{E}_{\pi} [ \sum_{k=0}^{T-t-1}\gamma^k R_{t+k+1} | S_t=s, A_t=a] = \mathbb{E}_{\pi} [ G_t | S_t=s, A_t=a]$$

i.e. it is fairly clear that the mean of Monte Carlo return $\mathbb{E}[G_t]$ from any specific starting point is the same as the value function from the same starting point, by definition (see Sutton & Barto or David Silver's definitions) so samples from it - taking specific measures $g_t$ - are unbiased estimates of it.

The variance $\text{Var}(G_t)$ might be written

$$\text{Var}(G_t) = \text{Var}(\sum_{k=0}^{T-t-1}\gamma^k R_{t+k+1}) \le \sum_{k=0}^{T-t-1}\gamma^k \text{Var}(R_{t+k+1})$$

The second step is technically equal only in a "worst case" scenario since to add variances directly, we need the variables to be independently sampled, whilst $R_{t}$ and $R_{t+1}$ could be correlated, and often are. So, technically we should account for covariance here between distribitions on different time steps. The maths for that is fiddly, and I won't show it here.

What you can say intuitively about the inequality, is unless $R_{t}$ and $R_{t+1}$ are perfectly correlated throughout each trajectory, then variance of $G_t$ will increase depending on the number of timesteps and possible rewards in the return, by some fraction between 0 and 1 of the total variance of all time steps. You can either add zero variance on a timestep, or some fraction of $\text{Var}(R_{t+n})$ depending on whether the relationship between [$R_t$ to $R_{t+n-1}$] and $R_{t+n}$ is fully deterministic ($+0$) or completely independent ($+\gamma^{n-1}\text{Var}(R_{t+n})$)*

The amount of covariance involved relies on details of the policy, state transition and reward functions. In general, the policy during RL will include some stochastic behaviour -such as $\epsilon$-greedy that will add to the variance on each time step. The state transition and reward functions might add yet more. In any case:

$$\text{Var}(G_t) \ge \text{Var}(R_{t+1})$$

and in practice, when $t+1 \neq T$, the inequality can be large, although highly dependent on the details of the MDP. For off-policy Monte Carlo with basic importance sampling in fact the variance can be unbounded.

This variance might be considered high, compared to the variance of typical single-step TD returns:

$$G_{t:t+1} = R_{t+1} + \gamma q_{\pi}(S_t, A_t)$$

The variance of which is limited to that inherent in a single time step (one reward function, one state transition and one policy decision). The problem of course being that

$$\mathbb{E}[G_{t:t+1}] \neq \mathbb{E}[G_{t}]$$

. . . unless $q_{\pi}(s, a)$ is perfect. Which is why we can say that TD returns are biased (whilst $q_{\pi}(s, a)$ is not converged, the expectation of the TD target does not equal the true value), but Monte Carlo returns are higher variance (when number of timesteps to end of trajectory is more than one, the effects of random factors in policy, state transition and reward function accumulate).

* You could explicitly construct an environment with covariance between time steps that can reduce overall variance of a MDP's retrun, e.g. the state includes reward so far, and future rewards are manipulated so that the return fits to a certain distribution. This would be a special case where variance of the return from some starting points could be lower than the variance from the first step, but this is an unusual set up and likely to be impossible for an agent to learn optimal control.


Backpropogation rule for the output layer of a multi-layer network - What does the rule do in ambiguous cases?

I assume you are considering a network where the activation function of the last layer is a sigmoid, so the output of your network is $$\tilde{y}=\sigma(W^{L}\cdot f(X, W^1, \dots, W^{L-1})),$$
where $X$ is the input vector, and $f$ is obtained by feeding the input to the network up to the layer $L-1$. Let's also call $Z:= W^{L}\cdot f(X, W^1, \dots, W^{L-1})$.
The error term is computed as $$E(y, \tilde{y})=E(y, \sigma(Z)),$$ where $y$ is the actual output. Let's get the derivative of the error with respect to the output of the last node (the input of the sigmoid)
$$\frac{\partial E}{z_i}=\frac{\partial E}{\partial \tilde{y}}\frac{\partial\tilde{y}}{\partial z_i}=\frac{\partial E}{\partial \tilde{y}}\frac{\partial\sigma}{\partial z_i}.$$
The update rule is $$z_i =z_i - C\frac{\partial E}{z_i}= z_i - C\frac{\partial E}{\partial \tilde{y}}\frac{\partial\sigma}{\partial z_i}.$$
Now we can analyse your questions.

To be close to the middle of the sigmoid means that $z_i$ is close to $0$; moreover the derivative $\frac{\partial\sigma}{\partial z_i}$ reaches its maximum value when it is evaluated in $0$. This means that the term $\frac{\partial\sigma}{\partial z_i}$ gets "large" as $z_i$ approaches the center of the sigmoid, contributing more to the update of the weight. Of course it is difficult to say what happens in general, as the term $\frac{\partial E}{\partial \tilde{y}}$ is also in the expression and it is possible that this term gets really small (or big) for $z_i$ close to $0$. You just know that the term $\frac{\partial\sigma}{\partial z_i}$ is trying to push $z_i$ further away from $0$, so the idea should be that the closer to the center the larger the update.
To be close to an extremum point of the loss means that the derivative of the loss with respect to $\tilde{y}$ is close to $0$. Since $\frac{\partial E}{\partial \tilde{y}}$ is backpropagated in a multiplicative fashion, the rule of thumb is that the closer you are to an extremum the smaller the updates get. Though, as above, is kind of difficult to say what will happen when updating a general node, as some of the term in the multiplication can be very large, making the update large even if $\frac{\partial E}{\partial \tilde{y}}$ is small.

For instance, what happens if you are close to the center of the sigmoid but also close to an extremum of the loss? You will have a multiplication of $2$ terms, one trying to make the update small and the other trying to make the update large, and what matter are the orders of magnitude involved.
In conclusion, the rules of thumb are as in the points 1. and 2., but they are no guarantee that you won't find any special cases.


How do I know when to use which Monte Carlo method?

They are all called Monte Carlo because all of them are a different version of the canonical Monte Carlo algorithm.

The canonical version of Monte Carlo algorithm is a stochastic algorithm to determine an action based in a tree representation. The differences among all these version are their exploration and exploitation mechanisms, and it is necessary to analyse each of them to define which one fits in your case.


Price difference predictions curve almost vanished

I have encountered similar problem while trying to predict forex prices. I understand it this way:

  The data based on which you try to model price differences are so poor that the lowest error is achieved by zeroing the predicted values

In other words, due to poor "correlation" in data, zero values are the best solution.

My advice would be to look for more correlated data to be used.


Balancing learning and structure in a neural network

Extending @mirror2image's comment, if you have a certain metric that allows you to measure how close the intended layer is to a low pass filter (something that compares its output with what a low pass filter would have produced, for example), the simplest way to achieve what you want would be to add a term in your loss function that calculates the value of this metric. This way, each time you do a training step, the network now is not only made to output the correct predictions but is also forced to do so while also keeping that specific layer's behavior as close to a low-pass filter as possible.
This is the most common way of tweaking the behavior of neural networks and is often encountered in many research papers.


How can I use machine learning to predict properties (such as the area) of simple polygons?

You can split each polygon into a collection of triangles and sum up the areas. Not really sure why you would bother with ML.

Anyway if you approximates these polygons as images you could maybe train a CNN. Look at the image classification networks which provide bounding boxes.


What are some good alternatives to U-Net for biomedical image segmentation?

Hey i am working on my Bachelor thesis at the moment and use UNET in combination with a GAN for image segmentation. I spend the last 5 months on that, so on my tests, the new approach of januar 2020, called Multires-UNET is quite a good choice for more texture orientated segmentation. I use the current github implementation. Its quite nice, maybe you notice that you can easyly tweak the number of parameters with "alpha" in the implementation, to scale the multiple Resnets in the Unet structure.

I tried also other segmentation networks like Mask_RCNN with different backbones or tried to construct various types of CAE on my own, but always had to come back to a UNET like structure. Same goes for ResU-NET and R2U, the multires one worked better for my purposes cause i didnt need any kind of LSTM modules.

Some examples which may clarify the difference in performance on a specific task:

Original Image:

Ground-Truth:

Classical UNET++ (Unet with skip-connections)(2.5 million parameters) more parameters (wide) didnt change the result.

Multires-UNET (alpha=1.67, think was about 7 million parameters)

Can you show some of ur microscopy images, how complex is ur task and what do you want to segment ?


DQN input representation for a card game

120 inputs can be handled by a complex enough network. Dealing with high complexity is one of NN's strengths.

Using a (120,) vector or a (3,40) matrix is the same, they're still 120 inputs. Your binary encoding should work. Another option is a single (40,) vector, with 0 being "still in deck", 1 being "in hand", 2 being "on table", 3 being "already played".

If the types of cards are irrelevant, you could actually have a (3,10) matrix with counters of cards (1 in hand, 1 in table, 2 already played). You can try different approaches and see what works best.


Transfer learning to train only for a new class while not affecting the predictions of the other class

Even if you want to re-train your model for just one new class you will have to prepare your training data such that it includes all or most of the classes which you want to predict. Most of the times last two layers of a network have the data of number of labels which are to be predicted and that should always be sum of the number of classes you already trained on and the number of classes you want to add for training.


Is true random number generation an AI concept?

As it can be easily pointed out that true random numbers cannot be generated fully by programming and some random seed is required.

This is true. In fact, it is impossible to solve using software. No software-only technique can generate randomness without an initial random seed or support from hardware.

This is also true for AI software. No AI design that uses deterministic software can do this - e.g. any Turing machine without a magic unexplained "random" function can be shown to remain deterministic, no matter how complex. That's because any combination of deterministic functions is deterministic. It may not be predictable without following the process exactly, it may be "chaotic" and depend critically on initial conditions, but it is 100% deterministic and repeatable.

  On the other hand, humans can easily generate any random number independently of other factors.

Typically not high quality randomness. It is not clear how we make random decisions, but entirely possible that internally we rely effectively on noisiness from the environment or our own internal "hardware" doing something simple such as racing decisions between neurons (timing will vary due to speed of electrical impulses, diffusion time of neuro-transmitters across synapses etc)

  Does this suggest that absolute random number generation an AI cocept?

I think it is orthogonal issue. We can already produce very high quality artificial randomness - better than human quality used by humans for conscious decisions (such as "choose a random number between 1 and 10"). These artificial random number generating systems are part of modern cryptography and are tested thoroughly.

Essentially "true" artificial randomness is a solved problem using hardware, and does not involve anything that has traditionally been called AI.

In reverse, AI systems often rely on stochastic functions in order to break symmetry, break ties, regularise models etc. So it does look like some kind of RNG is necessary within an artificial agent. However, even  pseudo-random number generators (PRNGs) seem to be fine for this purpose. Mersenne Twister is a very common choice for generating random numbers inside neural networks for weight initialisation, dataset shuffling, dropout regularisation, or when simulating environments for RL, or taking exploratory actions. Despite the fact that it is not "true" random, a PRNG will work just fine for these purposes.

Working definition of "true" randomness that I use: Even if you know the state of a system as accurately as modern physics allows, the output cannot be predicted with better accuracy than a fixed guess.

Human randomness already fails this test. If you ask someone to choose a number randomly between 1 and 9, you will generally have a better than 1 in 9 chance of guessing the correct value, based on statistical analysis. If we were able to take good state measurements of brains, it might be possible to predict with high accuracy - although this is unknown and not possible with current technology.


OpenAI spinning up convolutional networks with PPO

I have written a tutorial on using OpenAI Spinning Up in a image-based PyBullet + Gym environment here

In order to be able to use spinup for an image-based environment I had to fork it here and add CNN to PPO's core.py


What is the significance of weights in a feedforward neural network?

You described a single-layer feedforward network. They can have multiple layers. The significance of the weights is that they make a linear transformation from the output of the previous layer and hand it to the node they are going to. To say it more simplistically, they specify how important (and in what way: negative or positive) is the activation of node they are coming from to activating the node they are going to.

In your example, since there is only one layer (a row of input nodes and a row of output nodes) it is easy to explain what each node represents. However in multi-layer feedforward networks they can become abstract representations which makes it difficult to explain them and therefore explain what the weights that come to them or go out of them represent.

Another way of thinking about it is that they describe hyperplanes in the space of the output of the previous node layer. If each output from the previous layer represents a point in space, a hyperplane decides which part of the space should give a positive value to the plane's corresponding node in the next layer and which part should give a negative input to it. It actually cuts that space into two halves. If you consider the input space of a multi-layer feedforward network, the weights of the first layer parametrize hyperplanes, however in the next layers they can represent non-linear surfaces in the input space.


The mathematics in the CBOW and Skip-Gram models

The following figure from this article can be helpful:

This figure represents "Skip-Gram model structure. Current center word is 'passes'".


Why do popular object detecting models output heatmaps instead of coordinators of object directly?

To have a more powerful representational output. You can derive a bounding box from heatmap but not vice-versa. Also, in case of dense object detection it is hard to create bounding boxes for each object (people standing in front of each other).
That being the case, it is better to run a segmentation loss for these networks. It also leads to less confusion for the network. Also, it is a single shot creation of output. On the other hand, the bounding box approach will have the creation of a lot of bounding boxes then finding their objectness score and finally applying NMS (non-max suppression).
In regards to focal loss and use of bounding boxes, it is something that is a consequence of the heatmaps/activations that neural networks learns. They are built upon that and then regressed.


Is the 'direction' considered, when determining the branching factor in bidirectional search?

If I am correct, the branching factor is the maximum number of successors of any node

You are correct, they should also be the immediate ones:

If 11 is the goal state and I start going backwards, is 10 considered as successor of 5? Even if it do not leads me further to my start state 1?

No, there is also a bit of misunderstanding of bidirectional search: In bidirectional search you run 2 simultaneous searches, one forward from the initial state, and another one backwards from the goal(hoping they meet in the middle and save you steps), if actions are reversible ( going from node to node), the successor nodes become predecessors in one search and vice versa, and your goal becomes your initial state, in your case:

Reference/source
Artificial Intelligence: A Modern Approach, by S. Russell and P. Norvig.


How can I implement policy evaluation when reward is tied to an action outcome?

The renowned book Reinforcement Learning: An Introduction (2nd edition), by Sutton and Barto, provides a different update rule than your first update rule for policy evaluation. Their update rule is more similar to your second update rule. See section 4.1. They also provide the pseudocode for policy evaluation on page 75 of the book. You can also find the pseudocode here.

Moreover, note that the update rule doesn't need to change only because the rewards are tied to the outcome of an action. This information is encoded in the functions $p$ (the transition function) and $r$ (the reward function) of the Markov decision process, which is incorporated in the update rule. If you want to understand the update rule, you should read the relevant pages (especially, chapter 4) of the cited book.


What is the correct formula for updating the weights in a 1-single hidden layer neural network?

The correct formula for updating the weights between the hidden layer and the output layer is:
$$\Delta W_{j,k} = h_k \ \cdot \ o'_{j} \ \cdot \ (o_j - t_j),$$
where $h$ is the activated hidden layer and $o'$ is the derivative of output layer.
I found this formula in the book Artificial Intelligence: A Guide to Intelligent Systems by Michael Negnevitsky.


Is there a relationship between Computer Algebra and NLP?

There isn't, really. Natural language is way more complex and irregular than algebra, which is far more formalised and unambiguous.
So far, in NLP, most success/progress has been made in little toy domains, which exclude most of the complexities of real life, including many ambiguities.
When you say the rules of algebra are somewhat like grammar, then that is because it is essentially a formal language, for which we can specify a grammar. There is currently no complete grammar for any human language (and I doubt there ever will be), let alone a formal one that can be processed by computer.
This was one of the reasons why the first AI boom, where a lot of over-hyped promises where made about being able to translate Russian into English automatically, failed abysmally: natural languages are more than just formal grammars of lexical items.
Stochastic approaches have gone some way towards pragmatic solutions, but when it comes to understanding language they are basically a fudge. And don't get me started on deep learning approaches to NLP.
So the only relationship is that we use the term 'grammar' for the descriptive formalisms in both cases; a formal grammar of algebra would be very different from a grammar for a human language.
This doesn't mean, however, that approaches developed in the field of NLP cannot be applied to algebra: even those which failed in NLP because they were overly limiting. To find out more about this, look for Chomsky Hierarchy -- that describes the different expressive powers of formal languages.
But I would argue that human language is outside of that, because it is not a formal language.


Which objective tests have been developed to test the intelligence of AI?

For there to be an objective test for AI intelligence, there has to be an objective test for human intelligence.  Each of the standard testing organizations claim theirs to be the most objective, but none of the test creators publish rigorous systems theory to verifiably justify their claim.

The user stories for human selections are varied.

I want to find people to work on my team so the project is completed on time and in budget.
I need to admit the right people for our training so we can have success stories to tell.
We need a focus group with people who can communicate well in groups and think about our products and their market.

With artificial intelligence the bar is much lower.

We need to sort grapefruits for canning, fruit stand sale, use as a sweetener, or use as composting matter in a way to maximize profit.
We need to beat the other automated drone pilot companies.
Most of the world has online business but their documents are redundant with many nearly identical documents from copying and pasting, and most of them are poorly organized.  We need an automated librarian that organizes and indexes them.

The questioner may have hoped for some general intelligence test for AI better than Turing's imitation game, the current standardized intelligence testing only works on humans.  The AI that exists can't, without specifically training for the standardized tests they are going to take, achieve anything like a normal human score.  Cognition is something AI hopes to achieve.

Here is some work in that area.

https://slate.com/technology/2017/10/a-i-s-iq-is-still-somewhat-meaningless.html
https://arxiv.org/pdf/1712.06440.pdf


How should I choose the target's update frequency in DQN?

As you said yourself, it is a hyperparameter. Hence, no one (even you) can say what is the ideal update frequency. You have to test and try.
Having said that, remember one thing the target NN should mimic the actual network as closely as possible. Hence if you update it after a long number runs, then I think you will start losing the accuracy. On the contrary, if you update it too often, then you lose the benefit of using the target network (which is to boost the training rate and reduce training time) and the training will take a larger amount of time.
My suggestion is to try updating after every 5 to 8 episodes.


Why does my "entropy generation" RNN do so badly?

I think there are two problems with your network.
The first one, always having very similar outputs, is the rather simple one. As it seems, your network suffers from the so-called, very common Mode Collapse problem. The attached link provides both an explanation and some potential remedy to that problem.
The second problem is more fundamental. You say that you want your network to produce random numbers. Or numbers that at least appear as such. However, once training is finished, your model is going to be a static function which will not change any further. Given the same input x, it will always produce the same output y. Consequently, unless the inputs to your network contain some randomness already or are, at least, always slightly dissimilar, you will not end up having a random generator. So, whether that is going to be useful will depend on your usecase. But if you make sure that the true random variable (like date&time) serves as input to the RNN and the RNN just translates this into some different format, that might work again. Just keep in mind that randomness can never arise out of a trained model.


Is it suitable to find inverse of last layer's activation function and apply it on the target output?

The main difference your change would have is to allow you to apply a loss function to a different part of the network. This may affect training.

If you keep the same loss function (e.g. MSE), but apply it to the pre-transformed values, then you will have changed the objective of the network, perhaps significantly. Whether or not this is a good thing depends on how much you needed the original loss function. However, the fact that it would result in a different training target is usually going to be a bad thing if your original training target was correct. This will also be true if you pick a new arbitrary loss function that seems to fit the pre-transform representation better.

If you engineer a "correct" loss function such that the objective of the network remains unchanged, then the behaviour of the network will not change much - probably not at all. However, in some cases this can lead to more stable and/or faster training - it is often used for classifiers to avoid need to use exponentiation, see tf.nn.softmax_cross_entropy_with_logits in TensorFlow, which does exactly this.


Why is the E step in expectation maximisation algorithm called so?

In expectation step, firstly we calculate the posterior of latent variable $Z$ and then the $Q(θ | θ^{(t)})$ is defined as the expected value of the log likelihood of $θ$, with respect to the current conditional contribution of $Z$ given $X$ and the current estimates of $θ^{(t)}$. In maximization step, we update $θ$ using the argmax on $Q$, with respect to $θ$.
$$Q(θ | θ^{(t)}) = E_{Z|X,θ^{(t)}}[logL(θ;Χ,Z)]$$
To be more intuitive, think of k-means as a special case of EM, where in expectation step the $Z$ variables are defined, that is the latent variables indicating the membership in a cluster, and calculated in a hard assignment way. In maximization step the $μ$s of the clusters are updated. If you want to see the corresponding relation for $Q$ in k-means, I suggest you read the chapter 9.3.2 in C.Bishop's book: Pattern Recognition and Machine Learning.


Explanation of this L2 minimization equation

As in the second line, the first two terms are $\mathbb{E}_{\hat{y}}$, it means the variable of the expectation is $\hat{y}_i$ and you can take out $y_i$s from $\mathbb{E}_{\hat{y}}$s. Now we can use $\mathbb{E}\{\hat{y}_i\} = y_i$ and rewrite the terms of the second line likes the following:
$$
\mathbb{E}_{\hat{y}}(\sum_i y_i)^2 = (\sum_i y_i)^2 \\
\mathbb{E}_{\hat{y}}\left[(\sum_i y_i)(\sum_i \hat{y}_i)\right] = (\sum_i y_i) \mathbb{E}_{\hat{y}}(\sum_i \hat{y}_i) = (\sum_i y_i)^2
$$
The second line is written based on linearity of the expectation and $\mathbb{E}\{\hat{y}_i\} = y_i$. Hence, we can rewrite the second line like the following as $\sum_i y_i = \mathbb{E}_{\hat{y}}(\sum_i \hat{y}_i)$:
$$
\frac{1}{N^2}\left[\mathbb{E}_{\hat{y}}(\sum_i \hat{y}_i)^2 - \left(\mathbb{E}_{\hat{y}}(\sum_i \hat{y}_i)\right)^2\right]
$$
And the final step is using this formula $Var(X) = E(X^2) - (E(X))^2$  and take $X = \sum_i \hat{y}_i$:
$$
\frac{1}{N^2}\left[\mathbb{E}_{\hat{y}}(\sum_i \hat{y}_i)^2 - \left(\mathbb{E}_{\hat{y}}(\sum_i \hat{y}_i)\right)^2\right] = \frac{1}{N^2} Var(\sum_i \hat{y}_i)
$$
From variance to covariance, you can use this formula: $$Var(\sum_{i=1}^nX_i) = \sum_{i=1}^n\sum_{j=1}^n cov(X_i, X_j)$$


Could style transfer be used to transfer the style of a website from one to another?

The correct way for website styling to be encapsulated and centralized is through the use of one or more CSS style sheets.  For instance, the old  tag is frowned upon and using a text-align:center directive a proper class or ID based CSS selector is considered the proper way.  In such a case, the copying of a style is simply done by the copying of the appropriate style sheet file.

The place where AI and/or machine learning tools may be best used is in the conversion of non-CSS styling to CSS styling.  Once they are in that form, the copying and layering of styling becomes trivial.

To convert a legacy web site to proper CSS style sheet use, the input would be a set of all documents likely to be involved in the styling of a specific page (i.e. its style dependencies).  The output would be a set of corresponding documents with the styling moved into classes and IDs that remove styling redundancy (simplify and centralize the style).

The transformation of the document set could be best accomplished with current technique through transformations, much like code refactoring in typical IDE refactoring algorithms in that the output document set must be functionally the same as the input document set.  Furthermore, the appearance and responsive (size sensitive) aspects must be preserved in addition to functionality.

The greatest usability could be obtained by adding some automation to these refactoring transformations, as in Maxima simplification functions like trigsimp.  This is a typical search of options, perhaps with predictive branching, as in compiler optimizations.

The transformations, to guarantee preservation of function and appearance must essentially be elementary equivalent transformations, but the selection of transformation would probably need to be more advanced than basic antecedent and consequence productions.

Heuristically meta rules, fuzzy logic, neural nets, Bayesian inference, or other tools could be compared in terms of effectiveness in the realm of prioritizing transformations and in terms of scoring results in the search leaves.  Here's where machine learning would probably be necessary to have a truly useful tool.

Training could be accomplished through use of an array of input document sets and the careful scoring of transformations and/or outputs of trials could be performed with a software engineer with appropriate expertise in AI and/or machine learning, in conjunction with a CSS expert or a focus group containing two or more of them.

Once this process and the resulting rule set works, it would then be possible to simplify the CSS further by finding styling redundancy across pages and doing transformations that reduce an entire site to a minimal set of style sheets and style specifications within them.

Combining such tools with a simple crawler like wget could permit the copying of a desirable web site style as a starting point for another web site's styling, drastically cutting down the time to create a properly CSS styled result.


Linear regression: why is distance *squared* used as an error metric?

The squared form is sometimes called the Euclidean norm or L2 norm. One of its very helpful properties is that it has an easily defined derivative, which can be used in mathematical analysis and translated fairly easily into code.

Intuitively it is thought that it is advantageous to exaggerate the differences according to the value of the error, which squaring does. You might also use the powers 3 or 4, but the derivative is more complex.

A number of different norms may be used, according to the particular circumstances of the problem at hand.


Interesting examples of discrete stochastic games

Some of the domains in the International Probabilistic Planning Competition, such as the Wildlife Preserve benchmark, fit quite well the constraints you have given. Note that the problems are modeled with a high-level declarative language, RDDL. This means that you can define problems as big or as small as your heart desires with relative ease, since you can parametrize state description in terms of functions describing properties of an arbitrary number of objects.

There's also a quite useful project that allows to instance OpenAI gym environments from the declarative description of the environment, states and actions.


Can digital computers understand infinity?

I think this is a fairly common misconception about AI and computers, especially among laypeople. There are several things to unpack here.
Let's suppose that there's something special about infinity (or about continuous concepts) that makes them especially difficult for AI. For this to be true, it must both be the case that humans can understand these concepts while they remain alien to machines, and that there exist other concepts that are not like infinity that both humans and machines can understand. What I'm going to show in this answer is that wanting both of these things leads to a contradiction.
The root of this misunderstanding is the problem of what it means to understand. Understanding is a vague term in everyday life, and that vague nature contributes to this misconception.
If by understanding, we mean that a computer has the conscious experience of a concept, then we quickly become trapped in metaphysics. There is a long running, and essentially open debate about whether computers can "understand" anything in this sense, and even at times, about whether humans can! You might as well ask whether a computer can "understand" that 2+2=4. Therefore, if there's something special about understanding infinity, it cannot be related to "understanding" in the sense of subjective experience.
So, let's suppose that by "understand", we have some more specific definition in mind. Something that would make a concept like infinity more complicated for a computer to "understand" than a concept like arithmetic. Our more concrete definition for "understanding" must relate to some objectively measurable capacity or ability related to the concept (otherwise, we're back in the land of subjective experience). Let's consider what capacity or ability might we pick that would make infinity a special concept, understood by humans and not machines, unlike say, arithmetic.
We might say that a computer (or a person) understands a concept if it can provide a correct definition of that concept. However, if even one human understands infinity by this definition, then it should be easy for them to write down the definition. Once the definition is written down, a computer program can output it. Now the computer "understands" infinity too. This definition doesn't work for our purposes.
We might say that an entity understands a concept if it can apply the concept correctly. Again, if even the one person understands how to apply the concept of infinity correctly, then we only need to record the rules they are using to reason about the concept, and we can write a program that reproduces the behavior of this system of rules. Infinity is actually very well characterized as a concept, captured in ideas like Aleph Numbers. It is not impractical to encode these systems of rules in a computer, at least up to the level that any human understands them. Therefore, computers can "understand" infinity up to the same level of understanding as humans by this definition as well. So this definition doesn't work for our purposes.
We might say that an entity "understands" a concept if it can logically relate that concept to arbitrary new ideas. This is probably the strongest definition, but we would need to be pretty careful here: very few humans (proportionately) have a deep understanding of a concept like infinity. Even fewer can readily relate it to arbitrary new concepts. Further, algorithms like the General Problem Solver can, in principal, derive any logical consequences from a given body of facts, given enough time. Perhaps under this definition computers understand infinity better than most humans, and there is certainly no reason to suppose that our existing algorithms will not further improve this capability over time. This definition does not seem to meet our requirements either.
Finally, we might say that an entity "understands" a concept if it can generate examples of it. For example, I can generate examples of problems in arithmetic, and their solutions. Under this definition, I probably do not "understand" infinity, because I cannot actually point to or create any concrete thing in the real world that is definitely infinite. I cannot, for instance, actually write down an infinitely long list of numbers, merely formulas that express ways to create ever longer lists by investing ever more effort in writing them out. A computer ought to be at least as good as me at this. This definition also does not work.
This is not an exhaustive list of possible definitions of "understands", but we have covered "understands" as I understand it pretty well. Under every definition of understanding, there isn't anything special about infinity that separates it from other mathematical concepts.
So the upshot is that, either you decide a computer doesn't "understand" anything at all, or there's no particularly good reason to suppose that infinity is harder to understand than other logical concepts. If you disagree, you need to provide a concrete definition of "understanding" that does separate understanding of infinity from other concepts, and that doesn't depend on subjective experiences (unless you want to claim your particular metaphysical views are universally correct, but that's a hard argument to make).
Infinity has a sort of semi-mystical status among the lay public, but it's really just like any other mathematical system of rules: if we can write down the rules by which infinity operates, a computer can do them as well as a human can (or better).


How to design a fitness function for the 8-queens problem?

Here you can find an example of how to apply genetic algorithms to solve the 8-queens problem.
The proposed fitness function is based on the chessboard arrangement, and in particular, it is inversely proportional to the number of clashes amongst attacking positions of queens; thus, a high fitness value implies a low number of clashes.


What do you call a machine learning system that keeps on learning?

There are several terms or expressions related to such systems, such as online learning, incremental learning, continuous learning, continual learning, and lifelong learning. They are sometimes used interchangeably, but some of them have slightly different meanings. For example, online learning does not need to be incremental, which refers to algorithms that attempt not to forget previously learned information.

The opposite of online is offline. However, the expression batch learning is sometimes used as an antonym for online learning.


Is there a machine learning system that is able to understand mathematical problems given in a textual description?

There was a lot of work on this topic at UT Austin, which has now migrated to the Alan Institute.
There is no off-the-shelf software that will answer your question (if there was, DARPA would stop funding its development!), but you can read about the latest development in a number of recent papers.
This paper (Seo et al. EMNLP 2015) discusses the techniques that are used to interpret diagrams that accompany geometry problems, while this one (Hosseini et al. EMNLP 2014) talks about how to automatically parse verbs to interpret the meaning of a question. The 2015 TACL paper (Koncel-Kedziorski et al. 2015) completes this by discussing how to extract the relevant equations from a word problem. Once you have the equations, know what question is being asked, and can interpret any diagrams, you can do most high school math problems.
However, I don't think this is yet a fully reliable system. It is one part of a larger, long running effort to create a program that can achieve higher education certifications in many subjects. You can see many projects related to this at the Alan Institute's website.


Would a neuromorphic SNN of the same complexity as the human nervous system be 'smarter'?

The human nervous system is an extremely dynamic entity, having been formed by the processes of embryogenesis, mediated by various markers guiding neurons to grow to specific areas, and all of it laid down by millions of years of evolution. There are many different varieties of neurons in the human brain, indeed more so than pretty much any other animal on this planet.

It is hard to see that a spiking neural network (SNN) would be able to get all these details correct when we don't yet fully understand the biological analog.

I think for this to be successful, we need to understand much more about not only the variety of neurons in the brain, but details of the the embryogenic dynamics as well. And not just the neurons, but the glial cells as well.

Having said that, there is something to be said for using evolutionary approaches to resolve "a solution" that may work anyway. Taking this approach will require many more resources, but is perhaps doable.

All in all, I would not expect a naive attempt of the neuromorphic SNN to succeed. There is a lot of complexity involved in what the brain does, and it involves the glial cells to a large degree. Do we understand enough about the role of glial cells in the brain? We cannot ignore them. Are they only performing "housekeeping" operations, like, for example, the uptake of "spent" neurotransmitters? Or are they doing more, taking part in the computational and / or memory aspects of the brain?

There is much research in this and other pertinent areas that one should look into. And expect a lot of surprises.


What is the order of the genetic operations in NEAT?

You start by evaluating all of the initial neural networks and compute their initial fitness.
Then you speciate,
kill off the worst neural networks,
mutate and crossover to produce offspring, and
evaluate again.

The order of events is described on page 109 onwards in the original NEAT paper.

  Which neural networks are killed (or not) at each generation?

The neural networks with the worst performance are killed off after speciation. None of the neural networks survive - the entire population is replaced with the offspring of the nets remaining after the culling stage. That said, you can implement elitism, where you keep some small portion of the best-performing nets and carry them over to the next generation without mutating them, but that is optional.

  Who is being mutated and at what point?

At the end of each generation, after speciation and culling. To produce offspring, some of the remaining nets are subjected to mutation (think asexual reproduction - like single-celled organisms - but the offspring is mutated so that it differs from the parent). The rest are subjected to crossover in random pairs, so this would be the equivalent of sexual reproduction where you need two parents.

Hope that helps.


Could GA's determine fitness by "Fighting" against each other?

i'm the main developer of Neataptic, a Javascript neuro-evolution library.

Very effective! Realise that this is how real-life evolution happened as well: we kept on improving against other species, which forced them to improve as well.
Very practical, especially if you don't want to set up any 'rules' like you say, it makes the genomes find out what the rules are themselves.
Basically, you let each genome in the population play X games against other genomes, I advise you let each genome play against every other genome in the population. An example of scoring would be giving the genome 1 point for winning, and 0.25 or 0.5 for a tie. Each game should always have a result!
I'm not sure about this one, as I haven't implemented speciation.

I want to give you some examples that I have worked on:

Agar.io AI (neuro-evolved neural agents) - basically, I let neural networks evolve to get the highest score they can in agar.io, by competing against each other! It worked better than I expected.
Currently i'm working on new project, a kind of 'cops and robbers' style game.


Why does Deep Q Network outputs multiple Q values?

I think this was just a "clever" design choice. You can actually design a neural network (NN), to represent your Q function, which receives as input the state and an action and outputs the corresponding Q value. However, to obtain $\max_aQ(s', a)$ (which is a term of the update rule of the Q-learning algorithm) you would need a "forward pass" of this network for each possible action from $s'$. By having a NN that outputs the Q value for each possible action from a given $s'$, you will just need one forward pass of the NN to obtain $\max_aQ(s', a)$, that is, you pick the highest Q value among the outputs of your NN.

In the paper A Brief Survey of Deep Reinforcement Learning (by Kai Arulkumaran, Marc Peter Deisenroth, Miles Brundage and Anil Anthony Bharath), at page 7, section "Value functions" (and subsection "Function Approximation and the DQN"), it's written

  It was designed such that the final fully connected layer outputs $Q^\pi(s,\cdot)$ for all action values in a discrete set of actions — in this case, the various directions of the joystick and the fire button. This not only enables the best action, $\text{argmax}_a Q^\pi(s, a)$, to be chosen after a single forward pass of the network, but also allows the network to more easily encode action-independent knowledge in the lower, convolutional layers.


Difference between Neural Compute Stick 2 and Google Coral USB for edge computing

I do not know much about the Neural Compute Stick but I can tell you a little bit about the Coral Edge TPU, since I used it myself (which I think also applies to the Neural Compute Stick).
The Edgte TPU is a specialized ASIC which is very efficient at the main calculations (convolutions, relu etc.) for neural network inferencing. That means it cannot be used for training* a neural network but for deploying a neural network in production after it has been trained and optimized/quantized (precision reduced from float32 to int8). But I think you already new that, as it seems from your question.
Now to your actual question in terms of speed: You cannot really compare speed of such chip in terms of framerate alone, nor can you call it a visual processing unit or not. The google coral is a general ASIC that is very fast at doing convolutions/relus etc. For what you gonna use your neural network (like Image recognition, or maybe stock predicitons) is complitly up to you, and especially up to your neural network for which task it was trained for. The only limitation you have here is regarding the supported layer operations. E.g. it is not possible to do operations like 3D convolutions or some fancy new non-linear activation functions (There was an overview of supported operations in the docs that I cannot find right now).
Furthermore the framerate also depends on your NN architecture, image resolution etc. so a comparison here is completly misleading. If you want to have a general indication of speed look at how many int8 operations it can handle per second (TOPS) also under what energy consuption (Watts), if you care.
The main advantage of this unit compared to a GPU that is usually used for training+inferencing is much lower energy consumption and unit cost. With roughly 4 Watts on the Edge TPU (as far as I can remember) compared to e.g. 250 Watts GPU. Energy consumption also dictates the necessary cooling solution, which can just be a passive for the Edge Tpu in many cases. Regarding the unit costs, I guess you can easily sees that yourself, if you keep in mind that you can get "nearly" similar inferencing speeds than with a full workstation class GPU. Furthermore, such unit has a much smaller formfactor and weight, which makes it perfect for applications at the edge. (I should also add that the efficiency gain is also a lot due to the precision reduction to int8 (quantization). Quantization is also possible on GPUs.)

And lastly, do i need to keep my neural network in a single computer board for training or can i have it at a cloud?

I am not quite sure what you mean with this. The general workflow would look like this: you use a GPU for training your neural network, after training you optimize/quantize your network, and lastly you deploy your neural network to the "computer board" of your choice that just needs to have a CPU and the Edge TPU and do inferencing operations/predictions for your task.
*Transfer learning is possible of e.g. the last layer. But no backpropagation for full NN training.


Transformers: how to get the output (keys and values) of the encoder?

I have read the OpenNMT source code (https://github.com/OpenNMT/OpenNMT-py/blob/cd29c1dbfb35f4a2701ff52a1bf4e5bdcf02802e/onmt/modules/multi_headed_attn.py).
It seems like an extra linear layer learns the weights $W^{key}$ and $W^{value}$ (plus biases), so to get the output (keys and values), you multiply the output of the encoder's final add + norm layer by $W^{key}$ to get the keys, and by $W^{value}$ to get the values.
Additionally, these weights and biases seem to be independent across each of the decoding layer. So you feed the same encoder output (add + norm layer output), but multiply by different $W^{key}$ and $W^{value}$ matrices and add by different biases for each of the decoding layer, resulting in different keys and values for each layer


In a neural network, can colors be used for neurons in place of floating points and would there be any benefit in doing so?

To answer your question, I'll just simplify it and assume you are representing the activation of a neuron (the value it produces) by an RGB value. So a tuple with 3 values ranging from 0 to 1.

It's important to remember that every single machine learning model could be computed by a human on pen and paper using just numbers. So keeping that in mind, this neural network would essentially just be a combination of multiplying tuples of size 3 in a certain order (ignoring activation functions).

I will also assume that each RGB value has it's own unique set of incoming weights, like following:

this essentially amounts to just have 3 unique neurons. There's no difference:

All I did there was get rid of the circle. The calculations are no different. If you changed the calculation, say values are shared between the 3 RGB nodes, then you now just have 3 duplicate nodes. In all cases, you can just represent the equivalent calculation as a standard neural network.

However, in the same vein, in a convolutional neural network, if you create conv layers with an output of 3 channels, you can visualise their outputs as a colour image (assuming you are using an activation function that outputs numbers between 0 and 1), which can be cool to look at.


Para Generation and Drawing Conclusion from X give Articles

Paraphrase generation would be made by an abstractive text summarization tool like Tensorflow as described in Google's blog.

Abstractive tool summarizes the text but also it adds extra words to make text more human like.

About the second problem scenario I would say the answer depends quite much what you mean by the conclusions and how you think the conclusions would be made. I strongly believe this falls in the category of artificial general intelligence, which is emerging area of AI, supported by association AGI, where the focus is on strong artificial intelligence and more human like features of AI.

Judging about conclusions at least needs a vast knowledge base of the semantics of different words and pieces of text in general combined to possible conclusions and their legitimate implications.


Although it is common to represent a grid as two dimensional array in a computer program, this is not the only way to represent one. You could, for example, use a generalized graph structure made of linked nodes, with 4 links each. Many other representations are possible. Even if you use a 2d array, some languages would index the columns first rather than the rows (FORTRAN for instance).

That issue is really an aside though, because it conflates representation with reasoning. By choosing to represent the world in certain data structures, we can made it easier or harder for different AI algorithms to reason about them, but the reasoning can often proceed even if the representation is inefficient. That is, an AI algorithm can ask questions about the columns in the grid, even if the columns of the grid are not represented in a way that makes it programmitcally easy to group them together.


What are the likely AI advancements in the next 10 years?

The determination of likelihood of advancements in any science or technology over a decade are dependent upon several technical features of culture. Not all are technical because they are elements in hardware and software design. Some are.

Market forces
Perpetuation of traditions in education
Feasibility
Availability of resources
Cost of research
Cost of deployment and sustenance
Natural cultural balance of curiosity and fear

The low level cybernetic conversations with service applications like Siri and Cortana were correctly identified in the question, and Google Assistant should be included. If a survey is done of these conversations by working around public relations, perhaps by using a competitor search engine to locate PR unfriendly records of conversations, the low level becomes immediately apparent1. Anyone could confirm this by asking these service applications a set of the popular questions in a set of the StackExchange sites. Most if not all of them would produce answers that would be down voted if not deleted.2

These service applications are efforts to meet the criteria of Alan Turing's thought experiment, which Turing called The Immitation Game3.

  How high levels is it going to reach (likely) in future years?

Prediction of AI milestones is not possible without defining first the milestone criteria to be met. Let's pick seven coveted ones.

A service application the answers of which can gain StackExchange reputation faster than humans, which is a more demanding version of Alan Turing's thought experiment
A robot that can mow our lawns safely
A car without steering wheel or pedals that can take us to places we typically go as safely as a median human driver can
A-pets
Automated theorem proving of currently problematic proofs, and not just verification
Practical goal driven underwater robotic navigation
A home tutor that produces a better distribution of college board exam results in students than public or private educational institutions, better meaning higher mean and lower standard deviations

Based on the seven factors already listed above plus one more, the instantaneous state of AI research and its publication during each moment of the search for probabilities regarding the above seven milestone, some probabilistic predictions can be made about 2029. The bullet items primarily driving the numbers are at the top of this answer4, 5, 6.

< 1% — bullet item 2 too low, and 7 too skewed toward fear
40% — bullet item 1 too low
50% — bullet items 1, 2, 3, and 7 all favorable
2% — bullet item 2 too low, and 7 too skewed toward fear
< 1% — bullet items 2 too high, 3 too low, and 7 too skewed toward fear
80% — bullet items 1 and 3
30% — bullet item 3 high, but 1, 2, 3, 5, and 7 are unfavorable

There are a few that some will find unpalatable. Service applications and automatic theorem provers are not at all likely to become smart in the way defined in the milestones within the next ten years. Robots will likely be swimming and assisting with mining, drilling, and troubleshooting. Level 5 automated vehicles may already be driving around town.

Footnotes

[1] All intelligent systems, including biological ones, have weaknesses and strengths, and the strengths are not perfectly reliable. For instance AI on a CAD application may select a screw fastener to attach one part to another in a sub-assembly with 100% accuracy 99.99999% of the time, seven 9s of accuracy. Much of the talk of general, strong, and weak AI fails the burden of mathematical rigor or scientific verifiability and is largely fabrication and self-deluding conjecture to get cushy, high profile jobs. When published or aired, it is done so not because of credibility but to get paid for ad impressions.

[2] Please don't run this experiment fully, since it would burden moderators that are volunteering to keep the sites clean. Ask the questions but do not post the answers you find. The superficiality will be obvious enough without a focus group.

[3] See the section The Turing Challenge to Cartesian Thought in the post Can a brain be intelligent without a body? for more information on Turing's work in this space and its limitations.

[4] The parenthetic (likely) in the question in the body of the question is correctly placed. The question's tag predicting-ai-milestones is also correct. The likelihoods listed will change because, as the search for likelihoods of AI milestones progresses, the eight factors listed, which are involved in the predictions, will change. This answer author may or may not change the predictions accordingly.

[5] The reasons behind these probabilities cannot be fully disclosed for contractual reasons, but what is disclosed is explained where it can be. Furthermore, the results of prediction are based on the information available to this particular answer author. Others may have other numerical numbers to offer based on other quantitative or qualitative information and can disclose what they wish and are legally allowed to do in another answer.

[6] There is no guarantee made or implied in this answer.


Which reward function works for recommendation systems using knowledge graphs?

I found the answer further into the paper! I'll post it here for everyone.

Given any user, there is no pre-known targeted item in the KGRE-Rec (Knowledge Graph Reasoning for Explainable Recommendation) problem, so it is unfeasible to consider binary rewards indicating whether the agent has reached a target or not.

Instead, the agent is encouraged to explore as many “good” paths
as possible. In the context of recommendations, a “good” path is one that leads to an item that a user will interact with, with high probability. To this end, a soft reward is given only for the terminal state $s_T = (u,e_T,h_T)$ based on a scoring function $f(u,i)$. The terminal reward $R_T$ is given as -

where the value of $R_T$ is normalized to the range $[0,1]$. Details about $f(u,i)$, the scoring function, can be found in the paper.

References:

Reinforcement Knowledge Graph Reasoning for Explainable
Recommendation [Yikun Xian, Zuohui Fu, S. Muthukrishnan, Gerard
de Melo, Yongfeng Zhang]


Use Machine Learning/Artificial Intelligence to predict next number (n+1) in a given sequence of random increasing integers

I think your code works fine for what is meant to be doing - fitting a linear regression model. The problem here is that you are using a linear model. Linear model does not have an adequate approximation capacity, it will only be able to fit data that is described by a linear function. Here, you gave a random sequence of numbers, that is very difficult for linear model to approximate. I would advise you to try 2 things:

1) Try something simpler first. Instead of doing a random sequence of numbers do a linear sequence of numbers for example a function like $y = 2x$ or maybe affine function like $y = 2x + 5$. So you would have a sequence like:

$2, 4, 6, 8 ...$ or $7, 9, 11, 13, ...$

If you manage to get that working try a nonlinear function like $x^2$ for example.

2) Instead of using a linear model try a nonlinear model. For example a polynomial regression model. Especially powerful function approximators are neural networks. In theory, a neural network with single hidden layer can approximate an arbitrary continuous function under some conditions
(Universal approximation theorem) , so you could try to see how would a neural network solve the problem, there are several open source neural network libraries that you could try.


Is the Mask Needed for Masked Self-Attention During Inference with GPT-2

Answer to Q1) If sampling for next token do you need to apply mask during inference?

Yes you do! The models ability to transfer information across positions was trained in this manner, and changing it up will have unpredictable consequences. Let my try to give an example:

Tokens: 1:sally, 2:sold, 3:seashells, 4:on, 5:the, 6:____
In the above you are trying to predict 6 from {1:5}

Denote $n^{(m)}$ as the set of tokens the $n^{th}$ positional embedding has info from at the $m^{th}$ layer.

In both cases we see that $n^{(0)} = \{n\} \ \  \forall n$. Now though with a mask we get $n^{(i)} = \{k\}_{k\leq n} \ \  \forall n \ \ s.t. \ \ i \geq 1$ but without we see $n^{(i)} = \{k\}_{k \in [1:N]} \ \  \forall n$. This difference means at the final layer the mebeddings going in will differ completely, and unless we train for such an approach it will cause  error

Answer to Q2) What is the sample dimension?

It took me a couple reads to understand what youre asking for but I think I understand. The sample at each step is drawn from a distribution where its logits are linearly associated to a single embedding of dimension $d_{(model)}$ therefore that is our upper bound: $dim(sample) \leq d_{(model)}$ which in the example you gave is 768.


John R. Pierce led the Bell Labs research team that created the first transistor and gave it its name.  He was later the Chief Engineer at the Jet Propulsion Laboratory at CalTech.

His relationship to artificial intelligence research was mainly in regard to language translation.  He wrote the following.1

  The computer has opened up to linguists a host of challenges, partial insights, and potentialities. ... Certainly, language is second to no phenomenon in importance. ... The new linguistics presents an attractive as well as an extremely important challenge. ... The most reasonable government source of support for research in computational linguistics is the National Science Foundation. ... We estimate that work on a reasonably large scale ... would be justified at ... an annual expenditure of \$2.5 to \$3 million.

No official record of Pierce stating unequivocally that all artificial intelligence funding is stupid can be found.  Nils J. Nilsson's The Quest for Artificial Intelligence quotes Donald Knuth stating this.

  John R. Pierce, whom I have already mentioned in connection with the ALPAC report on machine translation (in section 7.2) and his negative comments about speech understanding (p 222) wrote me a very short letter
  in which he stated,

    Concerning artificial intelligence, I believe I invented the slogan, "Artificial intelligence is real stupidity."

    ...

    I resent artificial intelligence because I feel that it is unfair to computers. But then, artificial intelligence people did devise LISP, which is pretty good.

  The letter did not elaborate on the slogan or why AI is "unfair to computers."

The paragraphs that may have drawn the most negative attention (and possibly fabricated quips) are these.2

  It would be too simple to say that work in speech recognition is carried out simply because one can get money for it.  That is a necessary but not a sufficient condition.  We are safe in asserting that speech recognition is attractive to money.  The attraction is perhaps similar to the attraction of schemes for turning water into gasoline, extracting gold from the sea, curing cancer, or going to the moon.  One doesn't attract thoughtlessly given dollars by means of schemes for cutting the cost of soap by 10%.  To sell suckers, one uses deceit and offers glamor.

  It is clear that glamor and any deceit in the field of speech recognition blind the takers of funds as much as they blind the givers of funds.  Thus, we may pity workers whom we cannot respect.  People who work in the field are full of innocent (in their own view) enthusiasm.

In hindsight, we can see a few things Pierce called correctly.

Technology projections that are not based on realistic assessments of progress that has already been made and verified in the correspondence between theory, experimental results, and penetration into a market that exercises the technology in the real world can be unreliable in terms of both feasibility and cost in time and money.
Natural language translation and comprehension are important areas of research but research results from short term investments will not be substantial.  The problem complexity is high.

Whether Pierce meant that artificial intelligence is real stupidity literally or whether he was referring to the unrealistic expectations planted by researchers at MIT to obtain defense funding after the launch of Sputnik, we may never know for sure, but is easy to guess.

The comment about LISP is an obvious reference to Marvin Minsky's lab, which allegedly did not produce for the U.S. government what was promised.  It is merely an allegation because research failures can be fabrication to assist in classifying publicly funded research.

I say this as one who spent time with those in the closely linked AI lab at the United Technologies Research Center.  Everyone had national security clearances, and we all saw fabricated successes and fabricated failures.  There is good reason for mixing in misinformation when espionage is a known phenomenon.

Based on the way Pierce commonly wrote, the unfairness to computers was likely a satirical comment about research funding pitches and the unrealistic expectation they placed on the speed of computer technology development.  He is certainly not a person that would anthropomorphize a computer witlessly.

Pierce was given the leading role at the JPL because he gained credibility among fiscal stakeholders by telling the truth while at Bell Labs.  Some of the futurists in leading technology roles today might benefit from that history.

References

[1] Language and Machines, Pierce, 1966, ALPAC report to the U.S. National Science Foundation

[2] Whither Speech Recognition?, Pierce, 1969, Journal of the Acoustical Society of America, Vol. 46, No. 4


What does "Wide" vs. "Deep" mean in the context of Neural Networks?

The width refers to the number of neurons in a layer. The depth refers to the number of layers.

Have a look at the following question regarding the impact of these hyper-parameters on the performance of the neural network: https://stats.stackexchange.com/q/214360/82135.


Is the Turing Test a blinded experiment?

First, I think it is important to mention that the Turing Test as is currently accepted is an updated version of Alan Turing's proposed imitation game, so your question is twofold. I don't think Turing makes this distinction that you propose. To Turing the question was "Can machines think?" and he explores this by way of the imitation game.

  The new form of the problem can be described in terms of a game which we call the
  'imitation game." It is played with three people, a man (A), a woman (B), and an
  interrogator (C) who may be of either sex. The interrogator stays in a room apart front the
  other two. The object of the game for the interrogator is to determine which of the other
  two is the man and which is the woman. He knows them by labels X and Y, and at the
  end of the game he says either "X is A and Y is B" or "X is B and Y is A." The
  interrogator is allowed to put questions to A and B

  It is A's object in the game to try and
  cause C to make the wrong identification

  The object of the game for the third player (B) is to help the interrogator.

This is Turing's outline of the imitation game (shortened) between a man and a woman. The only difference between this game and the one we care about is that we replace party A with a machine and the only requirement on B is that they are human.

So yes, in Turing's imitation game the machine is privy to the information that it is not actually human. Whether the machine uses this information is up to the machine.

To talk about your statement on pretending to be a duck versus believing you are a duck lets examine the base imitation game with ducks and frogs. Let's say player A is a frog (either pretending to be a duck or truly believing that it is a duck)  and player B is a duck. Suppose for this example, the frogs and ducks are able to speak English and follow the rules of the game. The interrogator may ask "what is the color of your beak?" The duck will respond yellow, assuming its beak is yellow (I am no bird expert), and then our smart frog pretending to be a duck will look up the color of the beaks of ducks and respond yellow as well.

However, our frog that believes he is a duck will think "I am a duck so to discern the answer I must only look at my beak." And thus, our frog will say "I do not have a beak, but my mouth is green." This will oust the frog.


Training Neural Network with 'fake' data?

Yes, this is not advisable. If you train your model with random data your model is not learning anything useful, because there is no information to gain from those examples. Even worse it may (and likely is) trying to generalize off of your incorrect examples, which will lessen the effect your real examples have. Essentially, you are just dampening your training set with noise.

You are moving in the right direction though. 75 examples will not be enough if your problem has any complexity at all. And unless, you know some correlation between the inputs a,b and the output c you don't want to generate data (and even if you did know some correlation, it is not always suggested to generate data). If it is impossible to get any more data you might want to consider a statistical model, rather than a neural network.


These are really two questions -
1. How to assign lines in a dialogue to particular speakers
2. How to analyse the content of a speaker's utterances

I will attempt to answer the first one.

In discourse analysis (the field that studies conversations), there is the concept of adjacency pairs, linked utterances which typically follow on to each other. A well-formed conversation can be segmented into a series of adjacency pairs, such as greeting/greeting, question/answer/feedback (that's more of an 'adjacency triple'), statement/comment, farewell/farewell. Sometimes it's a bit more tricky, such as question/counter-question/answer/answer:

young person: What beers do you have? (Q)
bartender: How old are you? (Counter-Q)
young person: 21 (A)
bartender: We have a lager or a nice IPA. (A)

I don't know if this has been tried before, but it should be comparatively easy to identify the types of speech acts in a conversation and join them up to form relevant adjacency pairs. This would be a lot more reliable than looking at vocabulary, especially since conversations mostly use higher-frequency words which are part of everyone's active vocabulary.

You would need to take some conversations, classify the individual lines as speech acts (building up an inventory as you do that), and then identify adjacency pairs. This will give you the structure of the conversation, and should also allow you to assign speakers to utterances (assuming a dialogue, ie only two participants). Note that it is not always purely form that determines the speech act: How are you? is a question on the surface, but its function is part of a greeting sequence.

This conversational structure could also assist you in looking at the second part, the content. If you know someone asks questions, and the other participant answers them, then you know exactly where to look for salient utterances that might give you clues as to the topic of the conversation. At least it should enable you to ignore the 'housekeeping' utterances like greetings and farewells etc.

Your second question is hard to answer without knowing more about the purpose of your project. I would rephrase this and ask it as a separate question. People would need to know what kind of conversations they are, and what exactly you want to get out of it.

Oh, and I wouldn't compare conversations with poetry. Completely different things!  :)


I think that the tasks you are referring to are the Simultaneous localization and mapping (SLAM) and, in particular, the Structure from Motion (SfM).

These methods are usually based on geometrical constraints and do not employ neural networks, but there exist some recent methods that make use of CNNs (such as this one).

Structure from Motion algorithms are a fundamental component of the Google ARCore (former Project Tango) and the Apple ARKit.
Unfortunately, these kits usually provide an interface to put elements in a specific position, but they don't provide the access to the 3d reconstruction of the scene.

If you'd like to play with SfM algorithms, I suggest you to start from this repository, but you can find a lot of other valuable tutorials online.


How do you calculate the heuristic value in this specific case?

The most obvious heuristic would indeed simply be the straight-line distance. In most cases, where you have, for example, x and y coordinates for all the nodes in your graph, that would be extremely easy to compute. The straight-line distance also fits the requirements of an admissible heuristic, in that it will never overestimate the distance. The travel-distance between two points can never be shorter than the straight-line distance (unless you start involving things like... teleportation).

From an image like that, the straight-line distance might be difficult to figure out yourself, which is probably why they gave you the straight-line distances on the right-hand side of the image. If the image is perfectly consistent, I suppose you could theoretically figure out by inspecting some of the roads in detail how much distance is covered per pixel. Then, you can also figure out how many pixels the figure has along the straight-line paths you're interested in, and compute the straight-line distances yourself. I have no idea if the figure was actually drawn in a 100% consistent manner though.


CNN Pooling layers unhelpful when location important?

Pooling doesn't completely remove information about location of features within your image. If you don't want to use pooling but want to reduce the size of your neural net layers you should try a value of stride greater than 1 in your convolutional layers. e.g:

keras.layers.Conv2D(filters, kernel_size, strides=(1, 1), padding='valid', data_format=None, dilation_rate=(1, 1), activation=None, use_bias=True, kernel_initializer='glorot_uniform', bias_initializer='zeros', kernel_regularizer=None, bias_regularizer=None, activity_regularizer=None, kernel_constraint=None, bias_constraint=None)