Does summing up word vectors destroy their meaning?

Summing up a sequence of word vector maybe used in practice sometimes. However, the operation of addition is non-reversible, meaning that once you sum up a few numbers, you cannot get the original numbers back. However summing up a sequence of word vectors may work depending on your task. You should also normalize the values, or just use average value.

For details: https://towardsdatascience.com/document-embedding-techniques-fed3e7a6a25d#ecd3

To feed data with different lengths, you can also try padding and trimming it. Set a constant L for length of paragraph and trim/pad all list of word vectors to this length. Padding adds 0 vectors to the begining of the list and trimming trims the first part of a text until it is equal to L. Even in LSTM networks padding and trimming is still used as even though you can feed as long of a text to a LSTM as you want, you still have to process the word vectors in batches, which requires them to be the same length.

Example code in python on padding/trimming vector:

def pad_trim(list_vec, L):
    //list vec: {vec1,vec2,vec3......}
    //Assuming vecN have a size of 200
    return list_vec[-L:0] if len(list_vec) > L else [[0] * 200] * (L-len(list_vec)) + list_vec

However in inference, you can ignore maximum length if you used a RNN based method, although as the network has not been trained on lengths more than L, it may perform better or worse.

Generally speaking, you should go for concatenating if possible, so you can keep all information in the sentence. However both may work just fine depending on your task.

For RNN based and CNN based model example, you should check this out:
https://medium.com/jatana/report-on-text-classification-using-cnn-rnn-han-f0e887214d5f


Can a technological singularity only occur with superintelligence?

That would be a no for speed or strength, if you have a super strong entity but it cannot research new materials, it will be quickly limited, same thing for speed, Basically, you need something out of their field to improve them, which makes a runaway improvement impossible.

Though, we already have super strong and super fast machines, those are cars, trucks, hydraulic presses, industrial exoskeletons etc... But, even though we can build better ones through the use of the old ones, we still need to research stuff that can't be improved by old ones.

What we need for a singularity is a field where an improvement in it makes further improvement easier. And I don't know a field where this doesn't involve intelligence.
If there is one, that may be possible to have a non intelligence driven singularity there.


Can a convolutional network predict states for a RL Agent

It sounds like what you're suggesting is similar to what is done in methods that use a planner. These methods looks to learn the dynamics of the MDP to use to plan during training; that is they want to be able to learn the transition probabilities $p(s'| s, a)$.
In this paper that I read recently they note that learning to predict environment dynamics when the state/action space is high dimensional, as is the case with images, is difficult; so whilst it may be possible in theory it would be difficult to do and if you were predicting many steps into the future then the error would compound.
A way around this, as is done in the referenced paper, is to use predict environment dynamics in a latent space. This means that they use a latent variable to predict the next state using e.g. Variational Autoencoders.


What do prediction/mean and label/mean represent in this TensorFlow code?

All of these could be problem specific (except maybe accuracy). Most of it is documented here:

accuracy: Percentage of correct number of classifications
accuracy_baseline: Accuracy baseline based on labels mean. This is the best the model could do by always predicting one class. (source)
AUC or Area Under the (ROC) Curve is quite complicated, but tells you something about the true/false positive rate. In short: the AUC is equal to the probability that a classifier will rank a randomly chosen positive instance higher than a randomly chosen negative one.
auc_precision_recall: Is the percentage of relevant intstances, among the retrieved instances, that have been retrieved over the total amount of relevant instances.
average_loss: You're usually minimizing some function, and this is likely the average value of that function given the current batches.
loss: The current value of the loss (as above). Either the sum of the losses, or the loss of the last batch.
global_step: Number of iterations.
label/mean and prediction/mean: Not really sure, but I suspect that if you have two classes then the label/mean is the mean of the value labels, whilst prediction/mean could be the value of the corresponding predictions. (two classes could give you a value between 0 and 1)


How does NEAT find the most successful generation without gradients?

NEAT is a genetic algorithm (GA). A genetic algorithm maintains a population of individuals (or chromosomes) and evolves it using operations like the crossover or the mutation, so that the fittest individuals keep living and most other individuals die. The nature of the individuals depends on the problem. For example, in the case of NEAT, the individuals are neural networks. However, these individuals need first to be encoded into a (compressed) reprentation (for example, a vector) that allows the operations like mutation to be efficiently applied: this representation is often called the genotype (or chromosome).

How do you decide which individuals are the fittest? A function called the fitness function needs first to be defined. The fitness function measures the fitness (or quality) of the individuals (or solutions). In the case of neural networks, a fitness function could, for example, be the accuracy of the neural networks on a validation dataset.

Why don't GAs need gradients? The fitness (or quality) of the solutions is given by the fitness function in the case of GAs, so gradients are not strictly needed, even though they can also be used.

NEAT is a little bit more complex (and it is described in detail in the paper that introduced it Evolving Neural Networks through Augmenting Topologies), but this is the basic idea behind all genetic algorithms (including NEAT).


Is it possible to find same persons based on text analyzing?

The term you are looking for is stylometry, which is related to a technique in forensic linguistics called writeprint analysis. There are many different techniques to perform stylometric analysis, from the very basic 5-feature analysis classifying features such as the lexicon and idiosyncrasies unique to a person to more complex analysis utilizing neural networks and machine learning. Searching online for research papers focusing on stylometry should assist you in finding the best technique for the job.


How can I be sure that the final model, trained on all data, is correct?

The purpose of the test set is to test your model before deploying, otherwise, you would not need the test set in the first place. If you retrain your model by also including the validation and test datasets, of course, you cannot test your model anymore. You need to leave the test dataset separate and not use it for retraining, unless you have more data for testing.


What are the challenges faced by using NLP to convert mathematical texts into formal logic?

I can see several challenges, and the list below is not exhaustive:

i. The main problem is how to model a problem of translating a language test into a formal language. It will probably be something like the automatic translators, but with some guarantees that the proof semantics will be preserved. If you are more interested in this path, I recommend researching what PAC, Information Theory, Computational Proof theory, Complexity theory can contribute to this modeling.

ii. Another problem is how to get the data reliable. You commented that as people used it they would generate this data. But the problem is not just collecting the data. How much you will trust the data and how you will measure the model's performance in translation.

iii. Another problem is more humane, how do you get mathematicians to use such a system? And how to make the model self-explainable.

I believe that this is one of the most difficult problems in machine learning. I once saw this video a while ago and I don't know if it can help anything. I also recommend the stack exchange of theoretical computer science, there you will probably have a more complete answer.


How do I create chatbots without machine learning?

The easiest non-ML way would be to use a finite state machine. You could model various states of your conversation topics, and certain utterances of your bots could advance the bot's internal model along different paths. The complexity depends on the complexity of the topic.

You can then enhance the transitions with probabilities, and later move on towards ML by transforming it into an HMM.

However, even simple topics will probably lead to fairly complex state machines. But you should be able to keep track of what is going on in your conversation nevertheless.

Update: just to make it a bit clearer, I was thinking along the lines of having states for particular stages in the conversation. You could either have one model for the whole conversation, or one per participant.

Initially, there would be a state 'greeting'. Possible transitions would be to a further state 'greeting' (the response of the person who has been greeted), or that could be skipped to states such as 'statement', 'question', etc. 'Question' would have transitions to 'answer', 'ignore question', 'counter/clarification question' etc. The level of detail depends on your application.


Tabular Q-Learning does not explicitly create a model of the transition function. It does not generate any output that you can afterwards use as a function to predict what the next state s' will be given a current state s and an action a (that's what a transition function would allow you to do). So no, Q-learning is still model-free.

By the way, model-based RL does not necessarily have to involve creating a model of the transition function. Model-based RL can also mean that you assume that such a function is already given. It just means that you use such a function in some way.


Why the number of training points to densely cover the space grows exponentially with the dimension?

First, let's try to build some intuition for what we mean when we say that we want to "densely cover" a $d$-dimensional space $\mathbb{R}^d$ of real numbers. For simplicity, let's assume that all values in all dimensions are restricted to lie in $[0, 1]$. Even with just a single dimension $d=1$, there are actually already infinitely many different possible values even in such a restricted $[0, 1]$ range.
But generally we don't actually care about literally covering every single possible value. Generally, we expect that points in this $d$-dimensional space that are "close" to each other also "behave" similarly, that there's some level of "continuity". Hence, to get "sufficient" or "good" or "dense" coverage of the space, you can somewhat informally assume that every data point you have occupies some space around it. This is the intuition behind Lutz Lehmann's comment under your question: you can think of every point as being a $d$-dimensional cube occupying some volume of your $d$-dimensional space.
Now, if you have a $d$-dimensional space of size $[0, 1]$ along every dimension, and you have little cubes that occupy a part of that space (for instance, cubes of size $0.1$ in every dimension), you will indeed find that the number of cubes you need to fill up your space scales exponentially in $d$. The basic idea is: if some number of cubes $K$ is sufficient to fill up the $d$-dimensional space, and if you then increase the dimensionality to $d+1$, you'll need $dK$ cubes to fill the new space. When you add a new dimension, the complete previous space becomes essentially just one "slice" of the new space.
For dimensions $d = 1, 2, 3$, this is fairly easy to visualise. If you have $d=1$, your space is really just a line, or a line segment if you constrain the values to lie in $[0, 1]$. If you have a $[0, 1]$ line segment, and you have little cubes of length $0.1$, you'll need just ten of them to fill up the line.
Now imagine that you add the second dimension. Suddenly your line becomes an entire plane, or a $10\times10$ square grid. The $10$ cubes are now only sufficient to fill up a single row, and you'll have to repeat this $10$ times over to fill up the entire $2$D space; you need $10^2 = 100$ cubes.
Now imagine that you add the third dimension. What used to be a plane gets "pulled out" into an entire three-dimensional cube -- a large cube, which will require many little cubes to fill! The plane that we had previously is again just a flat slice in this larger $3$D space, and the entire strategy for filling up a plane will have to be repeated $10$ times over to fill up $10$ such slices of the $3$D space; this now requires $10^3 = 1000$ cubes.
Past $3$ dimensions, the story continues in exactly the same way, but is a bit harder for us humans to visualise.


What is the best loss function for convolution neural network and autoencoder?

There is no right answer to this. Finding the right loss function is a tough and difficult problem. So your goal as the architect is to try to find one that best suits your needs. So lets think about your needs.

You mention that you dont want lighting shifts to cause large error, so ill take a leap and assume you care more about the shapes and style of the image more than the coloring. To deal with this, maybe consider using difference of the gram matrices (this is considered common place in style transfer literature: A Neural Algorithm of Artistic Style) Note that you could use the encoder to get the representation of the output as well for the loss, $L(x) = D(Gram(Enc(x)), Gram(Enc(\hat x))$ where $D$ would be some distance metric like euclidian distance.

Maybe the outline is all you care about. You could use some known edge detector filter and compare those, ex: $D(Edge(x), Edge(\hat x))$

Maybe you just dont care about color shifts, you could do $D(x - \mu_x, \hat x - \mu_{\hat x}$).

Note that you can play around with whatever distance metric you use, whether it be with MSE, RMSE, MAE, etc... Each has their own small pros/cons based on the loss manifolds they create. In your case i dont think the difference there will be night and day though, but you never know.

Also mixing and matching is always nice: ex: $L(x) = \lambda_1 D(x, \hat x) + \lambda_2D(Gram(Enc(x)), Gram(Enc(\hat x)) + ...$

Takeaway: MSE might actually be fine, but it really depends on what you prioritize, and once you figure that out you can start getting clever and design the loss that fits your needs and problem


Can the inputs and outputs of a neural network be a neural network?

A neural network essentially is a function:

$$\mathbf{y} = f(\mathbf{x}, \mathbf{\theta})$$

Where $\mathbf{x}$ is a vector input, $\mathbf{\theta}$ are changeable or learnable parameters, and $\mathbf{y}$ is a vector output.

There are some variations of this in practice, as you can make special arrangements of $\mathbf{x}$ and $\mathbf{y}$, or use an internal state and feedback loops to allow either or both $\mathbf{x}$, $\mathbf{y}$ to be sequences. However, the above function is basically what a neural network is; how it works beyond that summary are details that you can study.

In supervised learning, you are interested in fixed sizes/shapes of $\mathbf{x}, \mathbf{y}, \mathbf{\theta}$ and trying to find a value of $\mathbf{\theta}$ such that

$$f(\mathbf{x}, \mathbf{\theta}) \approx g(\mathbf{x})$$

where $g(\mathbf{x})$ is some "true" function that you care about, and can find or generate examples of, but typically don't fully know. The value $\mathbf{\theta}$ is called the parameters of the neural network. In addition to these parameters, there are also hyper-parameters of the neural network, which include how many neurons there are in each layer, the valid connections between them, which non-linear function is applied after summing connections between them, etc.

There are learning algorithms used to find $\mathbf{\theta}$ - the many variations of gradient descent being the most popular. Some algorithms - mainly evolutionary approaches - can also vary hyper-parameters, although a more common approach is to repeatedly find $\mathbf{\theta}$ using gradient descent, and vary hyper-parameters in different learning trials, using some metric of performance using test data to pick the best one.

  Can the inputs and outputs of a neural network (NN) be a neural network (that is, neurons and connections), so that "if some NN exist, then edit any NN".

Yes - partially. This is a data representation issue. To use it as an input, you would need to express the state of a neural network as a vector - or sequence of vectors - for the input, and the output/edit would also need to be a vector. Probably the simplest way to do this would be to use one network directly output a fixed length vector for $\mathbf{\theta}$ of the target network given an existing value of $\mathbf{\theta}$. That would not allow you to change connections or layer sizes etc, but it would be a very straightforward way to express "one neural network altering another" (ignoring whether this was in any way useful for a task).

If the output was a full representation of the new network, then you would have a function that took as input the definition of one neural network, and output the definition of another network. It would be up to you to convert to/from implemented neural networks and the representations for input $\mathbf{x}$ and $\mathbf{y}$.

If the output $\mathbf{y}$ was an "edit" for a change $\mathbf{x} \rightarrow \mathbf{x}'$, then you would have to decide what edits are allowed, design the representation and write code that applied the edit (the NN would not actually make any changes to another NN by itself). There is no standard way to do this, although there are things you could base this on (such as NEAT).

The big unanswered question with both of the approaches though is what your "true function" $g(\mathbf{x})$ is supposed to be. Having a neural network that represents a neural network generator or edit function is only half of the problem. You also need a way to either generate "correct" outputs to learn from, or a way to assess outputs against a goal.

The goal cannot simply be "make a valid edit", as the number of valid edits that will do nothing useful vastly outnumbers the number of edits that have some specific purpose. This is a similar issue to the fact that there are roughly $2^{8000000}$ valid 1MB files, but only a small proportion of those will be valid image files, and a smaller proportion still will be valid images that represent a natural image that could be taken by a camera. Neural networks that generate natural images therefore must be trained using natural images as a reference, otherwise they will tend to produce meaningless static-like noise.

  I think that by creating NNs with various inputs and outputs, interacting with each other, and optimizing them with evolution, we can create strong intelligence

This is very broadly compatible with an Articial Life approach to AI. Although there are two differences between what you are proposing and typical A-life approaches:

Evolutionary algorithms need some measure of fitness, in order to select the best performing individuals to take forward. A-life solves this by implementing a very open environment that makes no direct judgement on outputs of functions, but allows virtual creatures that collect enough resources (defined in the environment) to procreate.

Your suggestion contains no hint that you are thinking of any kind of measure of success or fitness for either the editing NN or target NN. You will need some measure of fitness at least for the target network (and maybe the editing network too) if you intend to use evolutionary algorithms

A-life typically does not treat the evolutionary algorithm itself (the editing or the NNs) as a learning goal. You will not see the results of a good or bad editor until many simulations have passed, so this "meta-search" is likely to be incredibly slow.

A-life simulations are typically already quite slow to reach behaviour which is interesting (because it has emerged without direction from the developer), but usually quite simple for the given environment such as predators chasing prey.

From what we know of the evolution of life, a simple feedback mechanism of RNA molecules editing other RNA molecules - the "RNA world" - is considered a likely pre-life step. This has some parallels with what you are suggesting - and this or something similar perhaps, has resulted in intelligent beings such as ourselves. So your idea perhaps has some merit in a theoretical sense. However, it took biology billions of years to go from such a basic stage to self-aware creatures, and this took the full processing power of large numbers of atoms interacting in ways that even all the computers in the world could not simulate in a fraction of real time.

To speed things up, and turn the idea into something feasible, you would need to look into viable environments and evaluations that would focus the learning towards direct measures of intelligence. Also, you would do well to compare your idea about NNs editing NNs with learning approaches that don't use such a feedback loop.

There is no theory that suggests a NN that can edit or output other NNs would offer any advantage for research into strong AI, compared to other search methods. The idea is basically "AI alchemy" - an idea of an experiment that could perhaps be done, but without any theory backing it as being better or worse than other ideas.

Personally, I would expect the search for a NN which is a good NN editor for a NN which has some other task, to be too slow to be useful when faced with very broad tasks such as exhibiting high level reasoning.


Is radial basis function network appropriate for small datasets?

Not familiar with RBF but when you have a small data set image augmentation can help. You can do this easily with the Keras ImageDataGenerator, documentation is here. Alternatively you can create image augmentation yourself using image processing models like PIL or CV2.


This is a consequence of the Markovian assumption, which underpins all of RL.

The Markovian assumption says that it doesn't matter how we reached a given state, only that we reached it, when deciding how likely it is that we move to subsequent states. This naturally implies that our choice of actions must also depend only on the current state.

You are correct that the assumption is somewhat unrealistic. However, it usually can be made to yield a reasonable approximation of the true dynamics without involving too many state variables. By solving this approximation, we hope to find a policy that also works well in the real problem.

Here's an example. In robot navigation, the real dynamics of the robot do depend on where the robot has been in the past, because as the robot's battery drains, the voltage levels it outputs will change slightly, and its wheels may become more prone to slippage. Thus, logically, our choice of action should change based on both the current state and our choice of actions in previous states (which drained the battery to a greater or lesser degree). However, if we try to incorporate this into the model, we'll end up having a combinatorial explosion in size of the dynamics function $P(S_{t+1} | S_t, a_{t}, S_{t-1}, a_{t-1} ... S_0, a_0)$ that actually captures this process (in particular, it will now be a function of 2t inputs). This, in turn, will necessitate a combinatorial blowup in the complexity of our policy (it will increase in complexity by an exponent of 2t). To keep things tractable, we accept that the dynamics will become lossy, or we can add some extra detail to the local state (e.g. a battery level) to capture the more complex dynamics in a Markovian way. Either way though, we'll be back to a world where future actions won't depend on past ones.

To be clearer, if we assume that $P(S_{t+1} | S_t) = P(S_{t+1} | S_{t}, S_{t-i})$ for any $i$, then the other relationships you mention should not seem surprising. That assumption is the Markovian assumption. It converts the state transition function into a Matrix that represents a Markov Chain. If we don't make that assumption, then most RL algorithms do not apply to our problem.


This article on Dynamically Expandable Neural Networks (DEN) (by Harshvardhan Gupta) is based on this paper Lifelong Learning with Dynamically Expandable Networks (by Jeongtae Lee, Jaehong Yoon, Eunho Yang, Sung Ju Hwang)

This presents 3 solutions to increase the capacity of the network if needed retaining whatever useful information from the old model and train the new model:

Selective retraining
Dynamic Network Expansion
Network Split/Duplication

To me, it seems that such neural network is dynamic and improving. As such, they answer partially your question. If they don't sorry about that.


What is the meaning of "exploration" in reinforcement and supervised learning?

In reinforcement learning, exploration has a specific meaning, which is in contrast with the meaning of exploitation, hence the so-called exploration-exploitation dilemma (or trade-off). You explore when you decide to visit states that you have not yet visited or to take actions you have not yet taken. On the other hand, you exploit when you decide to take actions that you have already taken and you know how much reward you can get. It's like in life: maybe you like cereals $A$, but you never tried cereals $B$, which could be tastier. What are you going to do: continue to eat cereals $A$ (exploitation) or maybe try once $B$ (exploration)? Maybe cereals $B$ are as tasty as $A$, but, in the long run, $B$ are healthier than $A$.
More concretely, recall that, in RL, the goal is to collect as much reward as you can. Let's suppose that you are in state $s$ and, in the past, when you were in that state $s$, you had already taken the action $a_1$, but not the other actions $a_2, a_3$ and $a_4$. The last time you took action $a_1$, you received a reward of $1$, which is a good thing, but what if you take action $a_2, a_3$ or $a_4$? Maybe you will get a higher reward, for example, $10$, which is better. So, you need to decide whether to choose again action $a_1$ (i.e. whether to exploit your current knowledge) or try another action that may lead to a higher (or smaller) reward (i.e. you explore the environment). The problem with exploration is that you don't know what's going to happen, i.e. you are risking if you already get a nice amount of reward if you take an action already taken, but sometimes exploration is the best thing to do, given that maybe the actions you have taken so far have not led to any good reward.
In hyper-parameter optimization, you do not need to collect any reward, unless you formulate your problem as a reinforcement learning problem (which is possible). The goal is to find the best set of hyper-parameters (e.g. the number of layers and neurons in each layer of the neural network) that performs well, typically, on the validation dataset. Once you have found a set of hyper-parameters, you usually do not talk about exploiting it, in the sense that you will not continually receive any type of reward if you use that set of hyper-parameters, unless you conceptually decide that this is the case, i.e., whenever you use that set of hyper-parameters you are exploiting that model to get good performance on the test sets that you have. You could also say that when you are searching for new sets of hyper-parameters you are exploring the search space, but, again, the distinction between exploitation and exploitation, in this case, is typically not made, but you can well talk about it.
It makes sense to talk about the exploitation-exploration trade-off when there is stochasticity involved, but in the case of the hyper-parameter optimization there may not be such a stochasticity, but it's usually a deterministic search, which you can, if you like, call exploration.


When using experience replay in reinforcement learning, which state is used for training?

The way the states are used is as follows:
Typically your $Q$-network will state a state as input and output scores over the action space. I.e. $Q : \mathcal{S} \rightarrow \mathbb{R}^{|\mathcal{A}|}$. So, in your replay buffer you should store $s_t, a_t, r_{t+1}, s_{t+1}, \mbox{done}$ (note that done just represents where the episode ended on this transition and I add for completeness.
Now, when you are doing your batch updates you sample uniformly at random from this replay buffer. This means you get $B$ tuples of $s_t, a_t, r_{t+1}, s_{t+1}, \mbox{done}$. Now, I will assume $B=1$ as it is easier to explain and the extension to $B > 1$ should be easy to see.
For our state-action tuple $s_t, a_t$ we want to shift what the network predicts for this pair to be closer to $r_{t+1} + \gamma \arg\max_a Q(s,a)$. However, our neural network only takes the state as input, and outputs a vector of scores for each action. That means we want to shift the output of our network for the state $s_t$ towards the target I just mentioned, but only for the action $a_t$ that we took. To do this we just calculate the target, i.e. we calculate $r_{t+1} + \gamma \arg\max_a Q(s,a)$, and then we do gradient ascent like we would a normal neural network where the target vector is the same as the predicted vector everywhere except the $a_t$th element, which we will change to $r_{t+1} + \gamma \arg\max_a Q(s,a)$. This way, our network moves closer to our Q-learning update for only the action we want, in line with how Q-learning works.
It is also worth nothing that you can parameterise your Neural Network to be a function $Q: \mathcal{S} \times \mathcal{A} \rightarrow \mathbb{R}$ which would make training more in line with how tabular Q-learning but is seldom used in practice as it becomes much more expensive to compute (you have to do a forward pass for each action, rather than one forward pass per state).


What is the difference between local search and global search algorithms?

The difference between a local search algorithm (like beam search) and a complete search algorithm (like A*) is, for the most part, small.

Local search algorithms will not always find the correct or optimal solution, if one exists. For example, with beam search (excluding an infinite beam width), it sacrifices completeness for greater efficiency by ordering partial solutions by some heuristic predicting how close a partial solution is to a complete one. Beam search is a greedy algorithm.

Complete search algorithms will always find the correct or optimal solution if there is one, given enough time. An algorithm like A* uses heuristics to prune the tree as it goes along, but it will not converge to a sub-optimal solution. In a lot of practical cases, this is inefficient, but how much so is problem dependent.


Are iterative deepening, principal variation search or quiescence search extensions of alpha-beta pruning?

Are these algorithms an extension of the alpha-beta algorithm, or
  Are they completely new algorithms, in that they have got nothing to do the alpha-beta algorithm?

Most of them are extensions of the Alpha-Beta pruning algorithm. For example, Iterative Deepening is almost the same as Alpha-Beta pruning, but automatically keeps repeating the algorithm with gradually-increasing depth limits until some time limit is reached, rather than just running once for a pre-determined depth limit.

Principal Variation Search also still uses Alpha-Beta as a basis, but performs many searches with significantly smaller [alpha, beta] windows than the standard Alpha-Beta pruning algorithm.

In most cases, these extensions would start out from an existing Alpha-Beta implementation, and build from there with some adaptations in the code. This is not necessarily the case for all of those extensions though, just for most. For example, Transposition Tables are kind of a separate extension that could be plugged into vanilla Minimax, or Alpha-Beta, or Principal Variation Search, or whatever you're using.

  On this site, these algorithms fall into one of 4 categories, namely

  mandatory
  selectivity
  scout and friends
  Alpha-beta goes best-first

  Does this mean that the alpha-beta algorithm is split into four areas and that they are specialized optimization algorithms for each area?

Those four categories are not mutually exclusive, they're more like... broad "flavours". What they list under Obligatory are some of the more basic extensions that any programmer should probably look into first if they were developing a chess-playing program. The other category are different "flavours", different "broad ideas". For example, everything listed under "Selectivity" is about searching "interesting" or "exciting" parts of the search tree deeper than "less interesting" or "boring" parts. Many of those ideas could be used regardless of whether you're using Alpha-Beta, Iterative Deepening, or PVS, and probably all could be combined with Transposition Tables as well.

  How do I even begin to decide which optimized algorithm to pick?

This is really really difficult to decide just based on the names. In theory, which algorithm is the "best" will also highly depend on your specific game, and maybe even hardware. And, in many cases it's not even a choice between mutually exclusive parts; different ideas can be combined with each other in different ways.

The only solution here is really just to do lots of reading, lots of research, try implementing different things to better understand them.


How to take the optimal batch_size for training a model?

This Cross Validated post might answer your question.
In a nutshell:

A single batch (that is all your data in one batch) will result in a smooth trajectory on the loss surface. The drawback is that all your data might not fit into your memory. Which is highly likely for ~100k images.

One image per batch (batch size = no. examples) will result in a more stochastic trajectory since the gradients are calculated on a single example. Advantages are of computational nature and faster training time.

The middle way is to choose the batch size in such a way that your batch fits into memory and gradients behave less 'noisy'. To be honest there is no 'golden' number, personally I like to choose powers of two.
Don't worry that your data is not divisible by the batch size. Libraries will take care about that internally, the last batch will just be a smaller than the defined batch size ($N \text{ mod } b$).


How would I compute the optimal state-action value for a certain state and action?

It seems that you are getting confused between the definition of a Q-value and the update rule used to obtain these Q-values.
Remember that to simply obtain an optimal Q-value for a given state-action pair we can evaluate
$$Q(s, a) = r + \gamma \max_{a'} Q(s', a)\;;$$
where $s'$ is the state we transitioned into (note that this only holds when obtaining the optimal Q-value, if we were using a stochastic policy then we would have to introduce expectations).
Now, this assumes that we have been given/obtained the optimal Q-values. To obtain them, we have to use the update rule (or any other learning algorithm) that you mentioned in your question.


Can an AI model be based on the principles of rationalism?

It depends on how you define rationality.  In the field of AI, the game theoretic conception is utilized, in sense of a rational agent.

It's important to note that a decision making agent can be perfectly rational and still be "wrong" (i.e. make sub-optimal decisions.)

This is primarily because rationality is bounded for all non-trivial problems.  The quality of decisions depends on quality of information, time and space available to make the decision in relation to the problem size, and the quality of the analytic processes.

Logic itself has pitfalls, in that any formal system can either be complete or consistent, but never both.

I would say that the current, strong AIs are empiricists, in that the methods are statistical, and based on testing and evaluation of results.  For instance, a Monte Carlo Tree Search (MCTS) game algorithm will play billions of games against itself and analyze the results to make the strongest choice for any given state.

But, if one considers any decision-making agent, no matter how primitive or weak, to be a form of AI, then there can be non-empirical AI in the sense that the processes heuristic as opposed to statistical.


Given unlimited time and a perfect fitness function, could a genetic program solve any problem?

Yes.

A totally random algorithm could solve any problem given unlimited time and a perfect fitness function. All you need to do is give the GA some new random population members each generation and you're guaranteed to find the solution eventually. Even if you keep only descendants of the previous generation, setting the mutation rate and number of crossovers high enough could effectively get you random individuals.


Update in REINFORCE algorithm - step-wise or episode-wise?

Hi Seewoo Lee and welcome to our community!

The essence of your observation is that Sutton's version of REINFORCE is taking into consideration all of the trajectory to compute the returns while in the pytorch version only the future is taken into consideration, hence going in reverse to sum the future rewards and ignore the previous rewards. The consequence is that future actions are not punished for early mistakes. The guys at OpenAI refer to this as reward-to-go but personally, I find that it resembles Monte Carlo On Policy Control without Exploring Starts or First Visit from Sutton's book.

You can find more on REINFORCE and Policy Gradient in Spinning Up RL: Part3: Intro to policy gradient -  Don't let the past distract you.

Also, something to note is that even in Sutton's version, the whole trajectory is unrolled, i.e. the episode completes, and then the weights get updated. Otherwise it stops being  a Monte Carlo Method and it becomes a TD method. In addition, you can't make a change on a single point because sampling is a non differentiable operation, instead, the gradient is estimated by collecting a lot of trajectories.


Can Deep Learning be applied to Computational Fluid Dynamics

Read the paper Deep learning in fluid dynamics (by J. Nathan Kutz), and you will find your answer.


Eligibility vector for softmax policy with policy gradients

Calculation of gradient
\begin{align}
\nabla_{\theta} \log(\pi_{\theta}(a|s)) &= \phi(s,a) - \mathbb E[\phi (s, \cdot)]\\
&= \phi(s,a) - \sum_{a'} \pi(a'|s) \phi(s,a')
\end{align}
is only valid for linear function approximation with action preferences of form
\begin{equation}
h(s, a, \theta) = \theta^T \phi(s,a)
\end{equation}
and softmax policy
\begin{equation}
\pi(a|s) = \frac{e^{h(s,a,\theta)}}{\sum_{a'} e^{h(s,a',\theta)}}
\end{equation}
The gradient would be calculated as it is written. For example, if your current state is $s = (1, 1)$ and in that state you have actions $a_0 = 0$ and $a_1 = 1$ and probabilities for those actions are $\pi(a_0|s) = 0.7$, $\pi(a_1|s) = 0.3$ then gradient for action $a_0$ would be
\begin{equation}
\nabla_{\theta} \log(\pi_{\theta}(a_0|s)) = (1, 1, 0)^T - (0.7 \cdot (1, 1, 0)^T + 0.3\cdot (1,1,1)^T) = (0, 0, -0.3)^T
\end{equation}
Feature vector $\phi$ can be basically anything you want. For example you could stack state feature and action (like I did in small example), you could use polynomials, radial basis functions, tile coding, etc.

If you're using multilayered neural network you would have to propagate gradients through all layers, usually done with backpropagation algorithm. Easiest way is to use automatic differentiation software (e.g. Tensorflow) which can do that for you so you don't have to write your implementation. All you have to do is define your objective function that you want to optimize
\begin{equation}
J_\theta = \sum_t \log(\pi(a_t|s_t, \theta)) G_t
\end{equation}
and software will automatically calculate gradient $\nabla J_{\theta}$ and update weights.


A model is exactly what the name suggests.  A simplified representation of a solution to a real life problem.

For example, if you think of a simple formula for a falling object you may not take into account real life variables such as the imperfections on the surface of the object, atmospheric conditions, the exact composition of the air the object is falling through and how it may affect friction.  you just use a simple model to "model" the fall and get a reasonably accurate result.  its a model, it doesnt have all the details.  just like a model car wouldnt have all the details of a real car.

Here's a good read to get you started:
https://en.wikipedia.org/wiki/Mathematical_model


How to add weights to one specific input feature to ensure fair training in the network?

I think your approach to tackle this as an imbalanced problem is correct. The easiest thing you could do is to add weights to the samples, during training, so that the model "pays more attention" to the under-represented class.

There are also a couple of other ways for this to be done: oversampling and undersampling, but initially, I'd focus on adding weights, since its easier to implement.


Autoencoder network for feature selection not converging

The trick was to normalize the input dataset values with the respective mean and standard deviation in each column. This reduced the loss drastically, and my network is training more efficiently now. Moreover, normalizing the data also helps you calculate the weights associated with each input node more easily, especially when trying to find out variable importance.


Reinforcement Learning (Fitted Q): Qn on Concept & Implementation

1): The intuition is based on the concept of value iteration, which the authors mention but don't explain on page 504. The basic idea is this: imagine you knew the value of starting in state x and executing an optimal policy for n timesteps, for every state x. If you wanted to know the optimal policy (and it's value) for running for n+1 timesteps in each location, this is now easy to compute. The optimal action from state x is whichever one maximizes the sum of the reward for this timestep (r) and the value of executing an optimal n-step policy from the state you'd end up in after (or expected value if the problem isn't deterministic).

In the approach of the paper, you're not going to compute either the policy or the value explicitly (probably because it's too expensive), so you just approximate the Q function for the n+1 problem.

IIRC, as long as your problem has a discounting factor and the error in your function approximation isn't too large, there are proofs (see Russell & Norvig's chapter on RL (18?)) that your policy will eventually stop changing in-between updates, and will be consistent with the policy for an infinite number of steps. Intuitively, this is because the discounting factor causes a series for the rewards to be convergent.

2): I think that's right. When you build the training set, use the actions suggested by the results for the $$Q_{n-1}$$ network. That's an approximation of the reward for starting in each state and running for n-1 steps with an optimal policy. Then you're learning an approximation of $Q_n$ from that, which looks right.


What are examples of applications of the Fourier transform to AI?

The filtering (or convolution) process in Convolutional Neural Networks can be implemented using Fourier Transforms. Convolutions in image (or spatial) space are equivalent to multiplications in frequency space, and multiplications can be performed much more efficiently than convolutions.

The algorithm for using Fourier Transforms to calculate a convolution between an image and a filter kernel is as follows:

Assuming a 100x100 input image, and a 3x3 filter kernel (such as a Sobel Edge-Detection filter).

Calculate the 2D Fourier Transform (FT) of the image (C library, Python library)
Pad the filter kernel so that it is the same size as the image, and that the filter kernel is centred at image coordinate (0, 0). This means that some parts of the kernel will be placed in the other corners of the padded filter image. For example, consider a 3x3 kernel where the filter coefficients are [[a, b, c], [d, e, f], [g, h, i]], in this case the coefficients map to pixel positions on the padded kernel as follows:

a -> (99, 99)    //bottom right
b -> (0, 99)     //bottom left
c -> (1, 99)
d -> (99, 0) //top right
e -> (0, 0) //top left
f -> (1, 0)
g -> (99, 1)
h -> (0, 1)
i -> (1, 1)

Set all the other padded filter kernel pixels to zero
Calculate the 2D Fourier Transform of the padded filter kernel.
Perform an element-wise multiplication of the image FT and the padded filter kernel FT. Note that this is a complex multiplication as the FT consists of complex numbers. The result of this multiplication is therefore also an array of complex numbers.
Calculate the Inverse Fourier Transform of the result from step 4. This will return a real valued image which is the original image convolved with the filter kernel.

The advantage for performing the kernel convolutions in this way is that it is much faster than performing an element-wise convolution directly. This speeds up the training process for neural networks with Convolutional layers.


That the features are not stationary is a reason why LSTM is a good approach.  Calculating the ratio of adjacent values in the sequence is unnecessary.  Since the logarithm would remove a skew in the input of numbers close to zero, the acquisition is in question.  Why are the values not often varying?  Are their transient events followed by inactivity?

Aggregating unchanged items in the sequence and adding duration as a feature may add entropy to the input and improve LSTM performance.  This is called decimation, and can be useful in adding entropy (removing redundancy) in data that has regions of redundant oversampling.

If the values without the division are still distributed exponentially, then keep the logarithm but maybe make it a natural log.


Which algorithm should I choose for lead scoring

ML is a wide and deep topic to cover.

What you need is "classification" to predict an outcome. At first, you need to convert everything to numbers (check word2vec). Then, pre-process the data (Check what is a normalization for example). Train a classifier by using any ML techniques (i.e. linear classifier, SVM, deep learning, neural classifier). Then you can use that regression function to predict further values. If you need much more detailed way to do that, it is better to buy a 'thick' book that explains all other options.


What is the meaning of "easy negatives" in the context of machine learning?

OK, I think I understood what this means. Hard and easy negatives are ones which have a relatively large and small values for the loss function respectively.


How can we compare the intelligence of AI systems?

There are different ways to compare different kinds of AI techniques.

As a starting point, be aware that "AI System" can mean an incredibly broad range of things. In popular culture, we usually think of a deployed system that uses AI techniques. These systems can only be compared on the basis of their performance, and their performance may have relatively little to do with AI itself (e.g. their behaviours might be more strongly affected by user interface decisions, not the AI techniques under the hood).

In contrast, AI researchers are usually more interested in comparing the performance of different AI algorithms at solving the "AI-ish" parts of the problem a fully developed system aims to solve. A common way to do this is with benchmark problems. For example, in machine learning it is common to compare two algorithms by running each of them on a commonly used dataset, and comparing the performance of the models they create. In AI Planning, it is common to issue planning challenges to the community, and compare the quality of the plans on several different axes (e.g. average wait times, maximum wait times, whether goals were accomplished, how long it took to create a plan, etc.).

There is no generally agreed upon way to compare techniques across different areas of AI, but a commonly adopted approach is the Turing Test. In the Turing Test, we care only about the ability of the system to mimic something like human intelligence. It's fair game to ask about planning problems, or learning problems, or other topics, so you could in some sense judge one technique to be better than another. However, most judgements made in the Turing Test are subjective, so it's not clear that it really solves the problem you posed.


Are there names for neural networks with a well-defined layer or neuron characteristics?

Neural networks (NNs) are usually classified into feed-forward (i.e. NNs with feedforward connections), recurrent (i.e. NNs with recurrent connections) and convolutional (i.e. NNs that perform a convolution or cross-correlation operation). The term multi-layer NN may also be used to refer to feed-forward neural networks or, in general, neural networks with multiple (hidden) layers. There are also the perceptrons, which do not have hidden layers (i.e. the inputs are directly connected to the outputs).

You may still divide neural networks into classifiers (i.e. the outputs and labels are discrete) and regressors (the outputs and labels are numerical). Furthermore, you may classify them into generative models (e.g. the VAE) or discriminative models.

By analogy with linear algebra concepts, each layer of a feedforward neural network (FFNN) can be seen as a linear operation followed by an element-wise application of a linear or non-linear activation function.

$$
\mathbf{o}^{l} = \sigma \left(\mathbf{a}^{l} \mathbf{W}^{l} + \mathbf{b}^{l}\right)
$$

where $\sigma$ is the activation function, $\mathbf{a}^{l}$ the inputs to the layer $l$ and $\mathbf{W}^{l}$ the parameters of the layer $l$ (similar to the parameters or coefficients in linear regression) and $\mathbf{b}^{l}$ is the bias vector (a scalar bias for each neuron) of layer $l$. $\mathbf{o}^{l}$ will then be $\mathbf{a}^{l+1}$ (i.e. the input to the next layer).

A recurrent neural network (RNN) performs a slightly more complex operation.

$$
\mathbf{o}^{l}_t = \sigma \left(\mathbf{a}^{l}_t \mathbf{W}^{l} + \mathbf{o}^{l}_{t-1} \mathbf{R}^{l} + \mathbf{b}^{l}\right)
$$

where, in addition to the matrix $\mathbf{W}^{l}$ associated with the feedforward connections, it also uses another matrix $\mathbf{R}^{l}$ associated with the recurrent connections (i.e. cyclic or loopy connections of the neurons), which is multiplied by the output of the same layer but at the previous time step. $\mathbf{o}^{l}_t$ may actually just be the state of the recurrent layer, which is then used to compute the actual output of the layer, but, for simplicity, you can ignore this. Furthermore, note that there are more complex recurrent architectures, but this is the basic idea.

A convolutional neural network (CNN) performs the convolution (or cross-correlation) operation. If you are familiar with signal processing, e.g. kernels, convolution, etc., then you can view a CNN as performing a convolution (or cross-correlation) operation. It's actually possible to view the convolution as a matrix multiplication operation, but the details can easily become cumbersome and tedious to explain in an answer. A CNN may also perform other types of operations (such as downsampling) and it can also be composed of a feedforward part (usually, the last layers of a CNN are feedforward layers), but a CNN is a CNN because it performs the convolution (or cross-correlation).

In all cases, the matrices do not necessarily take any particular form (e.g. they are not necessarily square matrices). The dimensionality of the matrices depends on the number of layers and connections in the network, which can vary depending on many factors (e.g. the need for recurrent connections because they are useful for sequence modeling). These matrices (along with the biases) are the learnable parameters of the networks, but the learned values of these matrices highly depend on your data, the way you initialize them before learning, the architecture of the network, the learning algorithm, etc.

In the case of the FFNN, if the previous layer $l-1$ has the same number of neurons as the current layer $l$, then $\mathbf{W}^{l}$ is a square matrix. AFAIK, there is no name for a neural network with the same number of neurons for each layer. It could be called a rectangular neural network (but this is a name I've just come up with).

To conclude, there are many different neural networks and taxonomies for neural networks, so it is impossible to list or discuss them all in an answer, but, nowadays, the subdivision into feedforward, recurrent and convolution is the most common and general. See e.g. the paper A Taxonomy for Neural Memory Networks if you are interested in a more detailed taxonomy for memory-based neural networks (e.g. recurrent neural networks).


Can recovering a reward function using IRL lead to better policies compared to reward shaping?

Inverse Reinforcement Learning (IRL) is a technique that attempts to recover the reward function that the expert is implicitly maximising based on expert demonstrations. When solving reinforcement learning problems, the agent maximises a reward function specified by the designer, and in the process of reward maximisation, accomplishes some task that it had set out to do. However, reward functions for certain tasks are sometimes difficult to specify by hand. For example the task of driving takes into consideration many different factors such as the distance of the car in front of him, the road conditions and whether or not the person needs to get to his destination quickly. A reward function can be hand specified based on these features. However, when there exists trade offs between these different features, it is difficult to know how to specify the different desiderata of these tradeoffs.
Instead of specifying the trade offs manually, it would be easier to recover a reward function from expert demonstrations using IRL. Such a reward function can lead to better generalisations to unseen states as long as the features of driving do not change.
In the case where reward shaping fails to learn a task (such as driving), it would be better to have someone demonstrate a task and learn a reward function from these demonstrations. Solving the MDP with the learnt reward function will thus yield a policy that should resemble the demonstrated behaviour. The reward function learnt should also generalise to unseen states and the agent acting in unseen states should be able to perform actions that an expert would take when he is placed in the same conditions, assuming that the unseen states come from the same distribution as the training states.
While Reward Shaping might be able to perform the same task as well, IRL might be able to do better, based on some performance metric that will differ from problem to problem.


How should the neural network deal with unexpected inputs?

This is a very important problem that is usually overlooked. In fact, when training a neural network, there's often the implicit assumption that the data is independent and identically distributed, i.e., you do not expect the data to come from a distribution different than the distribution from which your training data comes, so there's also the implicit assumption that data comes from the same family of distributions (e.g. only Gaussians) and that all your training examples are independently drawn from the same distribution (specific mean and variance). Of course, this is a big limitation!
A partial solution to your problem is to use a Bayesian neural network (BNN). The idea of a BNN is to associate, rather than a single number, a distribution (usually a Gaussian distribution) with each unit (or neuron) of the neural network. Therefore, for each unit of the network, there are two learnable parameters: the mean and variance of a Gaussian distribution. Consequently, a BNN usually has the double of the number of parameters of a conventional (or non-Bayesian) neural network. However, by learning a distribution for each parameter, you also learn the uncertainty about the potential true value of each unit, based on the available training data.
The forward passes of such a BNN are stochastic, i.e. you sample from each of these Gaussian distributions for every forward pass, so the output of the network is also stochastic (i.e. given the same input example, the output may be different each time).
If your dataset is small, one expects the BNN to have wide Gaussian distributions, i.e. high uncertainty about the true value of the units. So, one expects a BNN to be able to deal with unexpected inputs more robustly. To be more precise, if you train a BNN with a small dataset, the hope is that Gaussian distributions will be wide and thus the outputs of the BNN will be highly variable (i.e. the model is highly uncertain). The more data is gathered, the less uncertain the BNN should be.
This doesn't completely solve your problem, but it should at least mitigate it, i.e. if you provide an unseen example to the BNN, then ideally it should be uncertain about the actual label of that input.
For simplicity, I didn't explain certain details of BNNs, but, at least, this answer gives you a potential solution. Of course, this doesn't exclude the possibility of having an "unknown" class. The approaches are not mutually exclusive. There may also be other solutions, but I am not aware of any.


Does machine learning continue to learn?

This may not seem trivial but yes, the models we train can potentially learn a variety of things they weren't intended to learn. There are already some examples in computer vision. A typical convolutional network learns things like edge detection, various potentially useful masks etc. in the early layers while learns more high-level features like eyes, nose etc. in higher layers.

It is reasonable too. Given the dataset size is moderately high and the model is trained for long enough, a sufficiently deep network learns various kinds of hidden representations, which may not even be specific to the task at hand. This is the reason transfer learning works very well even on a host of different datasets.

This is limited since not all the learnable things can be described using mathematics. So, the answer is a surprising no. The model does learn some extra things other than the task at hand.

P.S.: There was also a case when a group of researchers trained a model to make a robot walk. It turned out the robot had learned to recognize faces too and reacted in different ways on seeing different faces. I saw the video on YouTube a while ago and couldn't find the exact video to post the link here, anyways.


Should illegal moves be excluded from loss calculation in DQN algorithm?

I've implemented this exact scenario before; your approach would most likely be successful, but I think it could be simplified.

Therefore, when deciding on which action to pick, agent sets Q-values to 0 for all the illegal moves while normalizing the values of the rest.

In DQN, the Q-values are used to find the best action. To determine the best action in a given state, it suffices to look at the Q-values of all valid actions and then take the valid action with highest Q-value. Setting Q-values of invalid actions to 0 is unnecessary once you have a list of valid actions. Note that you would need that set of valid actions to set invalid Q-values to 0 in the first place, so the approach I'm suggesting is more concise without worsening the performance.
Since the relative order of the Q-values is all that is required to find the best action, there is no need for normalization. Also, the original DQN paper uses $\epsilon$-greedy exploration. Keep in mind to only sample from valid actions in a given state when exploring this way.

During training, when the agent is calculating the loss between policy and target networks, should the illegal actions be ignored (set to 0) so that they don't affect the calculations?

As noted in one of your previous questions, we train on tuples of experiences $(s, a, r, s')$. The definition of the Q-learning update is as follows (taken from line 6.8 of Sutton and Barto):
$$Q(S_t, A_t) \leftarrow Q(S_t, A_t) + \alpha \left[R_{t+1} + \gamma\max\limits_aQ(S_{t+1}, a) - Q(S_t, A_t)\right].$$
The update requires taking a maximum over all valid actions in $s'$. Again, setting invalid Q-values to 0 is unnecessary extra work once you know the set of valid actions. Ignoring invalid actions is equivalent to leaving those actions out of the set of valid actions.


Is there a way of computing a prominence score based on the prevalence of features in an image?

Saliency is typically discussed more in object detection or scene understanding than prominence.  There are lots of papers on saliency models.  "What do different evaluation metrics tell us about saliency models?" is a good paper on various metrics on saliency models.  It covers the following metrics:

Similarity or histogram intersection (SIM)
Pearson's Correlation Coefficient (CC)
Normalized Scanpath Saliency (NSS)
Area Under ROC Curve (AUC)
Information Gain (IG)
Kullback-Leibler divergence (KL)
Earth Mover's Distance (EMD)

Some other papers you may find interesting:

Saliency, attention, and visual search: An information
theoretic approach
Towards Instance Segmentation with Object Priority:
Prominent Object Detection and Recognition


What is the name of an AI system that learns by trial and error?

Near solution to your problem definition is reinforcement learning. You can define some reward using the objective function and define some possible state space for the machine and finally solve the problem by reinforcement learning techniques (near to trial and error by learning the preferences).


In reinforcement learning, does the optimal value correspond to performing the best action in a given state?

I am wondering which definition is correct.

The asterisk * in both the definitions stands for "optimal" in the sense of "value when following the optimal policy"
So this one is correct:

$V^*$ actually assumes the optimal action in a given state, meaning $V^*$ would be $50$ in the above case

However, you have got the definition of Q slightly wrong.
I think this is because you are omitting the parameters.
The state value function uses the state as a parameter, $V_{\pi}(s)$, it returns the value of being in state $s$ and following a fixed policy $\pi$. The * is used to denote following an optimal policy.
The action value function has two parameters - a state and an action that is possible in that state, $Q_{\pi}(s, a)$, it returns the value of being in state $s$, taking action $a$ (regardless of whether it is the best action or not) and following the policy $\pi$ after that point.
Your assertion in the question:

And therefore my $Q^* = 50$

is wrong, or rather not meaningful, as you have not stated the parameters. You already list all the possible values of Q with the parameters. You could say $\text{max}_a Q(\text{toy store}, a) = 50$, or to choose the best action $\pi(\text{toy store}) = \text{argmax}_a Q(\text{toy store}, a)
= \text{Lego}$


How to train a LSTM model with multi dimensional data

LSTM can be tricky, I'll give my $0.02.
LSTM input layer defines the shape so it would be something like this.
If I am understanding your question correctly, your data can be framed as 184 samples with 2 time steps and 70 features?
So the start of the code might look like this.
model = Sequential()
model.add(LSTM(184, input_shape=(50, 2)))


Should my agent be taking varying number of steps?

Your graph looks to me like a typical learning curve plotted for training process in reinforcement learning.
Looking at it in detail I can say:

There is clearly some learning occurring.

There is a strong random element throughout. As you say the epsilon is reduced to near zero by episode 2000, and I would assume a driving simulation is mostly deterministic but with a lot of state hidden from the agent, this mainly implies that episodes are started in different states.

There may be an effect from continued learning that causes the agent performance to vary so much. However, it is more likely towards the end that the agent is encountering new never-seen-before states and making poor decisions in them.

The resulting graph matches to my experience of training DQN agents where some of the hyperparameters are off. Probably you need to do some kind of exploration or search through those parameters.

From your comments:

Perhaps the model is subsequently overfitting? Or maybe there is some catastrophic forgetting?

These are possibilities that I cannot rule out by looking at the graph.
However, I think it is more likely that you have some agent design or hyperparameters that are not a good fit to the problem. Those hyperparameters could be almost anything - not enough episodes to cover variation, neural network too simple, neural network too complicated, epsilon decay too fast, poor choice of regularisation, experience replay memory too small, poor choice of optimiser, etc etc.
It is also possible you have limited the state model to the point that learning is hard or even impossible. A common beginner's mistake here is to use static sensor information such as a single current screenshot on each timestep, so the agent has no way to assess how fast it is going, or which way it is already turning etc (if your input includes direct knowledge of current speed, turning etc then this is probably not a problem for you, I am just raising one of many possibilities).
One thing that may help you understand the agent's learning performance better is instead of plotting the learning episode graph, plot a test graph, perhaps once every ~100 episodes, where you run some number of episodes without learning and with $\epsilon = 0$, and take average of the metrics you are interested in. This will remove some of the randomness from the values you are plotting and give you a much better sense of training progress than your current plot.
With Q learning, you may also get significant benefit by setting a non-zero minimum $\epsilon$, e.g. $\epsilon = \text{max}(\text{eps_decay}(n), 0.01)$ where $\text{eps_decay}(n)$ is your current epsilon decay function for step $n$. That is because the off-policy updates with exploration are still helpful at all stages of learning.


Get Neural Network to predict a tag/class on a certain word using the surrounding words as context [PyTorch]?

You may want to take a look at this article, but I'll summarize. You can use BERT (or some other tool) to make embeddings of every word in every sentence. Then for each word, make a contextualized embedding vector using the rest of the sentence. bert-embedding does all of this itself. Then keep the embedding vector for the important words.
For each important word, you would then have two pieces of information: the embedding vector and the correct label (which could easily be made into an integer from $0$ to $4$). Depending on the size of the embedding vectors, you could use PCA to reduce the size, although this may not be needed. Using this data, you can then train a neural network or use a k-nearest neighbors classifier.
There is more information in the article, which I suggest that you read. They do a better job explaining than me and also have some actual code you may want to look at.


When should you not use the bias in a layer?

The most usual case of bias=False is in layers before/after Batch Normalization with no activators in between. The BatchNorm layer will re-center the data anyway, removing the bias and making it a useless trainable parameter. Quoting the original BatchNorm paper:

Note that, since we normalize $Wu+b$, the bias $b$ can be ignored since its effect will be canceled by the subsequent mean subtraction

Similar thing happens in transformers' LinearNormalization and (as far as I understand how conditioning works) in the GANs' conditioning layer - the data gets re-centered, effectively cancelling the bias.
In my experience, that's the most frequent reason to see bias=False, but one can imagine other reasons to remove the bias. As a rule of thumb, I'd say that you don't include bias if you want to "transform zeros to zeros" - things like learned rotations can be an example of such (rather exotic) application.


YOLOv3 Model Structure: Why is filters = (classes + coords + 1) * num?

As said by @brale in the comment below the question:

filters = (classes + 5) * 3
= (classes + width + height + x + y + confidence) * num
= (classes + 1+1+1+1+1) * num
= (classes + 5) * num

YOLOv3 dectects 3 boxes per grid cell, so it is:

filters = (classes + 5) * 3


Why are we asking, "How can we simulate the brain?"

Human intelligence is very general / broad in its scope. This is self-evident, and whatever AI ends up to be, we'd like it to be a general problem solver as well (cf. Simon and Newell). Taking liberal interpretations of your question...

  Why AI in a computer?

Computers, to the extent that we can frame problems in general as a solvable computational problem, are also general problem solvers. Wether this is actually the case (can you compute meaning or feels?) is up for debate (cf. computational functionalism, hyper-computation), but it is part of the artificial intelligence project to make a claim on this statement.

  Why do we think a computational framework brings us any closer to an understanding of cognition / consciousness?

Good question, and frankly there is no good answer to that asides from "its the best thing we got".

TL;DR "computational functionalism", a lot of the literature in psychology and philosophy seems to converge towards an understanding of cognition as "computational" (as in information processing: the V1 stream in the brain processes "early visual information") and functional (goal directed grounded on "meaning", ex: "i scratch itch because itchy", as opposed to "i am moving atoms").

However the two theories don't mesh together well (cf. Chinese Room Argument, and the many other arguments in a similar flavour) despite their independent successes in the theory of mind. Why this is the case nobody quite knows...

  Why not AI in something that isnt a computer?

I don't know, but to the extent that our understanding of the world is grounded in math, then it being in a computer is sufficient anyways.

Maybe there are other paradigms of understanding the world though. Fingers crossed

  Why are we asking, “How can we simulate the brain?”

Because its the best tentative understanding we have of an "intelligent faculty", though it should be noted that various methods in machine learning don't seem to be directly inspired by biological implementation  (kNN, statistical methods, as opposed to neural nets)

Further reading: http://www.scaruffi.com/nature/mach01.html


Best architecture to learn image rotation?

This would likely suffer from the blurry image problem that autoencoders are known to suffer from. See also here.
On the other hand, using GAN's to sharpen your images doesn't seem particularly helpful since you seem to be lookng for a way to rotate general images, not ones of a specific domain. Moreover, there's almost certainly going to be some loss

It seems unreasonable/overkill to use neural networks to do this transformation. I suggest treating the rotation as a linear transformation, mapping certain pixel locations to other pixel locations (in fact, you can make this transformation differentiable with respect to its inputs using some deep learning library's autodifferentiation tools, if that's what you're looking for).
Since rotations are just linear maps that send pixels to other pixels, this will be much more computationally efficient.


Why not use the MSE instead of the current logistic regression?

The mean squared error (MSE), $J(\theta) = \frac{1}{2m}\sum_{i=1}^m(h_\theta(x_i)-y_i)^2$, is not as appropriate as a cost function for classification, given that the MSE makes assumptions about the data that are not appropriate for classification. Though, as an optimization objective, it is still possible to attempt to minimize MSE even in a classification problem, and thus still learn parameters $\theta$.

The new cost function has better convergence characteristics as it is more inline with the objective.

See link for the precise mathematical formulation that explains these loss functions from a probabilistic perspective.

Note that the absolute value is redundant because $\forall x:x^2\geq0$.

I hope this clarifies the matter.


What are some conferences for publishing papers on Deep Learning for Human Activity recognition?

If your using computer vision, the top recognised conference is CVPR (computer vision and pattern recognition).
You can also try to submit at ICML (International conference on machine learning) and NIPS (Neural Information Processing Systems), which focuses on applications of machine learning and deep learning.
I'd also recommend IJCAI (International joint conference on artificial intelligence).


How can I find words in a string that are related to a given word, then associate a sentiment to that found word?

There are many ways to solve this problem.  One way is to apply stemming or lemmatization to reduce your words. Using NLTK's Porter stemmer for example on healthy, healthier, healthiest, not healthy, more healthy, and zero healthy gives:
healthi , healthier , healthiest , not healthi , more healthi , zero healthi

This can help make word comparisons easier.
Sentiment analysis on the phrases will provide positive, neutral, and negative scores.  There are a lot of algorithms for doing this but a common one is Valence Aware Dictionary and sEntiment Reasoner (VADER).  Here is a recent article with code using NLTK and the VADER lexicon:

How To Run Sentiment Analysis In Python Using VADER by George Pipis

The following article also does sentiment analysis using NLTK and includes stemming and lemmatization.  Instead of VADER they use a Naive Bayes classifier on a labeled data set of tweets: How To Perform Sentiment Analysis in Python 3 Using the Natural Language Toolkit (NLTK) by Saumik Daityari.


Improving learning rate for classification task

You might find "What should I do when my neural network doesn't learn?" and "Reasons why your Neural Network is not working" interesting.


I think this is a risky project. There is a lot of specialist knowledge in neural networks and NLP. Trying to learn it "on the side" in order to complete your project may be biting off more than you can chew.
If you intend to write the neural network and train it yourself, you should be prepared for long sessions of training and testing variations of the neural network entirely separate from practicing any of the advanced C++ code that you are studying.
My recommendation instead of writing your own NN components, would be to learn the C++ API for Tensorflow and reference examples (like this one in Python/Keras) of using neural networks built in Tensorflow specifically for predicting sequence continuation.
I would also suggest you spend a little time trying out the Python and Keras-based networks to get a feel for what NN architectures work for your problem. If you use Python's tensorflow bindings, you may even be able to train a neural network separately using someone else's script. After training, in your C++ project you could then load the NN definition and use it for the NLP part of the project. You can search for examples of using C++ Tensorflow to load saved networks - this is a relatively common use case when someone wants to re-use a trained network for making new predictions on some edge device where the speed and efficiency of C++ make a large difference.


Is Python and a single computer sufficient to implement and train an RL-based agent for a turn-based war game?

Is Python mandatory?

Not at all. There is nothing magic about the Python language that makes it more suitable for learning algorithms. However, there are some very good machine learning and reinforcement learning libraries available for free that use Python. You might search for longer, or have more need to create your own code when using Objective-C.

  Is it hopeless to achieve anything with a single computer ?

That depends on the complexity of your game. The big projects using deep reinforcement learning use multi-million pound installations to research and train their game-playing algorithms. For instance, here is the spec used by Open AI learning to play DoTA:

    OpenAI Five plays 180 years worth of games against itself every day, learning via self-play. It trains using a scaled-up version of Proximal Policy Optimization running on 256 GPUs and 128,000 CPU cores — a larger-scale version of the system we built to play the much-simpler solo variant of the game last year.

Most of us cannot afford such a setup. If your game is as complex as DoTA, then this is an indication that you probably don't have the hardware to train an agent.

However, complexity works on a huge range of scales in RL. For example, it is possible to train an AlphaZero clone to play Connect 4 on a single machine with a decent GPU in around a day.

You will need to explore simpler games as you learn the algorithms, so that will give you a sense of your challenge. You can also present your bot with sub-problems within your game to learn. Doing this should help you get a better feel for the challenges of writing learning agents for your project.


What is the difference between machine learning and deep learning?

Deep learning is a specific variety of a specific type of machine learning. So it's possible to learn about deep learning without learning all of machine learning, but it requires learning some machine learning (because it is some machine learning).

Machine learning refers to any technique that focuses on teaching the machine how it can learn statistical parameters from a large amount of training data. One particular type of machine learning is artificial neural networks, which learn a network of nonlinear transformations that can approximate very complicated functions of wide arrays of input variables. Recent advances in artificial neural networks have to do with how to train deep neural networks, which have more layers than normal and also special structure to deal with the challenges of learning more layers.


beautify an image with reinforcement learning

The purpose of Reinforcement Learning is to maximize some notion of cumulative reward, leading me to the point (1) :
as far as I understand, there is no timesteps in your problem and the "reward" is immediate.
Thus, I don't think reinforcement is suitable here.

On an other hand, in supervised learning, linear regression is the task of approximating a mapping function (f) from input variables (X) to a continuous output variable (y).
It has much in common with your case. If I am not wrong, you are trying to approximate a function that maps image data to (x,y) coordinates
(the "sweet spot"). So, I think regression would be a better way to go for you.

You could either generate a dataset at first, with images data and associated (x,y) coordinates for only spots validated by your function F, and then train a regression predictive model. Or you could train your model with online learning, by generating batch of images and sweet spot coordinates at each step.

Concerning the point (2), it will highly depend on how is made your F function. Since overlapping spots cannot be sweet spots, the simplest would
be to make your F function return 0 for those spots.


Must all CNNs and RNNs not have a fully connected layer in order to be considered as such?

Not quite sure about RNN & LSTM (and it always depends on the task), but for CNN the answer is clearly no; CNN routinely include FC layers. Quoting from the highly popular (and recommended) Stanford course CS231n: Convolutional Neural Networks for Visual Recognition:

ConvNet Architectures
We have seen that Convolutional Networks are commonly made up of only
three layer types: CONV, POOL (we assume Max pool unless stated
otherwise) and FC (short for fully-connected).

You can easily verify that this is the case for practically all popular CNN models for computer vision in any relevant exposition (e.g. Illustrated: 10 CNN Architectures).
In fact, the all-convolutional NN (i.e. without FC layers) is considered a special case: see Striving for Simplicity: The All Convolutional Net.


Neural networks of arbitrary/general topology?

The simplistic neural networks that have been given away for free after they prove insufficient by themselves in field use consist solely of two orthogonal dimensions.

Layer width — the number of ordinal numbers or floating point numbers that represent the signal path through any given layer comprise of an array of layer elements
Network depth — the number of layers in the primary signal path, which is the number of activation function arrays or convolution kernels or whatever other elements Usually neural networks consist from layers

However, in large corporations that have AI pipelines, this is not the case.  We are beginning to see more interesting topologies in open source.  We see this in generative systems for images, text, and speech.  We see this in robotic control of robots.  The truth is that these more sophisticated topologies have been in play for years, but were just not appearing in the open source community because they were company confidential.  Enough academic work, releasing of portions of corporate IP, and the accumulation of independent OSS work has occurred to start to see these topologies in GIT repos.

Cyclic Not Acyclic

Artificial network topologies are generally cyclic, not acyclic in terms of their causality or their signal pathways, depending on how you depict them theoretically.  These are three basic examples from among dozens in the literature and in the open source repositories.

Back-propagation represents the introduction of a deliberate cycle in signal paths in a basic multilayer perceptron, making that topology a sequence of layers represented by vertices, connected sequentially by a set of directed edges representing forward propagation, and a set of directed edges in the reverse direction to distribute the corrective error determined at the network output according to the principle of gradient descent.  For efficiency, the corrective signal is distributed recursively backward through the layers to the $N - 1$ matrices of parameters attenuating the signals between $N$ layers.  Back propagation requires the formation of these $N - 1$ cycles for convergence to occur.
In a generative adversarial network (GAN), we have the signal path of each of the two networks feeding the training criteria of the other.  Such a topological arrangement is like negative feedback in a stable control system in that an equilibrium is formed between the generative network and discriminative network. The two directed edges, (a) the one that causally affects G with Ds result, and (b) the one that causally affects D with Gs result, create a cycle on top of the cycles in each of G and D.
In attention based networks being touted as theoretically advantageous over LSMT (which has been dominating over CNNs) has a much more complex topology and more cycles above those in supervisory layer than those in GANs.

Analysis of Answer One of Three

It is true that every directed graph can be realized in an arbitrarily large RNN because they are Turing complete, but that doesn't mean they are a great topology for all finite algorithms.

Turing was aware that his punched tape model was not the best general purpose, high speed computing architecture.  He was not intending to prove anything about computing speed but rather what could be computed.  His Turing machine had a trivial topology deliberately.  He wanted to illustrate his completeness theorem to others and resurrect the forward movement of rationalism after Gödel disturbed it with his two incompleteness theorems.

Similarly, John von Neumann proposed his computing architecture, with a central processing unit (CPU) and unified data and instruction bus, to reduce the number of relays or vacuum tubes, not to maximize parallel algorithm execution.  That topology as a directed graph has the instruction controller and the arithmetic unit in the center and everything else branching out from the data and address bus leading from them.

That a topology can accomplish a task is no longer a justification for persisting in the use of that topology, which is why Intel acquired Nirvana, which deviates from traditional von Neumann architecture, DSP architecture, and the current CUDA core architecture that NVidia GPUs use and offer for artificial network realization through C libraries that can be called via integrated Java and Python adapters.

There is definitely sense to seek for more general topologies, if they are fit for the purpose, just as with Turing's or von Neuman's.

Analysis of Answer Two of Three

General topologies exist, the most economically viable of which is the CUDA cores begun by NVidia, which can be configured for MLPs, CNNs, RNNs, and general 2D and 3D video processing.  They can be configured with or without cycles depending on the characteristics of the parallelism desired.

The realization of topologies unlike the Cartesian arrangements of activation functions in artificial networks or the kernel cells in convolution engines do have barriers to use, but they are not fundamental restrictions. The primary barrier is not one of hardware or software.  It is one of linguistics.  We don't think topologically because we don't talk topologically.  That's what is great about this question's challenge.

FORTRAN began to dominate over LISP during the time when general purpose programming began to emerge in many corporations.  That is not surprising because humans communicate in orthogonal ways.  It is cultural.  When a child scribbles, teachers are indoctrinated to say nice things but respond by drawing a shape.  If the child draws a square, the teacher smiles.  The child is given blocks.  The books are rectangular.  Text is justified into rectangles.

We can see this in building architecture dating back to Stonehenge.  Ninety degree angles are clearly dominant in artificial things, whereas nature doesn't seem to have that bias.

Although directed graphs were easy to implement and traverse in recursive structure and were commonplace in the LISP community.  FORTRAN with its realization of vectors and matrices in one and two dimensional arrays respectively were easier to grasp by those with less theoretical background in data structures.

The result is that even if learning EMMASCRIPT (JavaScript) which has its seed from the LISP community and is not biased toward orthogonal data structures, people tend to proceed from HelloWorld.js to something with a basic loop in it, with an underlying array through which the loop iterates.

There are three wonderfully inquisitive and insightful phrases in answer two of three.

Maybe learning is not converging in them — Interestingly an algorithm cannot learn without a cycle.  Directly applying a formula or converging using a known convergent series of terms does not qualify as learning.  Gradient descent relies entirely on the cyclical nature of a corrective action at the end of each sample processing or batch of them.
Maybe they generate chaotic [oscillations] — This gets into chaos theory and control theory's concept of stability.  They can do so, but so can a basic multilayer perceptron if the learning rate is set to high.
Maybe they generate bifurcations — Now we have fully entered the realm of chaos, which is arguably closely related to creativity.  Mendelbrot proposed the relationship between new forms of order and the apparent chaotic behavior arising from the appropriate level of feedback in a system with signal path components that cannot be modelled with a first degree equation.  Since then, we find that most phenomena in nature are actually strange attractors.  The plot of the training of a network from a continuous feed of consistently distributed data in phase space will reveal ... you guessed it ... a strange attractor.  When purtibations are deliberately injected into a training epoch from a pseudo-random number generator, the specific purpose is a bifurcation, so that the global optimization can be found when the training gets stuck in a local optimization.

Analysis of Answer Three of Three

General topologies exist and are promising and researchers are ready to work with them.  It is enthusiasts that can have a dismissive attitude.  They don't yet understand the demos they've downloaded and painstakenly tweaked to run on their computer, they're about to launch their AI carrier amidst the growing demand from all the media hype, and now someone is introducing something interesting and not yet implemented in code.  The motivational direction is generally to either dismiss or resist the creative proposals.

In this case, Google, CalTech, IBM, MIT, U Toronto, Intel, Tesla, Japan, and a thousand other governments, institutions, corporations, and open source contributors will solve that problem, provided people keep talking about topology and the restrictions inherent in purely Cartesian thinking.

Misunderstanding Topology to Mean Dimensionality or Topography

There has been some confusion in terms.  The SO reference in the question is an example of thinking that changing an array dimension is changing the topology.  If such were so, then there would be no change one could make to the geometry of an AI system that would not be topological.  Topology can only have meaning if there are features that are not topological.  When one draws a layer, they don't need to increase the height of the rectangle representing it if the number of activitations, the width of the layer, is changed from 100 to 120.

I've also seen academic papers that called the texture or roughness of an error surface its topology.  That completely undermines the concept of topology.  They meant to use the term topography.  Unfortunately neither the publisher nor the editor noticed the error.

Software or Tools

Most programming languages support directed graphs in recursive hashmaps.  LISP and its derivatives supported them at a more machine instruction efficient level, and that's still the case.  Object oriented databases and graph libraries exist and are in use.  Google uses them extensively in web indexing and lookup.  FaceBook's API is called the Graph API, because it is a query and insert API into the graph that is FaceBook's user data store.

The explosion is here in global software giants.  There is open source for it.  The revolution that is missing is among those who are not yet educated as to the meaning of topology, the difference between a hierarchy and a network or the role of feedback in any learning system.

Regarding Java and Python there are many barriers to the revolution in thinking, primarily these.

There are now key words in either Java or Python to directly deal with directed graphs other than the idea of a class with references to instances of other classes, which is quite limited.  None of these languages can add edge types with a single simple language construct.
There is no mapping to hardware yet, although Nirvana allegedly developed one, and Intel acquired Nirvana, so that barrier may evaporate soon.
The bias still exists in preschool, kindergarten, and first grade
Hilbert spaces are not generally taught in calculus

Graphviz and other graphing software that auto-generates diagrams from unconstrained directed or bidirectional graph representations have done much to bust through the barriers because the generated images are visible across the web.  It may be through visual representations of graphs that linguistic representations, thought, hardware, and software begin to emerge representing the paradigm shift the question investigates.

It is not that constraints are not useful.  Only some patterns and paradigms produce results, but since the results from the human brain demand attention, and the human brain is

Not at all orthogonal,
Not implemented using Cartesian neural patterns, and
Not topologically a box,

One can all but conclude that those are not particularly well chosen constraints.  Neither is the acyclic criteria.  Nature is cyclic, and intelligence probably requires it in many ways and at many levels.


How to label “other” while labeling image for object detection/classification?

From how you've phrased your question I'm going to assume you've jumped in without much structured training in data science, so I'll answer at a fairly high level.

This is an inherent problem with image classification in that if your final layer only has food classes then whatever you feed in will always be classified as a type of food regardless of what is actually in the image.

There are a few techniques you could try.  The simplest and fastest being using a pre-built classifier to screen the input data for the presence of "food" and then only using your own classifier to determine what type of food it is.  Any of the open source imageNet networks would be a good step here - if that finds something with a food label then use your classifier to identify the category.  This means you won't need to retrain although you'll be at the mercy of any errors in the pre-trained category classifier, which is outside of your control.  Here's a good how to guide: https://www.learnopencv.com/keras-tutorial-using-pre-trained-imagenet-models/

Another option is to add a negative class to your data set consisting of non-food items that you label "other".  This is probably trickier as you'd need to cover all the non-food categories that your network will see and ensure that it doesn't learn the background.

Take a look at your final level and make a call about whether to accept the top result or not. You're forcing a choice and these values may give a clue as to the level of error in the prediction.


Should I use single or double view for gender recognition?

I think your fundamental question is: can convolutional neural networks generalize to objects independent of the view?
The answer is mostly yes (given the dataset contains multiple views of objects). This is evident by looking at the results of various challenges: e.g. COCO object detection challenge results on youtube. You can see that no matter what the view is on the car, pedestrians, the detector is not biased towards any specific viewpoint.

Therefore, one can assume that you can build only one network to perform object detection and another network to perform classification.

If you really want to go even further:
- you can make a small change to the architecture of your detector (I guess you might be using something like SSD, YOLO or Faster-RCNN), in which you make gender classification for every bounding box prediction. If you think about it, it is intuitive because the detector is already doing classification (there is softmax + cross-entropy loss usually), you can just add another term in its tensor output and modify the loss. That way you don't even need another network! It would be much faster and simpler.
- you can predict the pose estimation of the object (and corresponding normals) with respect to the camera to capture the best viewpoint to perform classification.


Relative compute time for each type of layer in a neural network

The graph seems to show the relative compute time of each component of the learning process, independent of the layer, using the typical meaning of layer in the context of artificial networks.

The data is not unbelievable, in that one would expect a matrix-vector multiplication of an entire layer output would take more time than a matrix-vector multiplication of the size of a convolution kernel. Still, these representations are for a specific test condition and measurement methodology.

Most of the theoretical and practical limits of parallel processing are heavily biased by seventy years of sequential processing as the foundation of computer science study and practice. If an understanding of parallel processing is desired, the von Neumann architecture is not the place to look. At the time of that development, computers could either be made of wood, steel, vacuum tubes, or relays. Developing a computer with 65536 parallel data paths was unrealistic to the extreme. The invention of algorithms was a workaround to funnel all processing through a single CPU. We've been backing out of that mind set for the last thirty years, beginning with FPUs, GPUs, multiple buses, computer clusters, and multiple cores. More parallelism in VLSI form is coming, and soon.

The demonstration of parallel processing comes, as many things do, from biology. Look at farming and grain growth under the sun. Look at visual circuitry coming from the eyes of thousands of species. Consider the orbits of millions of planets and their satellites. The artificial parallelism to examine is the transportation systems and the Internet. Parallel processing is efficient when the process was designed as a parallel process rather than trying to make parallel that which was designed as serial. For instance, the human ear does the physical equivalent of a real time windowing FFT without performing a single multiplication operation.

The bottleneck of the floating point multiplication of an attenuation matrix with a vector (parameterizing the mix of signals from one layer to the next) is a key to why NVidia has done well with re-purposing rendering hardware for artificial network training. It is also one of the many reasons this question is interesting: If digital values are mere estimates, why not return to analog for AI?.

When surfing for GFLOPs vs compute time for different types of layers across GPUs and CPUs, using the term 'operation' instead of 'layer' in the search terms may help. Even though elementary operations are lower level than that of the comparison desired, the term layer is delegated strongly to the higher level meaning in the machine learning literature. Reliable data in this space is thin because much of the speed optimization has already been researched and developed, so whatever the visualization shows is useless unless you work on VLSI development at Intel, Samsung, NVidia, Qualcom, TI, or one of the other silicon giants.