If the AI goal is the protection of humans, will it always pursue this goal?

If the main goal of AI (which I assume you mean an AGI) is to protect humans and AI will be effective, then AI will always attempt to pursue its main goal (otherwise the assumption of its effectiveness does not hold), even at the expense of other less important goals that it might have. However, if the destruction of a human (or a group of humans) protected or avoided the destruction of other humans, then AI would face a dilemma. In that case, I think it is hard to predict the actions of the AI. Will it act rationally or irrationally? What would it mean for the AI to act rationally? Which parameters will it take into account? Only the number of deaths, or will take into account the future and weight the importance of the lives? How will it define the importance of a human life?


Any problems/games/puzzles in which exhaustive search cannot show that a solution does not exist?

This should be a comment, but I don't have enough reputation to comment. I will remove this answer if question is updated

Your question is not really clear. As I understand it, the definition itself of the exhaustive search show that it's always possible to determine if a solution is valid or not.

Exhaustive search is defined as :

For a given potential solution, can determine if this solution resolve the problem.
Test every possible potential solution

From this, there is no problem where every potential solution is tested, but the search cannot show that a solution exist : either it exist and the search found the candidate, or it does not exist because all possible candidates were tested.

Maybe what you meant when asking your question is : "Any problems in which exhaustive search cannot be applied ?"

And the answer is yes, there is plenty of problems where the search space is way to big to be integrally searched : for example the Rubik's cube 3*3*3 has 43 252 003 274 489 856 000 combinations (source).

My answer need more source, specifically about the exhaustive search definition. I would be happy to add it if you could share :)


Which local minima to choose according to the shape of the error surface?

Just sharing my thoughts on this:

A local minimum in the loss function is just a synthetic measure expressing how well the model is performing on the target dataset (I'm assuming it's the test set)
Letting aside the absolute values, you could also consider the steepness of that region as follows:

the first LM is at the bottom of steep region which means the related parametrization is quite unstable as if you change it slightly the performance drop quickly (loss increases quickly) so you could interpret this
as a parametrization with not great generalization capability (it is not robust against small changes): always remember that what you are actually looking for is model able to generalize well when in production (i.e. after you have deployed it so when training, validation, test is finished) and test set is typically a small proxy for the data observed in production (but it really depends on the final application itself)
the second LM is related to a much less steep region: even if you change the parametrization slightly you do not observe a big performance change which could be interpreted as a more stable parametrization hence with better possibility to generalize on unseen data


Is tabular Q-learning considered interpretable?

There is not a widely accepted definition of explainable AI (XAI). However, as a rule of thumb (my rule of thumb), if you can't explain it easily to a layperson (or even an expert), then the model or algorithm is not (very) interpretable. There are other concepts related to XAI, such as accountability (who is responsible for what?), transparency and fairness.

For example, the final decision of (trained) decision tree can easily be explained to (almost) any person, so a (trained) decision tree is a relatively interpretable model. See the chapter 4.4. Decision Tree of the book Interpretable Machine Learning: A Guide for Making Black Box Models Explainable.

An artificial neural network (ANN) is usually considered not very interpretable because, unless you attempt to understand which parts of the network contribute to the output of the ANN (for example, with the technique layer-wise relevance propagation), then you cannot immediately or easily understand the output or decision of the ANN, given that an ANN involves many non-linear functions, which produce unintuitive outcomes. In other words, it is more difficult to attribute the contributions of each unit of an ANN to the output of the same ANN than to explain e.g. the decision of a decision tree.

In the context of deep reinforcement learning (DRL), the ANN is used to approximate the value or policy functions. This approximation is, in the first place, the main reason behind the low interpretability of deep RL models.

Q-learning is an algorithm, so it is not a model, like an ANN. Q-learning is used to learn a state-action value function, denoted with $Q: S \times A \rightarrow \mathbb{R}$, which can then be used to derive another function, the policy, which can then be used to take actions. In a way, Q-learning is similar to gradient descent, because both are machine learning (or optimization) algorithms. The $Q$ function is a model of the environment, given that, for each state, it represents the expected amount of reward that can be obtained, so, in a certain way, the learned $Q$ function represents a prediction of reward.

Is the learned tabular $Q$ function interpretable? Yes, it is relatively interpretable, but how much? What kind of interpretation do you really need? It depends on the context and people that need the interpretation or explanation. A reinforcement learning researcher will usually be satisfied with the usual explanation of the inner workings of $Q$-learning, Markov decision processes, etc., because the usual RL researcher is not concerned with the really important problems that involve the life of people and other beings. However, for example, in the context of healthcare, doctors might not just be interested in the explanation "expected maximum future reward", but they might also be interested in the environment, the credit assignment problem, the meaning and effectiveness of the reward function with respect to the actual problem that needs to be solved, in a probabilistic interpretation of the results (rather than just a mere action that needs to be taken), possible alternative good actions, etc.

Recently, there have been some attempts to make RL and, in particular, deep RL more interpretable and explainable. In the paper Programmatically Interpretable Reinforcement Learning (2019), Verma et al. propose a more interpretable (than deep RL) RL framework that is based on the idea of learning policies that are represented in a human-readable language. In the paper InfoRL: Interpretable Reinforcement Learning using Information Maximization (2019), the authors focus on learning multiple ways of solving the same task and they claim that their approach provides more interpretability. In the paper Toward Interpretable Deep Reinforcement Learning with Linear Model U-Trees (2018), the authors also claim that their approach facilitates understanding the network's learned knowledge by analyzing feature influence, extracting rules, and highlighting the super-pixels in image inputs.

To conclude, deep RL should not necessarily be avoided: it depends on the context (e.g., it is usually perfectly fine to use deep RL to solve video games). However, in cases where liability is an issue, then deep RL should also be explainable or more explainable alternatives should also be taken into account.


Classification with deeplearning : clean start vs continue training

If the task involves only apples, orange and peaches, you should use method 1. As the number of classes is small, the network cannot generalize well to all classes. As a side note, you should start with the pretrained weights of YOLO v3 as some classes of YOLO v3 may be fruits, which can help your model converge faster.

If the number of classes is large, for example a hundred different fruits, you should use method 2.2 . The model should be able to generalize to all fruits and can converage faster as many fruits look the same. This is the case of transfer learning. In original YOLO v3 training, image net weights for dark net is used for the backbone network. It accelerates the training of YOLO v3.

For 2.1, it will not work as gradient descent will not consider the trained weight. The trained weights will be over written by the weights for peaches.

For a recommended method, it depends on your class size. If the model will continue to add more classes, you should perhaps use 2.2 but only start using 2.2 when you have a considerable amount of classes. Hope I can help you.


GANs: Should Generator update weights when Discriminator says false continuously

In general, you should train both discriminator D and generator G simultaneously.

Depending on the metric that you use as the target for your model, you may encounter a Vanishing gradient problem. It can happen when you implement original loss (i.e. JS-divergence). In that case D can become overconfident regarding fake samples and won't provide any useful feedback to the G. To find out if training fell into this problem, you should plot D and G loss. It will look as follows:

The original GAN has a lot of problems, that's why I suggest you to use Wasserstein metric. More information you can find in WGAN paper

Here you can find more information about GAN problems:

https://arxiv.org/pdf/1904.08994.pdf
https://arxiv.org/pdf/1804.00140.pdf


Why is this PyTorch implementation of the actor-critic algorithm inconsistent with the mathematical formulas?

There is no 'regular' formula for calculating policy loss, the regular thing when calculating policy gradient is to multiply gradient with advantage function which can be many things. Look at section 2 of this paper for coverage on basic advantage functions. Also, the expected discounted reward is the same thing as the state-action value function (Q value).
$$Q^\pi(s_t, a_t) = \mathbb{E}_{s_{t+1:\infty}, a_{t+1:\infty}}[\sum_{l=0}^\infty \gamma^lr_{t+l}]$$
So, the variations you posted roughly calculate the same thing.
Regarding the negative sign, in policy gradient methods, we want to maximize our performance function which has the following form:
$$J(\theta) = \sum_\limits{\substack{a}} \pi(a \mid s, \theta)A(s, a)$$
So, the higher our performance the better it is. When people write code, they use minimizers that minimize the objective function, but, in this case, we want to maximize it, so maximizing the objective function is the same thing as minimizing the negative objective function therefore the negative sign.


Maximum number of neurons in a layer given number of neurons in previous layer

The Universal Approximation Theorem states: In the mathematical theory of artificial neural networks, the universal
  approximation theorem states that a feed-forward network with a single
  hidden layer containing a finite number of neurons can approximate
  continuous functions on compact subsets of Rn, under mild assumptions
  on the activation function. The theorem thus states that simple neural
  networks can represent a wide variety of interesting functions when
  given appropriate parameters; however, it does not touch upon the
  algorithmic learnability of those parameters.

So clearly if you do not have infinite number of hidden neurons you cannot approximate a function with $\epsilon$ error tending to 0. So if we assume that the previous layer gives some sort of representation of the function to be approximated you still need infinite hidden neurons to approximate this representative function.

But probably it'll be more clear to you if we go into basic physical laws of mathematical approximation.

  According to the Fourier Theorem by Joseph Fourier: A Fourier series is a way to
  represent a function as the sum of simple sine waves. More formally,
  it decomposes any periodic function or periodic signal into the
  weighted sum of a (possibly infinite) set of simple oscillating
  functions, namely sines and cosines (or, equivalently, complex
  exponentials).

So theoretically, if your function is periodic it maybe perfectly approximated by an activation function like $sin$ or $cos$. The simple reason you cannot use an activation like $ReLu$ or $Sigmoid$ is because they are non-periodic and as per another related theorem called Fourier Transform you would require infinite number of sine waves to approximate non-periodic curves so the problem collapses in itself.

Since there are various ways to train a Neural Network other than Gradient Descent theoretically speaking you just might approximate a periodic function with some other methods (if activation is sine or cos), it is almost imposiible via methods like Gradient Descent.

So in short the answer to your question will be $\infty$ for the aforementioned reasons.


Detect visual attention area in an image

You can search for the following paper titles:

A Deep Multi-Level Network for Saliency Prediction.
Beyond Universal Saliency: Personalized Saliency Prediction with Multi-task CNN.

You can code in python using Pytorch framework.


Multi class text classification with imbalanced data

It is straightforward to combine k-fold cross-validation with a technique like oversampling or undersampling.

First, apply the balance-restoration technique to your training data. Then parametrize a model using k-fold cross-validation on the re-balanced training data. In Scikit learn, I believe you can even bundle these actions together into a single 'pipeline' object to make it easier to manipulate.

Precision/recall is probably a fine starting place for measuring performance.


How can I classify relevant and irrelevant images from the Database?

The classification of images as relevant or irrelevant can be done using LSTM, the design of which is perhaps best explained without the mathematical detail of the original paper by Eugene Kang's Long Short-Term Memory: Concept text and diagrams.

That the data is unstructured is not an issue if the images can be extracted and placed into raw pixel matrices with appropriate indices into  the database so the results of classification can be inserted into the database in a way where the image and the classification are associated.

The labels from a training point of view are not the indices into the database but the single bit indication of relevance.

The LSTM network must be trained on a sufficient sample of labeled example data before it will be usable to classify the images.  That requirement may further require a significant amount of manual classification unless some form of delegation, hints from associated features in the database, or crowd sourcing can be used to generate labels.


Does the performance of a model increase if dropout is disabled at evaluation time?

Dropout is a technique that helps to avoid overfitting during training. That is, one can use dropout only for training.

  units may change in a way that they fix up the mistakes of the other
  units. This may lead to complex co-adaptations. This, in turn, leads to
  overfitting because these co-adaptations do not generalize to unseen
  data.

If you want to evaluate your model, you should turn off all dropout layers. For example, PyTorch's model.eval() does this work.

More about dropout:

Improving neural networks by preventing co-adaptation of feature detectors
Dropout: A Simple Way to Prevent Neural Networks from Overfitting


Why am I getting better performance with Thompson sampling than with UCB or $\epsilon$-greedy in a multi-armed bandit problem?

The first thing to note here is that your results seem aligned with the results commonly found in the bandit literature.
Second thing to note would be that the performance of bandit algorithms is usually measured in terms of regret. This is the difference between (i) the amount of rewards accumulated by an oracle policy having prior knowledge about the true rewards of the bandit arms and (ii) the amount of rewards accumulated by the bandit strategy you are evaluating. In other words, this measures the loss your incur due to not knowing what the optimal arm is.
Ideally, the regret should be logarithmic. In theory, all three algorithms achieve logarithmic regret.
See [1] for the regret of decaying $\epsilon$-greedy and UCB and [2] for the regret of Thompson sampling.
Despite their similar regret bounds, these algorithms may behave differently in practice.
For example, according to experiments in [3]:

It appears that Thompson
sampling is more robust than UCB when the delay is long. Thompson sampling alleviates the
influence of delayed feedback$^*$ by randomizing over actions; on the other hand, UCB is deterministic
and suffers a larger regret in case of a sub-optimal choice.

For decaying $\epsilon$-greedy, the fact that it obtained the lowest rewards may be due to the fact that the action selected for exploration is chosen uniformly at random (even if it has a bad reward estimate). In addition, it will explore actions even if they have already been selected many times, unlike UCB which considers the number of action selections to refine the confidence bound.
That said, besides these aspects, there may be other factors that could influence the results obtained by a given bandit algorithm. These may include:

The reward distributions you chose to model your actions, and their
characteristics.
The number of actions you have, and how far each
action’s reward is from the optimal action’s reward.

$^*$ Delayed feedback occurs when the outcome of an action selection is not revealed immediately but after some time delay.


Formula for expected rewards for state–action–next-state triples as a three-argument function

Expectation of reward after taking action $a$ in state $s$ and ending up in state $s'$ would simply be
\begin{equation}
r(s, a, s') = \sum_{r \in R} r \cdot p(r|s, a, s')
\end{equation}
The problem with this is that they do not define probability distribution for rewards separately, they use joint distribution $p(s', r|s, a)$ which represents probability for ending up in state $s'$ with reward $r$ after taking action $a$ in state $s$. This probability can be separated in 2 parts using product rule
\begin{equation}
p(s', r|s, a) = p(s'|s, a)\cdot p(r|s', s, a)
\end{equation}
which represent probability for getting to state $s'$ from $(s, a)$ and then probability for getting reward $r$ after ending up in $s'$.
If we define reward expectation through joint distribution we would have
\begin{align}
r(s, a, s') &= \sum_{r \in R} r \cdot p(s', r|s, a)\\
&= \sum_{r \in R} r \cdot p(s'|s, a) \cdot p(r|s', s, a)
\end{align}
but this would not be correct since we have this extra $p(s'|s, a)$ so we divide everything by it to get expression with only $p(r|s', s, a)$. So, in the end we have

\begin{equation}
r(s, a, s') = \sum_{r \in R} r \frac{p(r, s'|s, a)}{p(s'|s, a)}
\end{equation}


Using GAN's to generate dataset for CNN training

I think you'll enjoy this work from Apple on improving the realism of synthetic images.  Essentially what you need to do is generate a synthetic image then have your GAN modify the synthetic image so that a 1) a discriminator thinks it is real while also 2) not changing the gross structure of the image very much (so the traffic sign doesn't move) - yes, this loss function is going to take a little work!

Making synthetic data realistic enough to allow models to generalize successfully in the real world is a very active and exciting area of research, not least with respect to robotics, and so the work you are doing now should make you very attractive indeed to the right employer.


Which is more important, doubt or reinforcement?

The single-sentence answer to this would be: it depends.

The core of this question seems to be very closely related to the well-known trade-off between exploration (similar to how you describe "doubt") and exploitation (similar to how you describe "reinforcement"). It is almost never the case that someone declares one of those two to be more important than the other, and only tries to pursue the "most important one". There is no single axis that measures "importance", no single line of numbers such that we can place "doubt" on one point, "reinforcement" on the other, and declare that the biggest number is the most important one. We almost always want a balance between exploration and exploitation. They are, almost always, both important.

Now, in some extreme cases, only one of the two may be important. For example:

If you have an environment where you can generate experience and evaluate policies completely free of any costs, you'll want to prioritize exploration / doubt. This is almost never the case though, you'll realistically always have at least time as a cost.
If you have a situation where you care very much about your performance right now, there's no point in doubting it too much or exploring too much. Consider, for example, DeepMind's AlphaGo team a few hours before their match against world-class human player Lee Sedol in 2016. At such a point in time, I highly doubt they'll be interested in exploring wildly different sets of parameters from the ones they have found during training so far; they won't have time anymore to thoroughly evaluate them, they'll want to stick to what they have, which they know works fairly well.

Also note that sometimes, you need to stick to something that already works well somewhere if you want to be able to realistically learn something new somewhere else:

In a large Markov Decision Process (e.g. Montezuma's Revenge), once you've already learned a good policy near the initial state, you don't want to explore too much anymore around that initial state because you won't reach interesting new states. You need to exploit for a while first such that you actually reach new states where it becomes interesting to explore again.
A similar situation, but now more closely related, using your terminology of "doubt" and viewing the space of all possible sets of values for all parameters of a large Neural Network as the space that we're searching in; suppose that we're learning a policy for an Atari game, with pixels as input. We first have a few Convolutional layers, then a few Relu's, etc., the standard setup. Intuitively, we expect the first few layers to "learn" how to "understand" the images, and transform the raw pixels to encodings that can be used in an efficient manner for the last few layers to compute a good policy / value estimates. If the first few layers already do a good job at transforming the raw pixel inputs to something more useful, we generally don't want to "doubt" those layers too much anymore, we don't suddenly want to jump to a completely different set of parameters anymore. If we suddenly completely change the parameters in those layers, we'll know for sure that the later layers will also be completely messed up. We'll simply have created a significantly more difficult learning problem for ourselves when it comes to optimizing the last few layers.

Some other notes:

There are lots of references to "networks" in the question, for example the quote below:

  We hear much about reinforcement, which is, in my opinion a poor choice of a term to describe a type of artificial network that continues to acquire or improve its behavioral information in natura (during operations in the field).

Note that Reinforcement (learning) does not necessarily have to involve any kinds of artificial (neural or otherwise) networks at all. There's also tabular RL, and RL with function approximation using non-network function approximators (e.g. linear).

Under the "What shows more intelligence?" header, you write:

  The ability to reinforce what is already learned

That is not an accurate description of what was previously described as

  Adjustments to what is considered the optimum, beginning with the optimum found during training prior to deployment

which, in turn, I suppose is intended to describe the kinds of updates that are typically performed using variants of the Bellman optimality equation (value-based methods) or variants of REINFORCE (policy gradient methods). These methods do not just "reinforce what is already learned". They can lead to learning completely new policies from completely new experiences.

This idea:

  Perhaps all neural nets should have one or more parallel doubting networks that can test remote areas of the search space for more promising optima. In a parallel computing environment, this might be a matter of provisioning and not significantly reduce the throughput of the primary network, yet provide a layer of reliability not found without the doubtful parallel networks.

sounds a lot to me like using Evolutionary Search to optimize the parameter of a network. Some interesting blog posts from Uber (with references to their papers) on such approaches may be:

Welcoming the Era of Deep Neuroevolution
Accelerating Deep Neuroevolution: Train Atari in Hours on a Single Personal Computer

I didn't get around to reading them in detail yet, so I don't know exactly how similar they are, but certainly related.


How can I calculate the shortest path between two 2d vector points in an environment with obstacles?

The usual way to solve this kind of problem is to construct a configuration space: extruding all the polygonal obstacles by sliding the polygon corresponding to the robot around them (some slides).

The exterior vertices of the configuration space can then be used as input to a path-planning algorithm, such as A*.


How to define machine learning to cover clustering, classification, and regression?

I report three definitions of machine learning (ML) and I also explain that ML can be divided into multiple sub-tasks or sub-categories in this answer. However, it may not always be clear why classification, regression, or clustering can be considered machine learning tasks or can be solved with ML algorithms/programs, so let me explain why these tasks can be solved with ML, based on Tom Mitchell's definition of an ML algorithm that I report below for completeness.

A computer program is said to learn from experience $E$ with respect to some class of tasks $T$ and performance measure $P$, if its performance at tasks in $T$, as measured by $P$, improves with experience $E$.

So, according to this definition, for a computer program to be a "machine learner", we need to identify $E$, $T$, and $P$, and show that the computer program improves with $E$ at performing the task(s) $T$, according to $P$.
For the case of classification or regression (these are the tasks, which is just a synonym for problems), let's suppose that we are given the training labelled dataset $D = \{(x_1, y_i), \dots, (x_N, y_N) \}$ of $N$ tuples $(x_i, y_i)$, where $x_i$ are the inputs to the program (or model) and $y_i$ (the label, aka class, hence the name classification) is the output that the program should produce. In the case of classification, $y_i$ is an element of a discrete set, e.g. $\{0, 1\}$ in the case of binary classification, while, in the case of regression, $y_i \in \mathbb{R}$ is a real number.
So, in this case, the experience $E$ is $D$ (the training labeled dataset). The task $T$ is classification or regression, depending on whether $y_i$ belongs to a discrete or continuous space. The performance measure $P$ can be e.g. the cross-entropy (which is typically used to solve the classification task) or the mean squared error (which is typically used to solve the regression task). (I will not recall the definitions of these performance measures here: you can take a book on ML for the details).
So, we have identified $E$, $T$ and $P$. Now, we also need to argue why a program's performance measured by $P$ would improve with $E$. Let's suppose that you are using a neural network trained with gradient descent to solve these tasks. After one epoch/iteration of gradient descent, the loss will be smaller, so the performance of the program will be higher. So, this would indeed be a machine learner, according to Mitchell's definition of ML.
In the case of clustering (the task $T$), the only difference is that the dataset $D$ (i.e. the experience $E$) is unlabelled, i.e. we do not have labels, but we are given only the inputs, and the goal, as you know, is to group these inputs based on their similarity, according to some notion of similarity, which is your performance measure $P$.
The other definitions of ML that I report in the other answer are also consistent with Mitchell's definition. More precisely, most of these definitions are based on the idea that an ML algorithm is an algorithm that "finds patterns in data". To solve the classification, regression and clustering tasks/problems, an ML algorithm/program needs to find patterns in the data (either explicitly, like in the case of clustering, or indirectly, like in the case of classification), in order for the program's performance to improve.
Moreover, given that you brought this up in the now-deleted comments, I don't think that the definition of machine learning necessarily implies prediction (i.e. the use of a model to forecast something about future data), but, yes, ML is often used for prediction.
It's also important to note that, when someone says "machine learning" without further details or information (like you did in your post/question), one may refer to

the field/area of study that studies and applies ML algorithms, or
an ML algorithm (and sometimes model).

Mitchell's definition is the definition of what it means for an algorithm/program to be called an ML algorithm/program. However, my definition of ML as a field immediately follows from the definition of an ML algorithm.
Finally, we could say that a problem that can be formulated in such a way that an ML algorithm can be applied is an ML problem. Note that Mitchell's definition does not mention "ML problem" anywhere, but just "problem", so this is not a circular definition. So, we could say that classification, regression, and clustering are "ML problems" because they can be solved with ML algorithms. More generally, all these three problems can be thought of as "function approximation" problems, i.e., in all these cases, the solutions are functions.


What is the current research in artificial intelligence in the field of data compression?

It is already combined.

Adaptive entropy techniques are already used in most of the best compression encoders.  This is true for file encoders, video encoders, and audio encoders.  We use it in the solar lab to optimize sample rates in data acquisition.

In fact, pattern recognition and compression are very tightly coupled if you consider autoencoders and other feature extraction schemes and compare them mathematically with what compression does.  See Data Compression – A Generic Principle of Pattern Recognition?, Gunther Heidemann, Helge Ritter, VISIGRAPP 2008

Zlib and lz4 have more like hyper-parametric learning, however they don't persist what they learn.  This work is interesting: Adaptive On-the-Fly Compression, Chandra Krintz, Sezgin Sucu, IEEE Parallel and Distributed Systems, v17 n1, January 2006.

**Suggested Project:

Create a theoretical framework and software POC that learns correlations between these two sets.

Quickly ascertainable features of documents or audio or video streams (i.e file path components, media titles, date, file type, and first N bytes)
The existing parameters that existing open source compression software learns during its existing pattern recognition algorithms

Persisting those correlations between compression invocations may considerably improve file transfer, kernel operations (since lz4 is now native in kernels like LINUX), and media streaming.

How much effort is made to persist features extracted (pattern recognition) between frames in media streaming is worth investigating too.


Do we need multiple parallel environments to train in batches an on-policy algorithm?

We don't need multiple environments.
On-policy algorithms require that new training samples are collected with the newest policy, so we can't use an experience buffer. However we can use the newest policy to collect multiple samples, even over multiple epochs, before updating the weights. This update can be a batch update.


Look at Google's Open Image Dataset @ https://storage.googleapis.com/openimages/web/index.html

They provide image-level labels, object bounding boxes, object segmentation masks, and visual relationships.

Here is the link for the traffic signs dataset.


What are some applications where tree models perform better than neural networks?

Hard to say in general. Speaking from my own experience and by looking at which models win Kaggle competitions (see here and here), I would say tree-based models e.g. Random Forests, Decision Trees, Gradient Boosting are favorable over neural networks when working with low-dimensional data and easy interpretable features (usually simple tabular data with numeric, ordinal or categorical features).
Whereas when working with everything high-dimensional like images, text, time series or other data with non-trivial features, I would recommend neural networks.
Of course there might be exceptions and the future may prove me wrong.


Can I filter barking sounds on the television?

OK, here is one approach.

Acquire a data set of 'clean' audio samples without barking dogs and an data set of barking dogs sounds.
Generate a training set by mixing random selections of clean audio with a random selections of barking dog noises at various volume levels. This is your input data. Your output data is the clean audio before you added the barking.
Find a suitable neural network. This might be a good starting point: https://github.com/andabi/music-source-separation
Train the network.
Apply the network to a load of the clean+bark mixes and check the result by ear. If the results are not good enough, go to 3 to tweak the network. If the results are still poor go to 2 and generate more training data, perhaps adding some distortions or other effects the the barking. Happy with the result? Go to 6.
Publish the results online for the betterment of humanity. (And/or write the network to  a miniature GPU board (such as the NVIDIA Jetson https://developer.nvidia.com/buy-jetson) and work out how to wire that up to the decoded audio in your telly before it heads to the speakers), then blog about it because that would be a great project.


Should I prefer the model with the lowest validation loss or the highest validation accuracy to deploy?

Okay, I think it's better if we distinguish loss and accuracy first via Jeremy's answer, and I agree with him with the sentence "low or huge loss is a subjective metric".
The loss value is easy to affect by noise from data and significant increase with a few error data points. My advice in this case is to use more evaluation metrics, and understand correctly what you need from your model.
For example, with Cifar 10, and you need the more correct label the better, you can believe in accuracy. However, if you want your model to make sure its result is the correct, area under receiver operating characteristic curve (AUROC) maybe the better choice.
For example, classification problem with 3 class, correct label y = 1:

Good accuracy, bad AUROC: the output probability from softmax [0.3,0.4,0.3]
Good accuracy, good AUROC: [0.1,0.8,0.1]

And with imbalanced dataset, Precision, Recall and F1-score will be more suitable.


Appropriate algorithm for RL problem with sparse rewards, continuous actions and significant stochasticity

(1) You might want look into RND (Random network distillation) which allows usage of a curiosity-based exploration bonus for the agent as an intrinsic reward. You can use the intrinsic reward to complement the sparse extrinsic reward return by the environment.

The general idea is to have a randomly initialized fixed target network which encodes the next state & a predictor network is trained to predict the output of the target network. The prediction error is used to "quantify the novelty of new experience". Stronger novelty will be a good indication for the agent that it maybe worthwhile to explore more.

The authors of this (A) paper were able to achieve SOTA performance in Montezuma's Revenge, which is notorious for it's sparse reward.

In appendix A.1, It is mentioned that: "An exploration bonus can be used with any RL algorithm by modifying the rewards used to train the
model (i.e., rt = it + et)." It is also mentioned that the authors combined this exploration bonus with PPO (which also works in continuous action space). In A.2, a pseudo code is provided.

I would also recommend this (B) paper (see section 3) if you're interested in exploring the available Bonus-Based Exploration Methods out there which may help in tackling hard exploration games with sparse rewards.

With regards to high stochasticity & variance, I found an interesting remark (on page 3, under Figure 2) in this (C) paper:

"our investigation of DDPG on different network configurations shows that for the Hopper environment, DDPG is quite unstable no matter the network architecture. This can be attributed partially to the high variance of DDPG itself, but also to the increased stochasticity of the Hopper
task."

The remark was made in the context where the authors were trying to "tune DDPG to reproduce results from other works even when using their reported hyper-parameter settings".

Have a look here for a different benchmark on how DDPG fair against other algorithms.

(2) From the information provided, I can't conclusively provide you a quantitative assessment on DDPG performance for your specific problem. However, I would recommend the following:

(a) I will encourage you to try different RL algorithms when face with a difficult problem so that you can benchmark & find out which is more suitable. Also in (A), the authors mentioned, "PPO is a policy gradient method that we have found to require little tuning for good performance."

(b) Try different sets of hyperparameters. There are many ways to tune them systematically but discussion about this will be out of scope for this question.


Should I model a problem with quantised output as classification or regression?

So this is considered Ordinal Regression. There are many ways to model this type of data, generally in some form of regression setting. I do not recommend the softmax route because as you mentioned, there exists prebuilt correlation to the outputs.

Some common ways to approach this
(Note that im assuming your looking for methodologies that can be optimized through gradient techniques because of the way you formulated your question)

Treat it as a normal regressor where you clip the output to your range, and then define thresholds arbitrarily, such as $.36 \rightarrow .4$, $.34 \rightarrow .3$, etc..
Use a bounded activation function then scale (if your outputs are [0,1,.1], you could use sigmoid and just scale by a factor of 1, but if [0,10,1], you could use sigmoid and scale by 10. Once again youll need to create arbitrary threshold between each 2 points for inference (this can also be incorporated into your loss)
Using the above two methods, but learn the threshold between each 2 ordered points. Have an output layer that learns the ideal threshold for inference

And if you didnt want to use neural netowrks, each of these approaches have a bayesian analog that work as well!


Why is creating an AI that can code a hard task?

AI has been applied to programming (check out TabNine, my favorite autocomplete engine) although not in as robust a fashion as you describe.

Programming requires a high level of abstract while AI is typically trained to solve a very specific task. Given thousands of examples of insert sort in Python I think a model could be trained (perhaps after autocomplete and syntax correction) figure it out. However at this point the field has not developed a more general intelligence that can apply the ideas of the algorithm to other problems.

Addition based on comments:

Big picture, training an algorithm to solve a general class of problems (say, web dev) requires a huge number of examples or an immense number of trials. Further, as the complexity of the problem grows the number of parameters necessary to build the model grows. Writing code is a very complex problem and would thus require a huge amount of data and a huge number of parameters making it totally infeasible with today's math and (because of how the math is solved) hardware.

Modern AI is has a very simple goal: find the model that solves a problem optimally. If we could quickly search every possible model this would be simple. Fields like machine (deep) learning and reinforcement learning are concerned with finding a good solution in a reasonable amount of time. At this point no such solution exists for a problem of such complexity.


Can CNN autoencoders be improved by treating the output layer as an inverted hidden layer?

It seems like what you need is reinforcement learning method rather than AE / unsupervised. Method has to be like
Input -> CNN Feature Extractor (encode side of AE, i.e. VGG) -> Classifier (i.e. DNN) -> Choose one of the actions -> Observe change in the game and check if it was a wrong (like character is dying) or good (character is moving forward) and give a rating to action (feedback of decision).

After a while, AI will learn how to jump over obstacles, avoid stuff etc.


Understanding Bayesian Optimisation graph

The original graph for the aforementioned Bayesian Optimisation is similar to the graph in these slides (slide 18) along with the calculations.

So, according to the tutorial the graph shown should actually have the term $p(D|m)$ on the y-axis, thus making it a generative model.Now the graph starts to make sense, since a model with low complexity cannot produce very complex datasets and will be centred around 0, while very complex models can produce richer datasets which makes them assign probability thinly over all the datatsets (to keep $\sum_{D'}p(D'|m) = 1$).


Why does Google Translate produce two different translations for the same Chinese text?

It's not quite clear what you are asking. So I'll answer in separate parts.

  Why is the translation different from the official title?

It could be simply because machine translation is not perfect, or our human translator took some creative liberties when translating. In this case it seems to be both.

Note that 龍爭虎鬥 properly translated doesn't mean either "Dragons fight" or "Enter the dragon". Literally translated, it means "dragon compete tiger fight". It belongs a family of well-formed idioms called "Chengyu", which describes a situation where there is fierce fighting or competition.

So you can see that neither translations fit.

  Why does Google Translate give me different translations on the same phrase?

Context matters when we read! So translating a phrase in isolation doesn't guarantee that the same phrase has the same meaning/translation in all other parts of the text.

For example, green is a color, but the word "green" can also be used as like: "Alice is green with envy" or "Bob has green thumbs", of which neither instances of the word "green" refers to the color.

Considering the technical side of things, Google translate probably uses some kind of RNN in its pipeline. RNN's are influenced by past states, meaning that what it outputs now as a function of what it reads in is dependent on the RNN's past state. Which is similar to the issue addressed above.


Could an AI be killed in an infinite loop?

This is a hard to answer question because a truly correct answer would involve static analysis of a given Intelligence to determine whether it has the computational capability to generate a looping state (e.g. some state which reproduces itself in the next instance), and in fact whether these looping states can even exist in the given architecture.

Generally speaking, assuming we can build AI, there is no reason it would be impossible build an AI that has the potential to get stuck in a computational loop and become non-functional. You can imagine a recurrent neural network where the continuous state inside the ANN dominates its behavior causing it to reproduce states with small variety (as input will always be relevant in a connected ANN) but are converged around a particular state space. Whether that is akin to death is a philosophical question for your own peace of mind.

However, if we designed an AI agent that commonly did this, that would reflect poor design; that wouldn't necessarily devalue the contribution of the designed agent but it would bring the agent's other activities under scrutiny. If this is a bi-product of some simulated intelligence, is the process going on really that intelligent?


State space search is a general and ubiquitous AI activity that includes numerical optimization (e.g. via gradient descent in a real-valued search space) as a special case.

State space search is an abstraction which can be customized for a particular problem via three ingredients:

Some representation for candidate solutions to the problem (e.g. permutation of cities to represent a Travelling Salesman Problem (TSP) tour, vector of real values for numeric problems).
A solution quality measure: i.e. some means of deciding which of two solutions is the better. This is typically achieved (for single-objective problems) by having via some integer or real-valued function of a solution (e.g. total distance travelled for a TSP tour).
Some means of moving around in the space of possible solutions, in a heuristically-informed manner. Derivatives can be used if available, or else (e.g. for black-box problems or discrete solution representations) the kind of mutation or crossover methods favoured by genetic algorithms/evolutionary computation can be employed.

The first couple of chapters of the freely available "Essentials of Metaheuristics" give an excellent overview and  Michalewicz and Fogel's "How to Solve It - Modern Heuristics" explains in more detail how numerical optimization can be considered in terms of state-space.

How shall the "search through possible plans" occur? The idea is to choose all 3 of the above for the planning problem and then apply some metaheuristic (such as Simulated Annealing, Tabu Search, Genetic Algorithms etc). Clearly, for nontrivial problems, only a small fraction of the space of "all possible plans" is actually explored.

CAVEAT: Actually planning (in contrast to the vast majority of other problems amenable to state-space search such as scheduling, packing, routing etc) is a bit of a special case, in that it is sometime possible to solve planning problems simply by using A* search, rather than searching with a stochastic metaheuristic.


Why is my model accuracy high in train-test split but actually worse than chance in validation set?

Looking at your code, one set of data transformations were applied to the train data and a different set of transformations were applied to the test data. Different data transformations could account for different evaluation metric performance.

It is best practices to put all data transformations in a function so they can be applied to all data in a similar way.

Since you are using scikit-learn, sklearn.compose.ColumnTransformer is designed for this purpose. Example code for the Titanic dataset is here.


How to determine the number of hidden layers and units of a deep auto-encoder?

You are right!
1- the number of hidden layers shouldn't be too high! Because of the gradient descent when the number of layers is too large, the gradient effect on the first layers become too small! This is why the Resnet model was introduced.
2- the number of hidden layers shouldn't be too small to extracts good features. It's proved that in CNN networks the first layers extract very simple elements like lines and curves but last layers extracts more complex features.
3- number of hidden units is a hyper-parameters and usually you should find it by testing or based on your background knowledge.
But what can you do at all?
As you can tests different parameters and compare their results there is some other options! One option is grid search you can check this tutorial https://towardsdatascience.com/grid-search-for-model-tuning-3319b259367e


Is "dataset size" and "model size" same thing?

Dataset and model refer to different things. Dataset means part of data available for training (training dataset) or validation (validation dataset). Model is the learning process goal, the state of the computer "brain" after it has been fully educated (or made its learning). Model size refers to size of the container which contains the model. In deep learning it can be measured by width and depth of the network used, I also found a site comparing different models by npy file size, that physically contains the generated model as computer code. In that case model contained a more complex structure which was documented and size in bytes was for comparison purposes.

So in short, it is roughly speaking the size of layers and neurons, if I have to take one of your options. Dataset is a different thing.

More precise explanation about what is model and what is dataset:

https://www.quora.com/What-are-different-models-in-machine-learning


What is the difference between encoders and auto-encoders?

Theory

Encoder

In general, an Encoder is a mapping $f : X \rightarrow Y $ with $X$ Input Space and $Y$ Code Space
In case of Neural Networks, it is a Generative Model hence a function which is able to compute a Representation out of some input (like GAN)

The point is: how would you train such an encoder network ?

The general answer is: it depends on what you want your code to be and ultimately depends on what kind of problem the NN has to solve, so let's pick one

Signal Compression

The goal is to learn a compressed representation for your input that allows to reconstruct the original input minimizing the loss of information

In this case hence you want the dimensionality of $Y$ to be lower than the dimensionality $X$ which in the NN case means the code space will be represented by less neurons than the input space

Autoencoder

Focusing on the Signal Compression problem, what we want to build is a system which is able to

take a given signal with size N bytes
compress it into another signal with size M<N bytes
reconstruct the original signal, starting from the compressed representation, as good as possible

To be able to achiebve this goal, we need basically 2 components

an Encoder which compresses its input, performing the $f : X \rightarrow Y$ mapping
a Decoder which decompresses its input, performing the $f: Y \rightarrow X$ mapping

We can approach this problem with the Neural Network Framework, defining an Encoder NN and a Decoder NN and training them

It is important to observe this kind of problem can be effectively approached with the convenient learning strategy of unsupervised learning : there is no need to spend any human work (expensive) to build a supervision signal as the original input can be used for this purpose

This means we have to build a NN which operates essentially between 2 spaces

the $X$ Input Space
the $Y$ Latent or Compressed Space

The general idea behind the training is to make a certain input go along the encoder + decoder pipeline and then compare the reconstruction result with the original input with some kind of loss function

To define this idea a bit more formally

The final autoencoder mapping is $f : X \rightarrow Y \rightarrow X$ with

the $x$ input
the $y$ encoded input or latent representation of the input
the $\hat x$ reconstructed input

Eventually you will get an architecture similar to

You can train this architecture in an unsupervised way, using a loss function like $f : X \times X \rightarrow \mathbb{R}$ so that $f(x, \hat x)$ is the loss associated to the $\hat x$ reconstruction compared with the $x$ input which is also the ideal result

Code

Now let's add a simple example in Keras related to the MNIST Dataset

from keras.layers import Input, Dense
from keras.models import Model

# Defines spaces sizes

## MNIST 28x28 Input
space_in_size = 28*28

## Latent Space
space_compressed_size = 32

# Defines the Input Tensor
in_img = Input(shape=(space_in_size,))

encoder = Dense(space_compressed_size, activation='relu')(in_img)

decoder = Dense(space_in_size, activation='sigmoid')(encoder)

autoencoder = Model(in_img, decoder)

autoencoder.compile(optimizer='adam', loss='binary_crossentropy')


Other deep learning image generation techniques besides GANs?

There are several generative models that have been proposed before or roughly at the same time of the GAN (2014). For example, the deep Boltzman machine (2009), deep generative stochastic network (2014) or variational auto-encoder (2014).


Context-based gap-fill face posture-mapper GAN

I believe you may want to use a Sum Product Network for this task. SPNs are the state-of-the-art approach for face completion, and there are several more recent papers on this topic since the original above.

Importantly, the SPN paper also covers other approaches that work well for this task. If lower-resolution results are acceptable for your task, PCA with 100 or more components works surprisingly well. If your dataset is very large, a nearest-neighbor approach can work decently too.


Are the Word2Vec encoded embeddings available online?

The gensim library for Python provides both a convenient way of manipulating Word2Vec embeddings, but also downloadable data sets and pretrained models. That last part is what you're looking for. Wikipedia2Vec also has multilingual embeddings. Then this guy (whom I've never heard of) has an assortement of languages I haven't seen other places. There's another 44 models hosted by the European NLPL. You can also search Kaggle, but a lot of this looks like training data, models I've already linked, or just cruft.


Are more than 8 high performance Nvidia GPUs practical for deep learning applications?

I did some recent research on this topic. It all comes down to parallelization.
Basically there are 2 ways to do it: model parallelization or batch parallelization.

Model parallelization is when you split the model by layers among multiple GPUs. As per my best knowledge you can't split a layer between GPUs, so 8 GPUs would serve 8 layers that is very extensive. Tensorflow supports this method. In my opinion more than 6 doesn't make sense this way.

Batch parallelization is when you run the entire model on each GPU parallel and you split your batch and process it in parallel. This is done via a trick to define a larger batch that will be split and become the desired batch size after split. In this case batch splitting and updating the weights is done on the CPU (in case of Tensorflow) and after 3 GPU any additional GPU has only a marginal improvement on training speeds (as per reports). So here even 4 doesn't make sense and 8 is just crazy.  Here is an example of batch parallelization.

Alternately if you are good in coding you may want to have a look at this paper, section 3.5 where it is explained how 8 GPUs were utilized to serve a 4 layer LSTM network. Probably you can do things like this to utilize DGX-1 but as far as I know Tensorflow doesn't support splitting a layer to multiple GPUs. My conclusion is that it's already very hard to utilize 8 GPUs and above that bus speed becomes the bottleneck.

Extension:
I double checked on the bus speed and I was wrong, it should not be a problem. Most of the time consumed in training is computational effort of backpropagation.

Actually PCIe speed is limited by the CPU and the mobo chipset. The CPU has PCIe lanes to be allocated by the mobo. The strongest single CPU at the moment is Broadwell-E with 40 lanes (Skylake rumored to have 44). Mobo allocates that bandwidth either x16 or x8 to PCIe peripherals. So with a 40-lane CPU you can run 2 cards at x16 (2 * 16 = 32 < 40) or 5 cards at x8 (5 * 8 = 40). Here it needs to be mentioned that M.2 also uses PCIe lanes so for the latter option forgot and M.2 drive. A single CPU system will not take 8 GPUs, that's why they need dual CPU in DGX-1. The next limitation is the mobo chipset. The most powerful at the moment is the X99 and the C6 series. X299 will be announced next week if memory serves and probably a C6 replacement follows soon.

Since PCIe speed is not the bottleneck in machine learning the updated answer for your question would be the more GPUs the better. It seems the limit is 5 at x8 using PCIe 3.0, but more than that is rarely needed since layers can't be split between GPUs. (And the deepest NN that still makes sense may be considered as 5-6 layers.)


How are these equations of SGD with momentum equivalent?

The first two equations are equivalent. The last equation can be equivalent if you scale $\alpha$ appropriately.
Equation 1
Consider the equation from the Stanford slide:
$$
v_{t}=\rho v_{t-1}+\nabla f(x_{t-1})
\\
x_{t}=x_{t-1}-\alpha v_{t},
$$
Let's evaluate the first few $v_t$ so that we can arrive at a closed form solution:
$v_0 = 0 \\
v_1 =  \rho v_0 + \nabla f(x_0) = \nabla f(x_0)\\
v_2 = \rho v_1 + \nabla f(x_1) = \rho \nabla f(x_0) + \nabla f(x_1)\\
v_3 = \rho v_2 + \nabla f(x_2) = \rho^2 \nabla f(x_0) + \rho \nabla f(x_1) + \nabla f(x_2)\\
\dots \\
v_t = \displaystyle \sum_{i=0}^{t-1} \rho^{t-1-i} \nabla f(x_i)
$
So the closed form update is:
$$x_t = x_{t-1} - \alpha \displaystyle \sum_{i=0}^{t-1} \rho^{t-1-i} \nabla f(x_i)$$
Equation 2
Now consider the equation from the paper:
$$
v_{t}=\rho v_{t-1}+\alpha \nabla f(x_{t-1})
\\
x_{t}=x_{t-1}- v_{t},
$$
We again evaluate the first few $v_t$ to arrive at a closed form solution:
$v_0 = 0 \\
v_1 =  \rho v_0 + \alpha \nabla f(x_0) = \alpha \nabla f(x_0)\\
v_2 = \rho v_1 + \alpha \nabla f(x_1) = \rho \alpha \nabla f(x_0) + \alpha \nabla f(x_1)\\
v_3 = \rho v_2 + \alpha \nabla f(x_2) = \rho^2 \alpha \nabla f(x_0) + \rho \alpha \nabla f(x_1) + \alpha \nabla f(x_2)\\
\dots \\
v_t = \displaystyle \sum_{i=0}^{t-1} \rho^{t-1-i} \alpha \nabla f(x_i)
$
So the closed from update is:
$$x_t = x_{t-1} - \displaystyle \sum_{i=0}^{t-1} \rho^{t-1-i} \alpha \nabla f(x_i)$$
As you can see, this is equivalent to the previous closed form update. The only difference is if $\alpha$ is inside or outside the summation, but since it is a constant, it doesn't really matter anyways.
Equation 3
As for the last equation
$$
v_{t}= \rho v_{t-1}+ (1- \rho) \nabla f(x_{t-1})
\\
x_{t}=x_{t-1}-\alpha v_{t}
$$
Let's do the same thing:
$v_0 = 0 \\
v_1 =  \rho v_0 + (1-\rho) \nabla f(x_0) = (1-\rho) \nabla f(x_0)\\
v_2 = \rho v_1 + (1-\rho) \nabla f(x_1) = \rho (1-\rho) \nabla f(x_0) + (1-\rho) \nabla f(x_1)\\
v_3 = \rho v_2 + (1-\rho) \nabla f(x_2) = \rho^2 (1-\rho) \nabla f(x_0) + \rho (1-\rho) \nabla f(x_1) + (1-\rho) \nabla f(x_2)\\
\dots \\
v_t = \displaystyle \sum_{i=0}^{t-1} \rho^{t-1-i} (1-\rho) \nabla f(x_i)
$
And so the closed form update is:
$$x_t = x_{t-1} - \alpha \displaystyle \sum_{i=0}^{t-1} \rho^{t-1-i} (1-\rho) \nabla f(x_i)$$
This equation is equivalent to the other two as long as you scale $\alpha$ by a factor of $\displaystyle \frac{1}{1-\rho}$.


Does epsilon-greedy approach always choose the "best action" (100% of the time) when it does not take the random path?

Epsilon-greedy is most commonly used to ensure that you have some element of exploration in algorithms that otherwise output deterministic policies.

For example, value-based algorithms (Q-Learning, SARSA, etc.) do not directly have a policy as output; they have values for states or state-action pairs as outputs. The standard policy we "extract" from that is a deterministic policy that simply tries to maximize the predicted value (or, technically, a "slightly" nondeterministic policy in that, in proper implementations, it should break ties (where there are multiple equal values at the top) randomly). For such algorithms, there is not sufficient inherent exploration, so we typically use something like epsilon-greedy to introduce an element of exploration. In these cases, both of the possible explanations in your question are identical.

In cases where your algorithm already produces complete probability distributions as outputs that do not so much focus all of the probability mass on a single or a couple of points, like the probability distribution you gave as an example in your question, it's generally not really necessary to use epsilon-greedy on top of it; you already get exploration inherently due to all actions having a decent probability assigned to them.

Now, I've actually personally mostly worked with value-based methods so far and not so much with e.g. policy gradient methods yet, so I'm not sure whether there tends to be a risk that they also "converge" to situations where they place too much probability mass on some actions and too little on others too quickly. If that's the case, I would expect an additional layer of epsilon-greedy exploration might be useful. And, in that case, I would indeed find your explanation the most natural. If I look through, for example, the PPO paper, I didn't find anything about them using epsilon-greedy in a quick glance. So, I suppose the combination of epsilon-greedy with "nondeterministic" policies (ignoring the case of tie-breaking in value-based methods here) simply isn't really a common combination.


What are the differences between A* and greedy best-first search?

Both algorithms fall into the category of "best-first search" algorithms, which are algorithms that can use both the knowledge acquired so far while exploring the search space, denoted by $g(n)$, and a heuristic function, denoted by $h(n)$, which estimates the distance to the goal node, for each node $n$ in the search space (often represented as a graph).

Each of these search algorithms defines an "evaluation function", for each node $n$ in the graph (or search space), denoted by $f(n)$. This evaluation function is used to determine which node, while searching, is "expanded" first, that is, which node is first removed from the "fringe" (or "frontier", or "border"), so as to "visit" its children. In general, the difference between the algorithms in the "best-first" category is in the definition of the evaluation function $f(n)$.

In the case of the greedy BFS algorithm, the evaluation function is $f(n) = h(n)$, that is, the greedy BFS algorithm first expands the node whose estimated distance to the goal is the smallest. So, greedy BFS does not use the "past knowledge", i.e. $g(n)$. Hence its connotation "greedy". In general, the greedy BST algorithm is not complete, that is, there is always the risk to take a path that does not bring to the goal. In the greedy BFS algorithm, all nodes on the border (or fringe or frontier) are kept in memory, and nodes that have already been expanded do not need to be stored in memory and can therefore be discarded. In general, the greedy BFS is also not optimal, that is, the path found may not be the optimal one. In general, the time complexity is $\mathcal{O}(b^m)$, where $b$ is the (maximum) branching factor and $m$ is the maximum depth of the search tree. The space complexity is proportional to the number of nodes in the fringe and to the length of the found path.

In the case of the A* algorithm, the evaluation function is $f(n) = g(n) + h(n)$, where $h$ is an admissible heuristic function. The "star", often denoted by an asterisk, *, refers to the fact that A* uses an admissible heuristic function, which essentially means that A* is optimal, that is, it always finds the optimal path between the starting node and the goal node. A* is also complete (unless there are infinitely many nodes to explore in the search space). The time complexity is $\mathcal{O}(b^m)$. However, A* needs to keep all nodes in memory while searching, not just the ones in the fringe, because A*, essentially, performs an "exhaustive search" (which is "informed", in the sense that it uses a heuristic function).

In summary, greedy BFS is not complete, not optimal, has a time complexity of $\mathcal{O}(b^m)$ and a space complexity which can be polynomial. A* is complete, optimal, and it has a time and space complexity of $\mathcal{O}(b^m)$. So, in general, A* uses more memory than greedy BFS. A* becomes impractical when the search space is huge. However, A* also guarantees that the found path between the starting node and the goal node is the optimal one and that the algorithm eventually terminates. Greedy BFS, on the other hand, uses less memory, but does not provide the optimality and completeness guarantees of A*. So, which algorithm is the "best" depends on the context, but both are "best"-first searches.

Note: in practice, you may not use any of these algorithms: you may e.g. use, instead, IDA*.


What would an implementation of this Neural Network look like?

In the paper there is a diagram of the network showing how there are 3 outputs which share features but have their own set of associated weights. Back propagation works by only looking at one output at a time.


Is it possible to count the number of squats with Computer Vision techniques?

Yes, this is a rather straightforward problem that should be able to be solved using openCV or ANNs.

Check out this paper to get you started: https://arxiv.org/pdf/1701.08936.pdf


Is a reward given at every step or only given when the RL agent fails or succeeds?

In reinforcement learning (RL), an immediate reward value must be returned after each action, along with the next state. This value can be zero though, which will have no direct impact on optimality or setting goals.
Unless you are modifying the reward scheme to try and make an environment easier to learn (called reward shaping), then you should be aiming for a "natural" reward scheme. That means granting reward based directly on the goals of the agent.
Common reward schemes might include:

+1 for winning a game or reaching a goal state granted only at the end of an episode, whilst all other steps have a reward of zero. You might also see 0 for a draw and -1 for losing a game.

-1 per time step, when the goal is to solve a problem in minimum time steps.

a reward proportional to the amount of something that the agent produces - e.g. energy, money, chemical product, granted on any stop where this product is obtained, zero otherwise. Potentially a negative reward based on something else that the agent consumes in order to produce the product, e.g. fuel.


If we want to classify something as either a cat/dog or neither, do we need 2 or 3 classes?

If you want to determine if something is either a

cat/dog or neither

you need 2 classes:

one for dog or cat, and
one for anything else.

However, if you assign all cats and dogs to the same class $A$, if an input is classified as $A$, then you won't be able to know whether it is a dog or a cat, you will just know that it is either a dog or a cat.
In case you wanted to distinguish between cats and dogs too (apart from neither of them), then you'll need $3$ classes.
Finally, if you specify only 2 classes:

dog, and
cat,

then your CNN will try to classify any new input as either a dog or a cat, even though it is neither a dog nor a cat (e.g. maybe it is a horse).


What is the effect of using pooling layers in CNNs?

Pooling has multiple benefits

Robust feature detection.
Makes it computationally feasible to have deeper CNNs

Robust Feature Detection

Think of max-pooling (most popular) for understanding this.
 Consider a 2*2 box/unit in one layer which is mapped to only 1 box/unit in the next layer (Basically pooling). Let's say the feature map (kernel) detects a petal of a flower. Then qualifying a petal if any of the 4 units of the previous layer is fired makes the detection robust to noise. There is no strict requirement that all 4 units should be fired to detect a petal. Thus, the next layer (after pooling) captures the features with noise invariance. We can also say it is local translation invariance (in a close spatial sense) as a shifted feature will also be captured. But also remember translation invariance in general is captured by the convolution with kernels in the first place. (See how 1 kernel is convolved with the whole image)

Computational advantage

The dimensions of the inputs in image classification are so huge that the number of the multiplication operation is in billions even with very few layers. Pooling the output layer reduces the input dimension for the next layer thus saving computation. But also now one can aim for really deep networks(number of layers) with the same complexity as before.


If neurons performed the operation of an entire layer, would that make the neural network more effective?

Say, a single neuron could do the function of an entire layer in a network, could that make the network more effective?

That depends what you mean by "more effective". In terms of number of neurons to achieve the same result, then you should need fewer units. In terms of being able to calculate an end result for any specific problem, then no, because you can generally solve a problem using simpler units by adding more of them.
If this could somehow be done using less resources, it might reduce overall costs. In a biological system, there are possibly overheads per cell in order to maintain it on top of the costs for calculation, so it may be better to do more than the simplest calculation in each cell. Further to that, there may be an optimal amount of processing that each cell could do (this is all conjecture on my part).
In an artifical neural network, the calculations are the only thing being considered, there is no separate overhead per neuron.
There are neural network architectures with complex "sub-units". Probably the most well known are the recurrent neural network designs for LSTM and gated recurrent units (GRU), plus "skip connections" in residual neural networks. For efficiency these are normally processed in groups per layer with matrix processing functions, but you can also view them as per-neuron complexities.

I am not sure of this- why would complex individual components make the network less effective?

If the complexity was not used, or not really needed, in some of the units, then it would be wasted capacity. In a biological system, this might correspond to maintaining cells larger than they needed to be. In an artificial system, it would mean using memory and CPU to calculate interim values that were not needed for the task at hand.


Can we use a pre trained Encoder (BERT, XLM ) with a Decoder (GPT, Transformer-XL) to build a Chatbot instead of Language Translation?

=> I don't have more ideas about BART and T5 Right now. but I had created a chatbot Based on GPT-2 Model-based on Microsoft DialoGPT. Which is fine-tuned on millions of parameters of Reddit. you can fine-tune on your own data using DialoGPT. I had not found a public decoding method. So, We tried to generate diverse responses using simple nucleus sampling at each time step. So, We used greedy nucleus sampling multiple times in parallel to generate multiple candidates responses. We generated 30 such responses for each turn and also used the last 3 turn conversation history as extra context for generator. So, We can generate responses related to the current Context of Conversation.

We used Code reference from here. you need to implement greedy nucleus sampling multiple on this code.

=>Now To Choose the best-matched response from response generated I created a Reranker.I used two sub-component.

=>Component one Counts Cosine similarity between Response and Query using sentence Encoder.

=>In Component Two we cross-entropy error for generating original query from the response.For that I used reverse-generator of DialogueGPT.

We Combined Score of Both Component as Score=norm(comp1Score)+norm(1-comp2Loss). Using this score you can find best response.

For Tutorial My Favourite Hugging Face they have BERT,GPT, GPT-2, XLNET,etc...
and Second Favourite Facebook ParlAI,
try Blender Model also.


Before we commence I will recommend that you refer to similar questions on the network i.e https://stackoverflow.com/questions/9706769/any-tutorials-for-developing-chatbots , https://stackoverflow.com/questions/55042/how-can-i-program-a-simple-chat-bot-ai and https://stackoverflow.com/questions/1748887/how-to-create-a-smart-chat-bot

In my answer I follow the approach laid out on two outstanding articles. Chatbots with Seq2Seq (Suriyadeepan R 2016) and Deep Learning for Chatbots Part 1 & 2 (Benny B 2016) which are available here http://suriyadeepan.github.io/2016-06-28-easy-seq2seq/ and here http://www.wildml.com/2016/04/deep-learning-for-chatbots-part-1-introduction/ respectively.

Chatbots are also known as conversational agents or dialog systems. Your goal is to build a friendly AI. Modeling personality may appear straight forward, however incorporating such knowledge into a model is still a challenging problem. This is because the models are trained on a lot of data from different sources and users. Below is a link to a paper on building such a Persona-Based Neural Conversation Model https://arxiv.org/abs/1603.06155

There are two important concepts that one needs to understand when learning about chatbots. That is the chatbot publishing platform and the chatbot development platform. A chatbot publishing platform is a medium through which the chatbot can be accessed by the users i.e. FB messenger, LINE, Telegram or WhatsApp, on the other hand a chatbot development platform is a tool that can be used to create a chatbot. Chatbot development platforms i.e. Beep Boop, Flow XO, Botsify and Chatfuel can be used to build chatbots without writing any code, this is done by simply using a drag and drop interface. Such development platforms however offer only limited functionality and customizability. (Marutitech blog)

A developer can choose to use a rule/retrieval based approach, here you simply write a pattern and a template. Such a bot replies with one of the templates when it encounters a similar pattern from the user. Rule based models tend to perform poorly when they encounter completely new sentences. (Suriyadeepan Ram 2016)

A different approach involves utilizing a generative model. Generative models construct responses word by word based on the query. Because of this the generated responses are prone to grammatical errors. However once they are trained, the generative model outperforms the rule based approach especially in handling previously unseen queries.

  In probability and statistics a generative model is a model that generates all values for a phenomenon, both those that can be observed in the world and "target" variables that can only be computed from those observed.

For our example below, we will first preprocess the data and then train the generative model using sequence 2 sequence. Seq 2 Seq is a general purpose encoder decoder framework for Tensorflow which can be applied to machine translation, summarizing text ,image captioning and conversational modeling. The seq2seq network connects two RNN's to work together to transform one sequence to another. An encoder network condenses an input sequence into a vector while a decoder network unfolds that vector into a new sequence.

Before training a dataset on a generative model we need to perform a preprocessing step called padding. Here we work on converting the variable length sequences into fixed length sequences. Some of the datasets which can be used to train a chatbot using a generative model approach include: the Cornell Movie Dialog Corpus, which is a collection of dialogs from movie scripts and the Ubuntu Dialog Corpus which is based on chat logs from the Ubuntu IRC public channels.

The next step will be to implement the Continuous Bag of Words (CBOW) model, which is a form of word embedding. CBOW is a model of simplifying representation used in NLP. In this step text is represented as a multiset (Bag) of words disregarding grammer but retaining the frequency of words.

The final step in our model is implementing the Attention Mechanism. This step is very important since it prevents the information loss that could have occured while compressing all the necessary information of a source sentence into a single vector.

For source code, I recommend implementing Siraj Rawal's Seq 2 Seq Chatbot because of its clear documentation. Its available here https://github.com/llSourcell/tensorflow_chatbot


Can we make an algorithm which can solve any high school (science) problem using ML and DL?

Its true that your AI model's performance depends on the quality of data that you use. However, high quality data alone is insufficient to guarantee that your model will learn effectively and score well on a particular dataset. Other factors such as smarter algorithms and the use of high performance computing infrastructure must be factored in for your AI system to perform well.

Although A.I research has made massive progress in the past decade, ML engineers are yet to build a system that can match the general scope and generalization ability of the human mind. Upto the first decade of the 2000's AI was dominated by expert systems that emulated the decision making ability of an expert. AI at this point couldn't process unstructured data and therefore it lacked the capacity to sit for and pass high school exams.

This was until 2011 when IBM Watson a question answering computer system competed against two former Jeopardy quiz show winners and placed first. IBM Watson was built on top of Deep QA  (a computer system that could answer natural language questions) and UIMA (a software achitecture to process and analyse unstructured information). Below is a link to a paper giving an overview of how IBM's Watson works https://www.aaai.org/Magazine/Watson/watson.php

In 2012 a team led by Geofrey Hinton won the ImageNet competition by exploiting deep convolution networks. This was soon followed by Dahl's team winning the Merck Molecular Activity Challenge using deep neural network architecture. Yann LeCun's work in CNN's, Geoff Hinton's back propagation and Stochastic Gradient Descent aproach to training datasets alongside Andrew Ng's large scale use of GPU's ignited accelerated progress in ML. This was frequently referred to as unreasonable effectiveness of Deep learning.

Following recent advances in fields such as image captioning, natural language processing, information retrieval and computer vision it is highly probable that current generation AI systems can pass high school exams such as SAT.

The Allen AI Institute has made significant progress in developing AI systems that can read, learn and express that understanding through question answering and explanation. Founded by Paul Allen Microsoft's co-founder, the Allen AI Institute's singular focus according to their mission is to conduct high impact research in the field of AI. Below is a news link covering their cognitive system passing high school exams fortune.com/2015/09/21/computer-artificial-intelligence-math/

So far Allen AI Institute has demonstrated a cognitive platform called Geos that is capable of answering geometry questions as well as the average high school student. While another system called Aristo can answer high school science exam questions by leveraging information extraction alongside knowledge representation and reasoning models. You can access AAI's GeoS service here http://allenai.org/euclid/ and Aristo here http://allenai.org/aristo/

Meanwhile researchers working on the Todai project in Japan have demonstrated a cognitive system that is capable of passing the Tokyo University Mathematics entrance exam. My conclusion from the above examples is that possibly we already have AI that can sit for and pass high school exams.


Which layer in a CNN consumes more training time: convolution layers or fully connected layers?

NOTE: I did these calculations speculatively, so some errors might have crept in. Please inform of any such errors so I can correct it.

In general in any CNN the maximum time of training goes in the Back-Propagation of errors in the Fully Connected Layer (depends on the image size). Also the maximum memory is also occupied by them. Here is a slide from Stanford about VGG Net parameters:

Clearly you can see the fully connected layers contribute to about 90% of the parameters. So the maximum memory is occupied by them.

As far as training time goes, it somewhat depends on the size (pixels*pixels) of the image being used. In FC layers it is straightforward that the number of derivatives you have to calculate is equal to the number of parameters. As far as the convolutional layer goes, let's see an example,let us take the case of 2nd layer:
It has got 64 filters of $(3*3*3)$ dimensions to be updated. The error is propagated from the 3rd layer. Each channel in the 3rd layer propagates it error to its corresponding $(3*3*3)$ filter. Thus $224*224$ pixels will contribute to about $224*224*(3*3*3)$ weight updations. And since there are $64$ such $224*224$ channels we get the total number of calculations to be performed as $64*224*224*(3*3*3) \approx 87*10^6 $ calculations.

Now let us take the last layer of $56*56*256$. It will pass it gradients to the previous layer. Each $56*56$ pixel will update a $(3*3*256)$ filter. And since there are 256 such $56*56$ the total calculations required would be $256 * 56 * 56 * (3*3*256) \approx 1850 *10^6$ calculations.

So number of calculations in a convolutional layer really depends on the number of filters and the size of the picture. In general I  have used the following formulae to calculate the number of updations required for filters in a layer, also I have considered $stride = 1$ since it will be the worst case:

$channels_{output} * (pixelOutput_{height} * pixelOutput_{width}) * (filter_{height} * filter_{width} * channels_{input})$

Thanks to fast GPU's we are easily able to handle these huge calculations. But in FC layers the entire matrix needs to be loaded which causes memory problems which is generally not the case of convolutional layers, so training of convolutional layers is still easy. Also all these have to be loaded in the GPU memory itself and not the RAM of the CPU.

Also here is the parameter chart of AlexNet:

And here is a performance comparison of various CNN architectures:

I suggest you check out the CS231n Lecture 9 by Stanford University for better understanding of the nooks and crannies of CNN architectures.


Why doesn't my image classification network get better with training?

The answer (as detailed quite thoroughly here and here) is that specifying

model.compile(optimizer = 'rmsprop', loss = 'binary_crossentropy', metrics=['accuracy'])

causes keras to guess, incorrectly, that because I am using binary_crossentropy for the loss function, that I would want to use binary_accuracy as the way of reporting the accuracy metrics. Apparently, one should specify that one wants the categorical_accuracy metrics if one, as I do, has more than two classes.


Can the decoder in a transformer model be parallelized like the encoder?
The correct answer is: computation in a Transformer decoder can be parallelized during training, but not during actual translation (or, in a wider sense, generating output sequences for new input sequences during a testing phase).
What exactly is parallelized?
Also, it's worth mentioning that "parallelization" in this case means to compute encoder or decoder states in paralllel for all positions of the input sequence. Parallelization over several layers is not possible: the first layer of a multi-layer encoder or decoder still needs to finish computing all positions in parallel before the second layer can start computing.
Why can the decoder be parallelized position-wise during training?
For each position in the input sequence, a Transformer decoder produces a decoder state as an output. (The decoder state is then used to eventually predict a token in the target sequence.)
In order to compute one decoder state for a particular position in the sequence of states, the network consumes as inputs: 1) the entire input sequence and 2) the target words that were generated previously.
During training, the target words generated previously are known, since they are taken from the target side of our parallel training data. This is the reason why computation can be factored over positions.
During inference (also called "testing", or "translation"), the target words previously generated are predicted by the model, and computing decoder states must be performed sequentially for this reason.
Comparison to RNN models
While Transformers can parallelize over input positions during training, an encoder-decoder model based on RNNs cannot parallelize positions. This means that Transformers are generally faster to train, while RNNs are faster for inference.
This observation leads to the nowadays common practice of training Transformer models and then using sequence-level distillation to learn an RNN model that mimicks the trained Transformer, for faster inference.


How to deal with changing video frame sizes in a CNN?

Well, the easiest way in order to make different frame sizes work with a convolutional neural network, is to process them using a different operations such as scaling, and or cropping in order to make them the same size. I do not know of any way currently where different frame sizes can be inputted directly into the neural network, as in order to do that you would need to almost completely change the neural network architecture. If the frames have a small difference in size, you may be able to get away with scaling the frame to the dimensions of the largest frame. This also depends upon what kind of problem you are trying to solve. While this method would work if you are trying to classify an image into a couple categories, it would not work if your are trying to do tasks such as object location. If you are doing one of the latter tasks, you should be able to pad the image with blank pixels in order to make the frame you are trying to process the dimensions of the largest one. This would work quite well, especially for object location detection. Finally, another solution is to crop the larger images to the size of the smallest one. While this will work just as well as padding, if you need any image data from the very edges of the frame, this will not work very well. So in conclusion, there are three different methods you could apply. Me personally, I would go with padding because it does not remove any important image data, and it will work very well for tasks such as object location detection. If you are doing a classification task though, scaling would work also. Finally, you could crop the images, but this would have to be evaluated for your specific task.


Is there a car detection software written in Tensorflow or Keras with Python?

I recall seeing several data sets on Kaggle on the topic of car images. Usually these data sets have associated with them user uploaded python kernels that do either classification( what make of car) or detection ( show a bounding box on the car).
I would recommend you go to Kaggle then select data sets and do a search for 'cars'. I did a quick look and found a data set at https://www.kaggle.com/jessicali9530/stanford-cars-dataset that has 18 kernels associated with it. There well may be many more data sets available. Good luck in your search!


How to train for own dataset really really fast while debugging

Assuming that you are using DarkFlow, you can train your model with --train flag and don't use --gpu flag since you don't want to use the GPU memory. And yes it's possible to train with just 10 images (basically over-fitting) just to confirm the working of your model. Place those 10 images in train/images directory and the annotations of these images in train/annotations directory (in PASCAL VOC format).
For faster performance, use pre-trained weights file for tuning your model so that you don't need to train for initial same layers (and since you just want to detect a single object - license plates - use tiny-yolo-voc.weights.
Further more, use --epoch flag to reduce the number of epochs.

flow --model cfg/tiny-yolo-voc-new.cfg --load bin/tiny-yolo-voc.weights --train --annotation train/Annotations --dataset train/Images --epoch 20
Change the number of classes and the number of filters as defined in the readme first. Then you can change the height and width of the input image the model will be trained on. Higher resolution for more accuracy while lower resolution for higher performance and speed. You can also change the activation functions for the layers, but I recommend leaving them unchanged.


This depends on whether the output is a continuous or discrete variable. If the output variable is discrete (there are a finite number of possibilities that it can be), as in a classification task (such as this one, where you are trying to place the input into one of 5 categories), you want to use one output neuron for each class. If the variable is continuous, however, you should only use one output neuron.

This is because of how the training process works. When training your network successively makes adjustments to try and reduce the errors. These adjustments are made in the direction of the error – so if the network predicts a value which is too high then the network's weights are adjusted to make the output value lower. On the other hand if the network's predicts a value which is too low the network's weights are adjusted to make the output bigger.

If you have output neurons labeled 0 to 4 and a training sample with some input value and a target prediction of 2 then the neural network will make its prediction. Once the prediction has made each neuron is adjusted individually – in this case neuron 2 will be adjusted in the direction of the correct probability and all the other neurons will be adjusted in the direction of the incorrect probability. In this way you have one prediction for each class.

Backpropagation is a about error attribution, and using multiple neurons allows the error of the neural network to be better attributed as the neural network can adjust each neuron individually, and thus adjust the required probabilities for each class.

Using a single neuron with a sigmoid activation function would be less good as the sigmoid function saturates values close to 0 and 1 so there would be an unnatural bias towards category 0 and category 4 over the other categories. The neural network could learn to overcome this, but it would take more time.


How does the gradient increase the probabilities of the path with a positive reward in policy gradient?

Think of a surface where z-axis is $U$, x and y axis are $\theta_{1}$ and $\theta_{2}$ accordingly. Since you are following the gradient direction with respect to $\theta$ vector, that means you are moving at the direction that increases the $U$. If $R(\tau)$ is positive, you are moving towards the uphill direction and vice-versa. More formally you will say the following:

In the policy gradient algorithm, our update step is:

$ \theta_{new} = \theta_{old} + \alpha \nabla_{\theta}U(\theta) $

So, if we select a very bad trajectory the sum of all rewards ($R(\tau)$) will be negative and the following update will shift the $\theta_{new}$ vector away from $\nabla_{\theta}U(\theta)$ vector.

If we select a good trajectory means $R(\tau)$ is positive, the update will shift $\theta_{new}$ vector towards the $\nabla_{\theta}U(\theta)$ vector. So, it will increase the probability of selecting path with positive $R$


Will/can AI be the worst enemy to humanity?

I highly suggest you read Hannu Rajaneimi's Quantum Thief trilogy, William Gibson's Neuromancer trilogy, and Phillip K. Dick's Do Android's Dream of Electric Sheep, Isaac Asimov's I, Robot.

It's a nuanced subject, and no one knows.

The most credible scenarios I've found for the destruction of humanity by automata involves not high-intelligence, but very low intelligence.  (The Grey Goo scenario.)  Dick, a gifted philosopher in his own right, proposes that empathy is a natural function of high intelligence, which provides a credible, increasingly validated ray of light.

But Hawking and Musk are credible imo per their comfort with very large numbers (complexity) and proven world-class abstract thinking capabilities.  It's possibly that strong "hyperpartizan" AI could run amok.  We've already seen issues with high-frequency trading algorithms, and automation is proceeding in weapons tech.

One might say "we are cursed to live in such interesting times."


Which online machine learning technique to use for multi-class classification problem with multiple inputs?

I suggest using Data Stream algorithms to try on your problems since you are asking for "online learning with singular values or minibatches as the dataset is too big too load all the training data into memory."

MOA is a good choice for these algorithms. Hoeffding Trees is also a good first  choice to try.


Why is reinforcement learning not the answer to AGI?

Some AI researchers do think RL is a path to AGI, and your intuition about how an agent would need to be proactive in selecting actions to learn about is exactly the area these researchers are now focused on.

Much of the work in this area is focused on the idea of curiosity, and since 2014 this idea has gained a lot of traction in the research community.

So, maybe RL can lead to AGI. We don't know for sure yet.

However, many of the classic arguments against AGI aren't addressed by the RL approach. For instance, if like Searle, you think computers just don't have the right kind of hardware to do thinking, then running an RL algorithm on that hardware isn't going to yield AGI, just ever increasingly robust narrow AI. Ultimately Searle's arguments get into issues of metaphysics, so it isn't clear that there exists any argument that would convince someone like Searle that a particular computer-based technique is AGI-capable.

There are also other arguments. For example, the cognativist school of thought thinks that statistical learning approaches to AI, and in particular, the black-box approaches of statistically-driven RL, are unlikely to lead to general intelligence because they do not engage in the kind of systematic reasoning process that proponents of cognativism assume is necessary for general intelligence. Some more extreme proponents of this school might say that a logical planning algorithm like STRIPS is innately more intelligent than any approach based on deep learning, because it involves sound logical deduction rather than mere statistical calculation. In particular, STRIPS can correctly generalize to any new domain, as long as it is fed the correct sense data, while an RL approach will need to learn how to act there.

So, while there are definitely reasons to be optimistic about RL as a direction for achieving AGI, it's definitely not yet settled.


It means that there is no explicit coding of action choices to promote to queen, it is the default assumption if the underpromotion actions are not taken.

The Alpha Zero chess implementation can represent promotion to queen by not selecting an underpromotion action, whilst moving a pawn so that it qualifies for promotion.