Understanding a paragraph about object detection with two objects

If I understood that post right (I just skimmed through, so it's possible I missed some details), they are using several predictors on the inputs with several rectangles. This basically means separate, not shared, weights to detect each rectangle.

It's fairly likely that these neurons will adjust to certain areas on the input image. As far as the neurons are concerned, they are "punished" if they detect the wrong rectangle, as if the other rectangle is the noise that the network should learn to ignore. As a result, the "left" predictor learns to fire more actively when it detects a rectangle on the left, because there's high chance it's the correct one. Same for the "right" predictor.

When, all of a sudden, the training image swaps the targets, each predictor gets confused. Because now what they used to detect as noise becomes the ground truth and vice versa. It certainly hurts the performance and makes the network be sure the most, when the target is around the center. When the targets are flipped, the dataset becomes more consistent and less confusing for the network, it's no wonder that the accuracy goes up.


How to select the most appropriate set of actions for a given environment or task?

One idea I had was to create an adversarial network which generates words

It may be better to use for e.g. numbers instead of words, then map these numbers to the language of your choice like English in your case. This mapping can be changed to other languages and you will have a robot that overcomes language barriers.


Could a multi-camera SLAM system that is accurate at low driving speeds be equally accurate at high driving speeds?

It would probably be accurate at higher speeds, if the processing system that processes the information from the camera system can process the information in real time. It looks like it can, but you will also have to account for motion blur. If there is too much motion blur, you will probably not be able to do anything other than speeding up the camera and processing system. However, if the motion blur is not that bad, and the system has been trained on the motion blur, it should be able to classify everything correctly.


What type of ANN architecture to choose?

It sounds like you have structured/tabular data. So, a fully-connected feedforward network should do the job.


What were the first areas of research and what were some early successes?

What might be classed as AI has of course changed over the years, but landmarks and research breakthroughs include:

Babbagge's Difference Engine (~1823) for tabulating/interpolating polynomials.
Frank Rosenblatt's 1957 invention of the Perceptron.
John McCarthy's invention of Lisp in the late 1950s.
Arthur Samuel's 1959 checkers player, which famously improved by playing against itself (it would have been nice if that had destroyed the myth about a program only being as 'intelligent' as its creator).
Newell and Simon's 1959 General Problem Solver applied Means-Ends analysis to solve a range of problems expressed as Horn clauses.
Davis, Putnam et al: 1962 invention of the DPLL algorithm which still forms the core of modern SAT-based theorem provers.
Lawrence Fogel et al: 1966 book Artificial Intelligence through Simulated Evolution.
Rechenberg and Schwefel: 1960s development of Evolutionsstrategie - an Evolutionary Computation approach using mutation and a form of Darwinian 'survival of the fittest'.
Lotfi Zadeh's 1965 invention of Fuzzy Logic.
John Holland's 1975 book "Adaptation in Natural and Artificial Systems" which introduced Genetic Algorithms.
The 1980 Hearsay II Blackboard Architecture by Hayes-Roth et al.
The 1980s invention of the backpropagation algorithm for Mutlilayer Perceptrons by Rumelhart, Hinton et al.


Is it possible to combine two neural networks trained on different tasks into one that knows both tasks?

Combining two different fully trained neural networks is not only feasible, it is commonly done. Let's look at the example given as two concepts involving integers, $C_a$ and $C_m$.
$$ C_a: \mathcal{Y} = f_a (\mathcal{X}) = x_0 + x_1 $$
$$ C_m: \mathcal{Y} = f_m (\mathcal{X}) = x_0 \, x_1 $$
Now let's define a palette of operations, including these two binary operations, that can be used to construct, a concept $C_e$, an expression comprised of an arbitrary hierarchy of addition, multiplication, constants, and substitution.
$$ C_e: \mathcal{Y} = f_e (\mathcal{X}) \; \text{, where} $$
$$ f_e \in \{f_a, f_b\} \; \land \; i \in \{0, 1\} \; \land \; ( \, x_i \in \mathbb{I} \; \lor \; x_i \in \mathcal{Y} \, ) \; \text{.} $$
Now, one artificial network can be trained to approximate $f_a$ within a concept class $\mathbb{C}$ of which $C_a$ and $C_m$ are members, using labeled examples of correct integer additions and another artificial network can be trained to approximate $f_b$ within that same concept class, using labeled examples of correct integer multiplications.
An expression involving both can be trained to approximate arbitrary product of sums or sum of products under specific conditions. It's unclear though if this is what you are looking for.
Normally, one wouldn't train a network to perform operations that are already known. Training is normally used to model operations that are not known.
(Bilinear Convolutional Neural Networks (B-CNNs), introduced in Bilinear CNNs for Fine-grained Visual Recognition, 2017, Tsung-Yu Lin, Aruni RoyChowdhury, Subhransu Maji, is an approach to using two CNNs in conjunction to provide two fine and course visual recognition in the same way the human visual system can have a dual awareness of detail and panorama. B-CNNs probably don't apply to the scenario given in the question.)


Is artificial intelligence vulnerable to hacking?

AI is vulnerable from two security perspectives the way I see it:

The classic method of exploiting outright programmatic errors to achieve some sort of code execution on the machine that is running the AI or to extract data.
Trickery through the equivalent of AI optical illusions for the particular form of data that the system is designed to deal with.

The first has to be mitigated in the same way as any other software. I'm uncertain if AI is any more vulnerable on this front than other software, I'd be inclined to think that the complexity maybe slightly heightens the risk.

The second is probably best mitigated by both the careful refinement of the system as noted in some of the other answers, but also by making the system more context-sensitive; many adversarial techniques rely on the input being assessed in a vacuum.


Why is the larger value, as opposed to the smaller one, chosen, in the hill climbing algorithm?

When we climb a hill:

We move higher in altitude. The person who is climbing, will always look for rocks/mud on the hill that are higher, so that he can climb higher.
That is what the algorithm does too. We are assuming that there is a hill of numbers. The larger numbers are placed higher than the smaller numbers. So if we want to climb up the hill, we search for the larger numbers.
Comparing this with the real world hill climbing, simply assume that every rock on the hill has a number written on it. The number will be the altitude of the rock, from sea level height. So if you want to climb up, you just have to search for the rock with a higher number than the rock you are standing on now.


Why do Decision Tree Learning Algorithm preferably outputs the smallest Decision Tree?

The bigger your tree is the more overfitting your model is. In machine learning, we always prefer a simpler model unless there is good reason to go for complication.


Which one is more important in case of different loss optimization algorithms, Speed or the Route?

The route/trajectory followed by the optimization algorithm basically depends your dataset and the loss function. However, what really matters, for the purpose of final accuracy performance, is the final point which the trajectory converges to.


How should I predict which characters are going to die in a certain movie?

This is more of a probability question. Having said that, based on each leaked script, you generate the possibilities of each of the characters dying (for example yes 0.2%, no 0.7%, unknown 0.1%) - you also give a weight for each leaked script based on on its perceived authenticity (from 0.1 to 1). You multiply the weight of each script by each of the possibilities and then you add the result for each possibility and each character across all scripts. So, for example, say you have 2 scripts. The first script has perceived authenticity of 20% (0.2) and the second script has perceived authenticity of 40% (0.4). In the first script, Character A has 50% (0.5) chance of dying, 20% (0.2) of not dying, and 30% (0.3) chance of unknown outcome. In the second script, character A has 20% (0.2) chance of dying, 50% (0.5) chance of not dying, and 30% (0.3) chance of unknown outcome. So the following applies for Character A:

Dying: (0.2 x 0.5) + (0.4 x 0.2) = 0.18 (18% chance of dying)
Not dying: (0.2 x 0.2) + (0.4 x 0.5) = 0.24 (24% chance of not dying)
Unknown outcome: (0.2 x 0.3) + (0.4 x 0.3) = 0.20 (20% unknown outcome)

The probability of not dying is 24%, which is the highest.

Note: I am not saying that probability has nothing to do with AI (in fact, it is very important in AI).


Can supervised learning be used to solve the inverted pendulum problem?

Of course, you can, and yes this has been researched and done before. By using supervised learning, you give the machine some data, and it will try to figure the best way to analyze and predict the next movement in case of an inverted pendulum problem.
I found a complete paper for this problem: Neural network control of an inverted pendulum on a cart by Valeri Mladenov et al.


Detect street and sidewalk surface in aerial imagery (neural network)

Yes, in fact neural networks (NNs) are very efficient at segmentation and it seems to me that your problem matches the capabilities of neural networks very well.

I think it best for you to truly understand what a NN is before using it. First, let's start with the architecture. A NN has 3 regions, the input layer, the hidden layers and the output layer. The input layer depends on the number of features in your dataset. The hidden layers, you can have multiple layers all of different breadth (number of nodes per layer). The output layer depends on the number of classes in your dataset.

An easy example is applying a NN to the MNIST datatset. This is a dataset which contains handwritten digits between 0-9. Let's assume each of these images is 16*16=256 pixels. Thus, you will need 256 input nodes. And you will need 10 output nodes, one for each possible output. The hidden layers can be set in any way you can creatively imagine. There are however ways to optimize your hidden layer to get the best performance possible while not spending too much computational power.

This is always how a NN works. The beauty of a NN is that you only need to code it once and it can learn any function. All you need to do is change your architecture, but the underlining principles will always be the same.

In your case, you want to do segmentation. This is often done using a window around the pixel you want to classify. Popular choices are 3*3 or 5*5 pixels. The choice of your considered window will determine the number of nodes in your input layer. Then you want to classify them as one of two classes, thus you need 2 output nodes. You can also use just 1, but I don't recommend it, I can expand on this if you care.

One caveat of NN is that you will need quite a bit of training data. So get ready to classify a lot of pixels manually.

How to know how many layers in hidden layer? How to know how many nodes per layer in the hidden layer?

In general, for simple operations like the one you are trying to learn you do not need to have multiple layers. One hidden layer should be enough, at most 2. But, how do you determine the number of nodes you should use? You need to use some model validation techniques to do this. One way is through grid search and cross-validation. Train, and re-train your model with multiple number of nodes and then compare their performances to identify the optimal number of nodes. To get good results this does require a large dataset.

Rule of thumb: 1 hidden layer for NN! Don't get dragged into deep models if you don't need them.


Why can't my implementation of the Actor-Critic algorithm achieve good results in the 2048 game?

Interesting project. First thing I'd do is normalize your state by the maximum cell value. This way you can represent multiple situations at once (eg A grid of all 4s and 8s would look the same as a grid of all 16s and 32s).
Also making the reward = max_cell/2048 might do better as ActorCritic methods seem to do better with rewards within 0-1.

Another reward setup is giving +1 per timestep. It's simple but it also means that maximizing the time to stay alive is the best, which is what I end up doing most of the time when I play anyway.

Good luck!


What are other examples of theoretical machine learning books?

Pattern Recognition And Machine Learning is a great theoretical book. I don't know anything better on standard ML. I read several pages from it myself and all my colleagues researchers suggest to look there if you are not sure about some concepts. The 2 problems with it are that it's huge and it doesn't cover almost all deep learning models known for today.
So, in addition, I'd suggest you look at Deep Learning by Ian Goodfellow et al.
Your concerns about not studying topology, measure theory and group theory are groundless. These sections of math aren't prerequisites in any way, they aren't even discussed anywhere I know.
Actually, ML theory is more like probability theory and statistics. Especially, statistical learning theory (which is nothing more than probability theory and statistics). I haven't read any books on SLT so have a look at this answer.


How does the uniform-cost search algorithm work?

Uniform Cost Search is also called the Cheapest First Search. For an example and entire explanation you can directly go to this link: Udacity - Uniform Cost Search.

In this answer I have explained what a frontier is. To put it in simple words you can describe UCS algorithm as 'expanding the frontier only in the direction which will require the minimum cost to travel from initial point among all possible expansions' i.e. adding a point on the graph (which can be reached from the frontier without going through any other point) which has the shortest route from the initial point. We keep on doing this until a path has explored the goal frontier, this path is the cheapest path from the initial point.

I strongly suggest you check out both the links for examples and better understanding.


Is there any specific SW framework, libraries or algorithms (supported by any theory) designed for implementing a practical AGI system?

Not to my knowledge. The problem is that this is such an enormous task, it cannot really be tackled at once. So the obvious solution is to reduce the scope. In early AI people were using toy domains, whereas nowadays AI systems work more generally (but still perform better if the domain is restricted).
So while (slow) progress is being made putting the individual building blocks together, we're still a long way off building an overarching general system from them.


What are the standard problems for CNNs and LSTMs?

It's more domain- or task-specific. There is no obvious baseline anymore because these models and this field has evolved into too large of an ecosystem. Nonetheless, I'll list a couple of notorious examples below.

Image classification:

MNIST
CIFAR
ImageNet

Detection/segmentation:

PascalVOC
COCO
CityScapes

Pose estimation:

MPII
LEEDS

Text classification:

IMDB
yelp

Question answering:

SQuAD

Translation:

WMT
IWSLT

This is just a taste. There are tons more both in each category and the number of categories, a good source is the Papers with Code website.

Therefore, there is no single standard problem, given that there are too many that all in one shape or form use CNNs or RNNs (or others).


Why do action recognition algorithms perform better on ucf101dataset than HMDB51 dataset?

It is true that at first look, one could expect that classification between 101 categories would be harder than classification between 51 categories. However, many aspects play a role when it comes to action recognition applications.

For instance, the HMDB51 contains several categories about different facial movements like smiling, laughing, chewing,... and several other categories like eating, drinking. Such categories are not present in the list of categories of the data set UCF101 and are obviously among the most difficult categories to deal with. It also claims to have some bad quality challenging videos.

It is hard to predict in advance how difficult a data set will be to classify. We can imagine that when the state-of-the-art reach accuracy beyond 90%, it is time to build a data set that makes these methods fail to look for even more robust solutions. I don't know these data sets well, but the videos are present most probably more variability in terms of viewpoint, camera motions, illumination changes, image quality,... in the most difficult to classify data set.

Also, check on this page, the results announced for the UCF101 data set. I don't know where you found you accuracy value because the official website announces less than 43.9%. Some publications do not use the complete data set and use only part of it to show the performance of an approach they designed.

Finally, the official website of the HMDB51 data set reports the following:
"The UCF group has also been collecting action datasets, mostly from YouTube. There are UCF Sports featuring 9 types of sports and a total of 182 clips, UCF YouTube containing 11 action classes, and UCF50 contains 50 actions classes. We will show in the paper that videos from YouTube could be very biased by low-level features, meaning low-level features (i.e., color and gist) are more discriminative than mid-level fears (i.e., motion and shape)."
This could also explain why better results can be achieved...


How can neural networks deal with varying input sizes?

Three possibilities come to mind.

The easiest is the zero-padding. Basically, you take a rather big input size and just add zeroes if your concrete input is too small. Of course, this is pretty limited and certainly not useful if your input ranges from a few words to full texts.

Recurrent NNs (RNN) are a very natural NN to choose if you have texts of varying size as input. You input words as word vectors (or embeddings) just one after another and the internal state of the RNN is supposed to encode the meaning of the full string of words. This is one of the earlier papers.

Another possibility is using recursive NNs. This is basically a form of preprocessing in which a text is recursively reduced to a smaller number of word vectors until only one is left - your input, which is supposed to encode the whole text. This makes a lot of sense from a linguistic point of view if your input consists of sentences (which can vary a lot in size), because sentences are structured recursively. For example, the word vector for "the man", should be similar to the word vector for "the man who mistook his wife for a hat", because noun phrases act like nouns, etc. Often, you can use linguistic information to guide your recursion on the sentence. If you want to go way beyond the Wikipedia article, this is probably a good start.


Why is the effective branching factor used for measuring performance of a heuristic function?

As you found $N$ is the number of nodes that are expanded. The cost of expansion of each node is equal to the number of children of that node. Hence, we use $b^*$ for each node. In other words, the total number of nodes that are involved in the expansion process is $N \times b^*$.


The regularization terms are applied to the loss functions by default.
However, their gradients do appear in the update step as the gradient of loss appears in the update step.


Is it okay to think of any dataset in artificial intelligence as a mathematical set?

It's true that your original dataset can contain duplicates, so it should not be called a set, in order to be consistent with the mathematical definition of a set. There are mathematical objects known as multi-sets that can contain duplicates, but the order of the elements is still not relevant. There are also tuples and sequences, where the order of the elements matters.
If you want to get rid of the duplicate elements in your dataset, you could perform a pre-processing step where you remove them. Even if you do that, it is often the case that, if you are learning with mini-batches (i.e. using mini-batch stochastic gradient descent), these mini-batches could contain the same elements, because you may sample the same element in different batches or even in the same batch (this is known as sampling with replacement). Of course, this depends on how you sample your training dataset to build the batches (or mini-batches). So, if you do not want duplicates even in the mini-batches, you need to perform sampling without replacement.
Moreover, there are datasets that contain elements whose order in the dataset can be relevant for the predictions, such as datasets of time-series data, while, in mathematical sets and multi-sets, the order of the elements does not matter.
So, yes, it is often called a dataset (or data set), but it is not necessarily a set in a mathematical sense. In general, it should just be interpreted as a collection of data. In scenarios where the order of the elements or the existence of duplicates in the dataset (or any other information or property of your collection of data) is relevant, you should probably emphasize/note it.


How can I implement a GAN network for text (review) generation?

As @Clement mentions, text_gen_description gives a good overview!, but the paper seqGAN paper describes the REINFORCE approach more in depth, as they are the first to do it (i believe). This is probably the approach most take now of days when going the GAN route.

Note that just basic MLE training has shown promise with openAI's GPT2. When i need a text generator, fine tuning one of the provided models is usually my goto.

Also if your looking for seq gans code base (you asked for example code) here is is: git repo

Good Luck!


What is the "Hello World" problem of Reinforcement Learning?

MNIST (along with CIFAR) may be the "Hello World" of supervised learning for image classification, but it is definitely not the "Hello World" of all machine learning techniques, given that RL is also part of ML and MNIST is definitely not the "Hello World" of RL.
I don't think there is a single "Hello World" problem for RL. However, if you are looking for simple problems (or environments) that are usually used as baselines to assess the quality of RL agents, then I would say that the simple grid worlds where you need to move from one place to the other, the CartPole, MountainCar, Pendulum or other environments listed here are often used.
The environment that you choose to train and test your RL agent depends on your goals. For example, if you designed an algorithm that is supposed to deal with continuous action spaces, then an environment where you can take only a discrete number of actions may not be a good option.
The mentioned environments are very simple (i.e. toy problems). In my opinion, we need more serious environments that can show the applicability of RL to other areas other than (relatively simple) games.


What to do if CNN cannot overfit a training set on adding dropout?

Sorry if this is a bad use of answer to add comment but since my reputation is not high enough this is only way to leave a comment to OP's question.
I think some of the answers misunderstood the OP's intention.
Over fitting is used as a means to test the complexity of the model - if a model cannot overfit a small dataset then it's likely not able to generalize well.
It's not that OP misunderstood the meaning of over fitting.
For Instance, I think this discussion is relevant: https://stats.stackexchange.com/questions/492165/what-to-do-when-a-neural-network-cannot-overfit-one-training-sample


How to prevent image recognition of my dataset with neural networks and make it hard to train them?

If the model is trained and held constant, then there are so-called adversarial attacks to modify images such that the model classifies them incorrectly (see Attacking Machine Learning
with Adversarial Examples).
However, if you want to make images that are untrainable, you are probably out of luck. Deep neural networks can learn to recognize even random images with random labels (see Understanding deep learning requires rethinking generalization), though if there's no rhyme or reason to the randomness, they won't generalize in meaningful ways.


Multicamera Tracking vs Single Fisheye Camera

Fisheye camera is always worst. Both convolutional networks object detectors and feature-based object detectors rely on the "isometry" of planar image - lack of strong distortions. Multiple camera have added benefit of several independent sources of information - ensembling. If each camera processed by separate network that may help in verification of objects by voting. With very small objects no simple methods would help. Multiple camera may help a little if all camera output processed together (stacked) as single network input, but don't hold much hope - detecting few pixel objects is very difficult, if possible at all.


For good or bad, AI is the next step in automation. The impact which is already visible, and trends show will continue in the future, is the eradication of repetitive and body-straining labor.
Hopefully, the transformation will be gradual enough for the global labor market to re-adjust, otherwise, we'll face a problem of growing unemployment. It seems to me that we've become aware enough to foresee bad outcomes of our inventions, hence in almost every dimension affected by AI, a plausible and either positive or negative future can be presented, depending on the sentiment of the storyteller.
Regardless of what different experts and sci-fi writers tell us about the future, actually predicting it is a futile endeavour. Considering that predictions made for a dynamic system, even when we have a lot of data and good models (like about the weather), become unreliable just for a few weeks ahead.


I came across a news article from 2018 where the president of India was saying that Sanskrit is the best language for ML/AI.

A very interesting statement indeed! Globally, there is very little interest in NLP for Sanskrit compared to very many other languages.

  Especially given the fact that humans still have to first learn Sanskrit in case NLP is done in Sanskrit.

Most people would say that NLP tools are meant to cater to the needs of humans, instead of requiring humans to learn a new language to be able to benefit from tools (which is a huge barrier).

To answer your question more clearly, it is not that the research community selects languages for research based on "suitability" of the language. Instead, in general the intention of NLP research is to grapple with problems that are relevant, impact many people - or are funded (unfortunately :-).


What is the difference between feature extraction with or without data augmentation?

There are two ways that you could perform data augmentation:

Up front, by expanding the input dataset into a larger one, performing a range of changes to each input then storing the result. This appears to be what you are suggesting.

Just in time, by sampling from possible augmentations on each epoch, or even per sample when building a mini-batch. This appears to be what Chollet is suggesting.

Chollet's approach allows for augmentation to include finer degrees of augmentation that are different each time an input is considered, e.g. rotations of any angle, selecting a slightly different area from a larger image each time. For your approach you could consider the same set of augmentations but they would have to be "frozen in" at the time of building a dataset, and you would not be able to consider all possible variations for each image because it would make the dataset too large.
Both approaches are valid, and both would be called data augmentation. Chollet's approach obtains better re-use of image samples in the long term, and may result in better generalisation in the final trained network. Your approach may allow for more efficient use of CPU time to obtain a result that passes a threshold in accuracy. In some cases the difference between approaches may be minor compared to that caused by other changes in hyperparameters.


What AI concept is behind the Mars Exploration Rover (MER)?

The Mars Rover is a highly successful example of the 'New AI' that emerged from work by Rodney Brooks in the 1990s.

In a quote from Brooks:

  In 1984 I joined the faculty at MIT where I have been ever since. I set up a mobile robot group there and started developing robots that led to the Mars planetary rovers.

Together with the 'Allen' paper, the foundational AI articles in this area are:

"Elephants don't play chess"
"Intelligence without representation"

Although Brooks initially had difficulty getting this work published, preprints were widely circulated within the AI community. Brook's "Physical Grounding Hypothesis" (essentially: "intelligence requires a body") has now largely supplanted the preceding symbolist approach.

The capabilities of the MARS Rover are organized in a Subsumption Architecture. Rather than maintaining an integrated and complex 'world model', increasingly sophisticated behaviors are stacked in hierarchical layers. For example, 'walking' is a relatively low-level competence, with 'avoiding obstacles' and 'wandering around' being higher-level ones.

Each layer is represented by a Finite State Machine that reacts to stimuli appropriate to that level. The activity of lower levels can be suppressed ('subsumed') by higher level ones.

Here is a schematic of the bottom two layers of 'Allen', Brook's first subsumption robot:


How to convert string to number and number to string efficiently?

What are you trying to achieve?

If you need to encode it to some integer use hash table. If you are using something like linear regression or neural network it would be better to use dummy features (one-hot encoding). So for your dictionary of 5 words ("America", "Brazil", "Chile", "Denmark", "Estonia") you get 5 features (x1, x2, x3, x4, x5) which indicate if some word is equal to one in dictionary. So "Brazil" is represented by (0,1,0,0,0), "Germany" is (0,0,0,0,0). Number of features grows with number of words in dictionary making some features practically useless.

If you are using decision trees you don't need to convert string to integer unless specific algorithm asks you to do so. Again, use hash table to do it. In R you can use factor() function.

If you convert your string to integers and use it as single feature ("America" - 123, "Brazil" - 245), algorithm will try to find patterns in it by comparing numbers but may fail to recognize specific countries.


Is there a tool to convert from the brat standoff format to CoNLL-U format?

The brat annotation tool has provided some useful scripts for converting the annotations format including standoff format to CoNLL. Please see this source code from brat GitHub repo for converting the .ann and .txt inputs (standoff format) to .conll file: https://github.com/nlplab/brat/blob/master/tools/anntoconll.py


How can we design the mutation and crossover operations when the order of the genes in the chromosomes matters?

If I understood correctly, your problem is about finding the optimal way to execute a series of tasks in order to maximize the results, using Genetic Algorithms.

In few words, you're trying to solve the salesman problem.

If I am correct, you're looking for Crossover and Mutation algorithms that allow you to work with ordered sets of elements. For these scenarios you usually go for the classic PMX
(Partially Mapped Crossover) and Interchange Mutation. But, there are plenty of other crossover algorithms you can use OX1, OX2 (both variants of the Order Based Crossover), Shuffle Crossover, Ring Crossover, etc. Let's start from the mutation, that is easier.

For simplicity I'll represent the ordered genome like an array of integers: int[] genome = {1, 2, 3, 4, 5};

Interchange mutation

The concept is pretty basic: to mutate an ordered genome you just swap two elements. Easy.

    public int[] InterchangeMutation(int[] genome)
    {
        int i1 = random.Next(0, genome.Length);
        int i2 = random.Next(0, genome.Length);

        var copy = genome.ToArray(); //just making a copy here
        copy[i1] = genome[i2];
        copy[i2] = genome[i1];

        return copy;
    }

PMX Variation Crossover

This is a bit more complicated as we have to take repetitions into account. From experience, I like to use this variation of the Partially Mapped Crossover. It is way easier to implement than the original one (you can find the paper online) but it will cost some more computational complexity. Longer the genome, higher the price you will pay.

Start by selecting two parents to use for the crossover.
From the first parent (P1) select a random section that will be passed over.
For the remaining values:
3A. If they are not in the copied section, take them from P2
3B. If they are in the copied section, take them from P1
3C. End up filling the gaps with the missing values in the order they are in P1

 public int[] PMX2Crossover(int[] P1, int[] P2)
    {
        //Initializing child genome
        int[] child = new int[P1.Length];
        for (int i = 0; i < P2.Length; i++) child[i] = -1;

        //Step1: getting random section to copy over
        int i1 = random.Next(0, P1.Length);
        int i2 = random.Next(0, P1.Length);

        //Step 2: Copying over section from P1
        for (int i = Math.Min(i1, i2); i < Math.Max(i1, i2); i++) child[i] = P1[i];

        //Step 3A: Copying values from P2
        for (int i = 0; i < P2.Length; i++) if (child[i] ==-1 && !child.Contains(P2[i])) child[i] = P2[i];

        //Step 3B: Copying values from P1
        for (int i = 0; i < P2.Length; i++) if (child[i] == -1 && !child.Contains(P1[i])) child[i] = P1[i];

        //Step 3C: Copying remaining values from P1
        int emptyGene = child.IndexOfFirst(-1);
        while (emptyGene != -1)
        {
            child[emptyGene] = FirstMissingGene(P1, child);
            emptyGene = child.IndexOfFirst(-1);
        }

        return child;
    }

    private int FirstMissingGene(int[] parent, int[] child)
    {
        foreach (var gene in parent) if (!child.Contains(gene)) return gene;
        return -1; // should never get here
    }

You can lower down the complexity of the crossover to O(n) (from O(n*n)) simply using a hashmap that keeps track of the genes already added to child.

To get the first child call PMX2Crossover(P1, P2); and for the second just swap the parents PMX2Crossover(P2, P1);

Hope this helps you.

Source: I have been a bachelor professor of AI for a period.


Can neural networks be used to find features importance?

This should be possible, considering universal approximation theorem you should be able to build a ann that approximates features that gives the most likely best feature set for a different net to train on. I would us a rnn for with a softmax output layer that ranks features by performance.

You can find a good explanation of softmax here: https://developers.google.com/machine-learning/crash-course/multi-class-neural-networks/softmax
basically it will assign probability values for each output node with all of these values adding up to 1.0


How to (theorically) build a neural network with input of size 0?

Whether a neural network has learned anything or not, it is a function that maps some input to an output. Training is the process of tweaking the weights so that the output is something that we want. Thus there is always in input of some sorts.

The problem you have presented, of generating even numbers, is much like a Generative Adversarial Network (GAN). In a GAN there are 2 networks: a Generator that tries to generate a sample from a target distribution and a Discriminator that tries to tell real samples from fake samples. The classic analogy being a criminal making counterfeit money and a copy trying to tell what is real money or not.

The generator input is usually a random number (or a matrix of random numbers). The generator then learns to transform a particular random input to a particular point in the target space.

So to answer your question, no there can't be a neural network with 0 inputs as there must always be an input of some kind. Even if the network was to generate a sequence instead of one instance, it would still need something to start with.

For your example, there would have to be some input for the  network to start with. A really simple NN that could solve your problem might look like:

_input = [RandomInteger()]
neuralNetworkWeights = [2]
result = _input * neuralNetworkWeights
result is always even


How will the input be preserved as we go deeper in CNN, where dimensions decrease drastically?

You can also think of a convolutional neural network (CNN) as an encoder, i.e. a neural network that learns a smaller representation of the input, which then acts as the feature vector (input) to a fully connected network (or another neural network). In fact, there are CNNs that can be thought of as auto-encoders (i.e. an encoder followed by a decoder): for example, the u-net can indeed be thought of as an auto-encoder.
Although it is (almost) never the case that you transform the input to an extremely small feature vector (e.g. a number), even a single float-pointing number can encode a lot of information. For example, if you want to classify the object in the image into one of two classes (assuming there is only one main object in the image), then a floating-point is more than sufficient (in fact, you just need one bit to encode that information).
This smaller representation (the feature vector) that is then fed to a fully connected network is learned based on the information in your given training data. In fact, CNNs are known as data-driven feature extractors.
I am not aware of any theoretical guarantee that ensures that the learned representation is the best suited for your task (probably you need to look into learning theory to know more about this). In practice, the quality of the learned feature vector will mainly depend on your available data and the inductive bias (i.e. the assumptions that you make, which are also affected by the specific neural network architecture that you choose).


Would a general-purpose AI need to collaborate?

In my answer, I have often switched between AGI and ASI for reference. This is fine as an AGI will reach ASI as it is optimizing itself and learning.
I think it is not only important by necessary that AGI and ASI are of collaborative nature. Nick Bostrom, in his book Superintelligence: Paths, Dangers, Strategies in Chapter 10 described three ways in which an ASI might function:

  As an oracle, which answers nearly any question posed to it with accuracy, including complex questions that humans cannot easily answer—i.e. How can I manufacture a more efficient car engine? Google is a primitive type of oracle.
      As a genie, which executes any high-level command it’s given—Use a molecular assembler to build a new and more efficient kind of car engine—and then awaits its next command.
      As a sovereign, which is assigned a broad and open-ended pursuit and allowed to operate in the world freely, making its own decisions about how best to proceed—Invent a faster, cheaper, and safer way than cars for humans to privately transport themselves.

This is when and if AGI and ASI are of controlled manner and their output are as expected. By controlled I mean they don't start seeing human as a threat and start eliminating the human race. More on this here. As you can see, all the above ASI are of collaborative nature. Either they are collaborating with humans or they need to collaborate with other systems.

Now to your answer:
First, collaborative nature is of great use in terms of efficiency and performance. This is the reason why Distributed Systems are being made. We even have distributed OS now. Also, Modular approach in coding/ developments and huge success of Object Oriented Model are proofs of advantages of using collaboration among different entities.
If you think about it even the AGI is collaborating and using resources from other places in some way. As your AGI is learning, it is gaining information from the internet. It reads the information and tries to structure is accordingly (This will depend on its neural schema) and create knowledge (or something valuable) for itself. It is collaborating using network protocols with the outer world (other systems). If it doesn't collaborate then the firewall of the system might not allow it to use the service. Different services on the internet require a different set of protocols to be followed. So, if AGI wants to communicate for information it needs to follow those protocols. This way the AGI will learn to collaborate with different entities even at an early stage.
From AGI point of view communicating a web server and communicating another AGI machine is very similar. As we, humans, don't store all the information in our brain; similarly AGI won't find it efficient to store all the information within. Not all information is needed all the time. The memory hierarchy is proof of that. Even if AGI is made to store all the information within, with time it will figure out how inefficient it is and will re-program itself to only hold very vital information and use the internet for less frequently used information.
I will like to add one more thing to this. Let us begin with the human analogy first. We, as humans, collaborate with other humans. But we also collaborate with our different body parts. Like, what about the collaboration within us. Collaboration (or coordination) between our legs, hands, our body, and mind, etc. This leads us to the question of who as a person we are, our whole body, or just our brain. If I remove my hand and replace it with an organic implant, would I be still me?
Similarly, what is AGI? Is it the whole structure, or just the code. Does the RAM, ROM, hard disks also a part of AGI? If you think the hardware is not the part of AGI then your AGI is co-ordinating with these devices too, using certain protocols. To some level, this is collaboration too.


If vanishing gradients are NOT the problem that ResNets solve, then what is the explanation behind ResNet success?

They explained in the paper why they introduce residual blocks. They argue that it's easier to learn residual functions $F(x) = H(x) - x$ and then add them to the original representation $x$ to get hidden representation $H(x) = F(x) + x$ than it is to learn hidden representation $H(x)$ directly from original representation. That's the main reason and empirical results show that they might be right. Better gradient propagation might be an additional bonus but that's not why they originally introduced the idea.
"Normal" networks work too but, at some point, they become too deep and start working worse than shallower versions (they empirically showed that in the paper). Again, they argue that the reason for that might be that at deeper layers hidden representations become approximately similar $H_n \approx H_{n+1}$ because representation is already well learned and you only need some slight adjustments.  That would mean that transformation for deeper layers is similar to identity transformation and that ordinary layers might have trouble learning that, while for residual blocks it would be easy to learn slight modification and add that to the already existing representation from the previous layer.


What exactly is the eigenspace of a graph (in spectral clustering)?

In spectral clustering we not find the eigenvectors of a graph (a graph is not a matrix) but the eigenvalues/eigenvectors of the Laplacian matrix related to the adjacency matrix of the graph:
graph => adjacency matrix => Laplacian matrix => eigenvalues (spectrum).
The adjacency matrix describes the "similarity" between two graph vertexs. In the most simple case (undirected unweighted simple graph), a value "1" in the matrix means two vertex joined by an edge, a value "0" means no edge between these vertex.
So, the space under the adjacency matrix is the space of connectivity, being row "i" of a column vector a measure of the connectivity with vertex "i". In other words, the adjacency and Laplacian matrix map from vertexs to vertex connectivity.
Example
Assume a simple graph with 3 vertex {1,2,3} and edges (1,2) and (2,3). The respective Laplacian matrix is:
$$
A=\begin{pmatrix}
1 & -1 & 0\\
-1 & 2 & -1\\
0 & -1 & 1
\end{pmatrix}
$$
a) vertex 1, than in vertex space is (1,0,0) maps to:
$$
A\begin{pmatrix}
1\\
0\\
0
\end{pmatrix}
=
\begin{pmatrix}
1\\
-1\\
0
\end{pmatrix}
$$
if we analyze the product result, component by component, it means:

vertex 1 is connected to 1 node.
vertex 2 is connected to vertex 1
vertex 3 is not connected to vertex 1.

b) the set of vertexs 1 and 2, that is represented in vertex space as (1,1,0), maps to:
$$
A\begin{pmatrix}
1\\
1\\
0
\end{pmatrix}
=
\begin{pmatrix}
0\\
1\\
-1
\end{pmatrix}
$$
meaning that:

vertex 1 is internal or external to the set {1,2}, not frontier (in this concrete case, it is internal: belongs to set and has no edge with any node out of the set).
vertex 2 is a vertex in the set and connected to one vertex out of the set (internal frontier).
vertex 3 is a vertex not in the set but connected to it (external frontier).

Finally, see what happens if multiply (inner/scalar product) previous result by the vertex vector again:
$$
\begin{pmatrix}
1 & 1 & 0
\end{pmatrix}
A\begin{pmatrix}
1\\
1\\
0
\end{pmatrix}
= 1
$$
it gives the number of edges that connects the set of nodes {1,2} with the remainder graph.


Why does 'loss' change depending on the number of epochs chosen?

You are using two optimisers here: Stochastic Gradient Descent (SGD) and Adam (which is a more complex variant of SGD).

So the "Stochastic" part means that it's random.

The stochastic gradient descent works by taking a smaller random part of the training data, called "mini batch", and back propagates (trains) on this. Doing this until the entire dataset is processed once is often called one epoch*.

This is how gradient descent works in a nutshell: Imagine you're going down a U-shaped hill. You're pretty far down in the U-shape, and you want to go further down by jumping. You figure out what direction is "down" for you: and then you jump. But darn it: you jumped too far and you ended up further up on the other side of the U!

That is just a simple example. You are probably working in WAY bigger dimensions, which complicates this analogy a bit.

Anyway this results in the effect that the loss might go up from time to time when you train another epoch. If you are training a lot of epochs and the loss keeps going up, you should check the learning rate (which basically decides how big a "jump" is).

Hope it helps :)

*: There are other ways of defining an epoch, but it all goes in variants of this.


What are the exact meaning of "lower-order structure" and "higher-order structure" in this paper?

Low order/low level information refers to the most granular level of information. This is the most informative in terms of volume of information, but it can often be difficult to conceptualise for humans.

High order/high level information refers to abstractions of the low level information to more intuitive but less easy to describe technically concepts.

An example would be images of faces. The low level information might be the raw $x, y, z$ values of the pixels: their position and colour value. Some high level information might be the direction the face in the images is facing, from what direction the lighting in the image is coming etc.

In the paper cited, the example they use is low level information (node and edge values); high level information (motifs).


What kind of algorithm is used by StackGAN to generate realistic images from text?

The paper StackGAN: Text to Photo-realistic Image Synthesis
with Stacked Generative Adversarial Networks should provide the answers to your questions.
Here's an excerpt from the abstract of the paper.

Synthesizing photo-realistic images from text descriptions is a challenging problem in computer vision and has many practical applications. Samples generated by existing text-to-image approaches can roughly reflect the meaning of the given descriptions, but they fail to contain necessary details and vivid object parts. In this paper, we propose stacked Generative Adversarial Networks (StackGAN) to generate photo-realistic images conditioned on text descriptions. The Stage-I GAN sketches the primitive shape and basic colors of the object based on the given text description, yielding Stage-I low resolution images. The
Stage-II GAN takes Stage-I results and text descriptions as inputs, and generates high resolution images with photorealistic details. The Stage-II GAN is able to rectify defects and add compelling details with the refinement process.


Are there OpenAI Gym continuing environments (other than inverted pendulum) and baselines?

I'm not sure why you need a continuing environment, but actually you can make most (if not all) OpenAI Gym environments continuing. When you perform a step, you receive information about the next state, the reward, a termination signal and a dictionary with additional information. Simply ignore the termination signal if you want the episodes to continue indefinitely. In some cases you will need to modify a variable of the environments called $\texttt {_max_episode_steps}$, which may force the simulation to stop or to restart.

About your second question, check a resource called Spinning Up, from OpenAI too. They explain several methods and their implementation.


How to formulate discounted return in cartpole?

It's not true that finite horizon MDP can't use a discount factor. Any discount factor $\gamma \leq 1$ is fine for finite horizon.
Whether or not cartpole is a finite or infinite horizon depends on the environment implementation. The default environment in openai gym uses $T=200$ as the horizon. This is customizable, and most papers I see which use cartpole as a test environment use $T=1000$ instead. The task is considered "solved" when the agent can reach $T$ timesteps consistently (e.g., the last 100 episodes have all reached $T$ timesteps).
If you want to treat cartpole as an infinite horizon task, that's fine too. Clearly you need $\gamma < 1$ here, and you can just $T$ to some arbitrary large number.


How can I oppose two AI agents with keras / tensoflow?

Keras/Tensorflow are mostly used of developing/training/deploying neural networks. For descision making problems, if you want to use machine learning, reinforcement learning is in most cases applied. Some reinforcement learning methods use neural networks (and therfore tensorflow) internally. You can check baseline implementations of different methods here.

When using reinfocement learnig, for most of the reinforcement learning methods, you do not need and preexisting data, but you need and environment (esentially a simulations of your game). If you interface your game to one of the baseline agents you can start the agents playing ant training ocurs during they play. You can find a tutorial to start an agent here.

If you only have a feedback (a reward) after a full game of 20 decisions, it will be really hard to train the agents, as you are facing a good example for what is called the problem of spare reward.

There are some ways to deal with sparse reward, like Hindsight Experience Replay and maybe Curiosity, but improving the density of the reward would be very beneficial.


Q-learning agent stuck at taking same actions

I am confused. For the initial $Q$-values, you generate one for each possible row $(1, 0, 0), (0,0,0), \ldots$ so you would have 4 states.
However, from the first paragraph it seems that the states themselves are matrices (one row for each user), so the state space is a set of such matrices.
That means that your $Q$-table should have a row for each possible matrix, and a column for each possible total assignment of items to users.


Is a team of ML scientists an "intelligent agent"?

Is there a term for the humans who do [machine] learning?

Typically you will see "AI researchers" for people studying machine intelligence in general, or "data scientists" for people working with statistics or studying specific solutions in machine learning. Both those terms are used quite flexibly, and generally understood to be scientists/engineers that work on machine intelligence problems. There are also other specialisms that might apply, such as "computer vision researcher".
If you are writing only about researchers who are focused mostly on learning systems - such as new types of learning model, or the best way to train a model for some group of tasks - then "machine learning researchers" or "ML researchers" would be fine. If they are specific people, you may want to ask them directly what title they prefer, in case you accidentally misrepresent their role and work.

Can we call a "team of machine learning scientists with their computers working on some ML problem" an intelligent agent?

No. That might be confused with something that the team is working on or creating. An "intelligent agent" can be the product of specific AI research. For instance, it is reasonable to consider AlphaZero to be an "intelligent agent" (not everyone would call it that, it depends on context, but not relevant to your question).

Is "cognizer" a right term?

No. That is an obscure term meaning something like "a being that perceives or knows". No-one will naturally associate that word with a team of researchers and their computers.

I just want a shorter term for their role in cognition, learning.

I think the most usual way to do this is to describe the team, and name them. For example:

"The MIT computer science researchers are using the X1000 system to investigate  learning of abstract spaces. The MIT team said yesterday . . ."

"Professor Hilary Smith is working with a team of graduates on the Learny McLearnface framework. Smith's team have discovered that . . ."

This is the normal way that you see a research team introduced and referred in science journalism in e.g. New Scientist or Scientific American.
If you need to describe some abstract approach about a team working in general, then you can use a similar approach, naming your teams according to how you need to differentiate between them.


What's stopping Cepheus from generalizing to full poker games?

The reason why Cepheus can't generalize has to do with the number of decision points.

The same authors recently let loose Deep Stack (DeepStack: Expert-Level Artificial Intelligence in Heads-Up No-Limit (HUNL) Poker) which is freaking many professional poker players out.

In the DeepStack arxiv paper, they say

  AI techniques (Cepheus) have previously shown
  success in the simpler game of heads-up limit Texas hold’em, where all bets are of a fixed size resulting in just under 10^14 decision points.

...

  The imperfect information game HUNL is comparable in size to go, with
  the number of decision points exceeding 10^160

...

  Imperfect information games require more complex reasoning than similarly sized perfect
  information games. The correct decision at a particular moment depends upon the probability
  distribution over private information that the opponent holds, which is revealed through their
  past actions.

Using the same strategy for HUNL as Cepheus did is out of the question. Rather, taking an educated guess or using intuition based on the previous play (referred to as Continual Re-solving in the paper) is a method which can better handle this gargantuan game. For more information check out the DeepStack website.


Does there exist a resource for vetting banned words for chatbots?

I have not found one other than scraping a few pages from Urban Dictionary, I built my list via crowdsourced style and got a number of interesting words I had not considered.

Start with the worst words you can think of, then try slang and accidental or on purpose misspellings of them


I find the chapter on machine learning from Russell & Norvig is a pretty good place to start with SVMs. I think this is Chapter 18?

One way to understand an SVM is as a kind of neural network, but this is not usually an intuitive approach for a beginner (unless your NN knowledge is already quite good).

A better way to understand SVMs is as consisting of three simple ideas rolled into one algorithm. Here's an attempt at a "For Dummies" answer though:

Maximum Margin Classification. SVMs are usually used to find a pattern in a set of data. Often, the data allow an infinite set of possible patterns that are all equally descriptive. For example, maybe The relationship is "Lives within 5 miles of a Coast -> Income High". It's easy to imagine that this pattern is just as good as "Lives within 5.0001 miles of a Coast -> Income High" or "Lives within 4.999 miles of a Coast -> Income High". There might actually be a lot more play than that in the data (e.g. 3 miles might work out too). If all these are equally good, then the maximum margin idea says you should pick the one that's "in the middle" of the data. So maybe all values between 5.5 and 4.8 are equally good. In that case, we might pick 5.15 (in the middle). This example is super simplified. Real world data would have a lot more variables, and the idea of "in the middle" ends up being a little more complex, but this is the intuition. It turns out that finding the maximum margin pattern is easy when the patterns are linear. That is, when they can be represented by drawing straight lines through a plot of the dataset.
Projection into higher dimensions. This one needs a bit of math to visualize. Consider a dataset consisting of a circular pattern (for instance, maybe the pattern is that higher incomes are found in the middle of the city). There is no linear relationship that captures this pattern. That is, you can't draw a straight line through the data, and say something meaningful about all the values on one side or the other. However, if you add a new feature to your data that is the square of the original coordinates, it's easy to find such a pattern. Basically, if you pre-compute "circular" functions of the original data, you can add them to the dataset, and then find a pattern that is a linear function of this new feature. This idea generalizes: if you compute a complex enough function of your original data, and then apply the maximum margin idea, you can learn any pattern you like. The problem is that it's slow: adding more features makes it take longer to find the patterns you want.
The Kernel Trick. The thing that made SVMs useful was the kernel trick: finding the maximum margin didn't depend on anything except the product of the coordinates of the various points. It turned out that this product could be computed first, and then run through certain functions to produce a problem that was identical to the one you'd get by first adding extra features and then doing the multiplication. However, computing the problem this way didn't require adding any new features! This made SVMs one of the first reliable, well understood, and fast methods for finding non-linear patterns in data.

Hope that provides a starting point. Consider reading Russell & Norvig as a next starting point, or Bishop if you want to go deeper.


I remember a problem similar to this  https://vladris.com/puzzles/facebook/puzzle_master/snack/breathalyzer.html
this is given as a problem in facebook engineering puzzles.

But i doubt that it is not much of a machine learning/AI problem. you could implement an algorithm that converts each word into a set of its characters , then pick the word in your master list with minimum distance based on its character list .

Even when humans solve jumble we do it in a systematic/algorithmic way , if the jumbled word is not present in our memory we can't do anything otherwise we can solve the jumble.

But human brain can simply recognize scrabled words if some of the structure is retained and not fully scrambled like an anagram .

We generally index words based on the first syllable/character in our brain , like  or even this

In the second picture we recognised the words even if they are not made of english characters because our brain doesn't scan the text character by character like "7-H " but treats it like an image.

So the model should not immediately classify the segmented characters but should find the "nearest" characters in every class that optimises the combined probability of characters of the word being one of the classes of words we have in our dictionary.


Finding the optimal policy from a set of fixed policies in reinforcement learning

The quickest way to do this would be to use policy evaluation methods. Most of the standard optimal control algorithms consist of policy evaluation plus a rule for updating the policy.
It may not be possible to rank arbitrary policies by performance when considering all states. So you will want to rank them according to some fixed distribution of state values. The usual distribution of start states would be a natural choice (this is also the objective when learning via policy gradients in e.g. Actor-Critic).
One simple method would be to run multiple times for each policy, starting each time according to the distribution of start states, and calculate the return (discounted sum of rewards) from each one. A simple Monte Carlo run from each start state would be fine, and is very simple to code. Take the mean value as your estimate, and measure the variance too so you can establish a confidence for your selection.
Then simply select the policy with the best average value in start states. You can use the variance to calculate a standard error for this, so you will have a feel for how robust your selection is.
If have a large number of policies to select between, you could do a first pass through with a relatively low number of samples, and try to rule out policies that perform badly enough that even adding say 3 standard errors to the estimated value would not cause them to be preferred. Other than that, the more samples you can take, the more accurate your estimates of mean starting value for each policy will be, and the more likely you will be to select the right policy.

I was wondering if meta learning is the right area to look into?

In general no, but you might want to consider meta learning if:

You have too many policies to select between by testing them all thoroughly.

The policies have some meaningful low dimension representation that is driving their behaviour. The policy function itself would normally be too high dimensional.

You could then use some form of meta-learning to predict policy performance directly from the representation, and start to skip evaluations from non-promising policies. You may need your fixed policies to number in the thousands or millions before this works though (depending on the number of parameters in the representation and complexity of mapping between parameters and policy function), plus you will still want to thoroughly estimate performance of candidates selected as worth evaluating by the meta-learning.
In comments you suggest treating the list of policies as context-free bandits, using a bandit solver to pick the policy that scores the best on average. This might offer some efficiency over evaluating each policy multiple times in sequence. A good solver will try to find best item in the list using a minimal number of samples, and you could use something like UCB or Gibbs distribution to focus more on the most promising policies. I think the main problem with this will be finding the right hyperparameters for the bandit algorithm. I would suggest if you do that to seed the initial estimates with an exhaustive test of each policy multiple times, so you can get a handle on variance and scale of the mean values.


Is this proof of $\epsilon$-greedy policy improvement correct?

The weights do sum to one. Note that in the second line where we have
$$\frac{\epsilon}{|\mathcal{A}(s)|} \sum_a q_{\pi}(s,a) + (1-\epsilon)\max_aq_{\pi}(s,a) \; ,$$
the sum is over the whole action space, including the greedy action, so the sum of the weights will be $\frac{\epsilon}{|\mathcal{A}(s)|} \times |\mathcal{A}(s)| + (1-\epsilon) = 1$.


What is the difference between Bayes-adaptive MDP and a Belief-MDP in Reinforcement Learning?

Both Belief-MDPs and Bayes-Adaptive MDPs (BAMDPs) are special cases of POMDPs and their state space is augmented with a belief over their unobserved/hidden variables.
In a belief-MDP, the hidden variables can change over the course of an episode. (Eg. Both the position and the uncertainty in the position of the robot can vary during an episode).
In a  BAMDP, the hidden variables are usually the attributes of the transition/reward function and are held constant during an episode. (Eg. In a robot locomotion task, ground friction or load - dynamics attributes, current goal location - reward function attributes. Though variables defining these attributes are unknown to the agent and the agent has to infer them, the actual variables remain unchanged during an episode)
References:

M Ghavamzadeh, S Mannor, J Pineau, Aviv Tamar - Bayesian reinforcement learning: A survey. Foundations and Trends in Machine Learning, 8(5-6):359-483, 2015.

L Zintgraf, K Shiarlis, M Igl, S Schulze, Y Gal, K Hofmann, and S Whiteson. Varibad: A very good method for bayes-adaptive deep rl via meta-learning. arXiv preprint arXiv:1910.08348, 2019.


Should I use anthropomorphic language when discussing AI?

If clarity is your goal, you should attempt to avoid anthropomorphic language - doing so runs a danger of even misleading yourself about the capabilities of the program.

This is a pernicious trap in AI research, with numerous cases where even experienced researchers have ascribed a greater degree of understanding to a program than is actually merited.

Douglas Hofstadter describes the issue at some length in a chapter entitled "The Ineradicable Eliza Effect and Its Dangers" and there is also a famous paper by Drew McDermot, entitled "Artifical Intelligence meets natural stupidity".

Hence, in general one should make particular effort to avoid anthropomorphism in AI. However, when speaking to a non-technical audience, 'soundbite' descriptions are (as in any complex discipline) acceptable provided you let the audience know that they are getting the simplified version.


how to benefit from previous training weights in training again to increase accuracy?

First, I assume you've tuned your hyperparameters. Because, instead of re-train the network (use the weights that resulted from the previously training process) that needs more times, I'll invest more on hyperparameters tuning of the available network.

Then, there are several methods and considerations:

You can use the weight resulted from your first network as the initial of your next training process. But as the network will face the same data/problem if you use "initial" value for your hyperparameters (e.g. high learning rate) then I'm afraid it will lead to overfitting. Because it's simply "nothing change" in your architecture.
If you previously didn't use adaptive learning for your training process (e.g. Adadelta, Adam), you can re-train your network with a smaller learning rate. So, your model can find a better result.
Or, you can use the concept of the selffer network, you use the weight for some layers from previous training process (you can freeze it or not) and randomly initialize other layers. And then, train the network using "initial" value of hyperparameters

You can read more about the transfer learning concept (or selffer network) to get the most appropriate method for your case. There is also a paper about "incremental training on CNN" that I think is similar with the selffer network but with some modifications.

Hope it helps


Does learning content from additional encyclopedias consume much less amount of storage?

It seems easy for this to be sublinear growth or superlinear growth, depending on context.

If we imagine the space of the complex AI as split into two parts--the context model and the content model (that is, information and structure that is expected to be shared across entries vs. information and structure that is local to particular entries), then expanding the source material means we don't have much additional work to do on the context model, but whether the additional piece of the content model is larger or smaller depends on how connected the new material is to the old material.

That is, one of the reasons why Watson takes many times the space of its source material is because it stores links between objects, which one would expect to grow with roughly order n squared. If there are many links between the old and new material, then we should expect it to roughly quadruple in size instead of double; if the old material and new material are mostly unconnected and roughly the same in topology, then we expect the model to roughly double; if the new material is mostly unconnected to the old material and also mostly unconnected to itself, then we expect the model to not grow by much.


Predicting probabilities of events using neural networks

You can set up a neural network to predict whether there is an event in a randomly picked interval. I.e. if there is an event in this interval in your trainingsdata you train to output a 1 otherwise you train to output a 0.

If you use the quadratic loss function the prediction of the NN should approximate the probability of such an event.

Overfitting can be monitored with by splitting your trainingsdata into train- and testset.

If you train an RNN by inputting the intervals between events, these intervals should be more similar than the exact event times. Modelling a time series like this also makes more sense.

Of course these details depend on what exactly the data is representing. And it is also possible that decent predictions are impossible for this dataset.


What kind of body (if any) does intelligence require?

This is something of an orthogonal answer, but I think Brooks didn't go about his idea the right way. That is, subsumption architecture is one in which the 'autopilot' is replaced by a more sophisticated system when necessary. (All pieces receive the raw sensory inputs, and output actions, some of which turn off or on other systems.)

But a better approach is the normal hierarchical control approach, in which the target of a lower level system is the output of a higher level system. That is, the targeted joint angle of a robot leg is determined by the system that is trying to optimize the velocity, which is determined by a system that is trying to optimize the trajectory, which is determined by a system that is trying to optimize the target position, and so on.

This allows for increasing level of complexity while maintaining detail and system reusability.

That said, I don't think you actually need what one would naively call 'embodied cognition' in order to get the bottom-up hierarchy of competencies that Brooks is right to point towards. The core feature is the wide array of inputs and outputs, which are understood in a hierarchical fashion that allows systems to be chained together vertically. I think you could get a functional general intelligence whose only inputs and outputs involve going through an Ethernet cable, and doesn't have anything like a traditional body that it actuates or senses through. (This is a claim that the hierarchical structure is what matters, not the content of what we use that structure for.)

(The main place to look for more, I think, is actually a book about human cognition, called The Control of Perception by William T. Powers.)


Can you use machine learning for binary data?

Of course you can use AI (specially Deep Learning) in your application. your covariates will be the input to your AI model and the model should predict probability of presence. The model has no problem with binary data and binary data is common in this field.
Also note that 1:100 ratio is not good and the network will probably learn to output absence for any input (this way it gets 99% accuracy but really it's not doing anything). So you should probably balance them (using almost same data, or telling network to pay more attention to presence data (by weighting the related loss)).
I think nowdays you can find Deep Learning in any popular coding language. But most of DL community use python and its really easy to learn.
If you want to learn Deep Learning there are a lot of sources on internet. But I suggest you the Deep Learning courses of Deeplearning.ai in coursera (If you have a lot of time) and CS231n of Stanford university on youtube (If you have time)


Why are there transition layers in DenseNet?

The point of DenseNet was to go as deep as ResNets, if not deeper, and keep multiple skip connections to preserve the gradient flow back better as well as to keep the earlier layers context (which prevents overfitting). With layers as deep as 120, having a single block being fully concatenated to all the previous ones would mean having a way large feature map, which, I guess, would be computationally very expensive and not feasible.

About transition layers (convolution + pooling), I think it's just a way of downsampling the representations calculated by DenseBlocks slowly upto the end as after transition layers the representations go from $56 \times 56$ to $28 \times 28$ to $14 \times 14$, and so on.

The authors state it this way

  To further improve model compactness, we can reduce the number of feature-maps at transition layers


How we are calculating average reward ($r(\pi)$) if the policy changes over time?

You are correct: to evaluate a policy, we need to fix it.

We can temporarily fix it, just to evaluate it over a number of test cases. For a fair comparison, we should fix the start states and random seeds used for the transitions.
We can wait until convergence / until we are satisfied. The resulting policy would be what we implement in the "true", trained agent. This is important when exploration might be harmful in the "real world" domain where the agent will be operating.
We can also measure average reward of the "non-stationary" policy, and assume that, once the agent is doing well, this should be close enough to evaluating the fixed policy. This is not ideal, but on the other hand it is trivial to implement, and is often used to track the learning process. If you have a life-long learning agent, this might be the best you can do.


Such a system can and does solve the problem of hyperparameter tuning. Google's AutoML does this. Here is another example that uses a Genetic Algorithm to breed new neural network structures.
AutoML has been shown to outperform humans in the rate that it improves network designs. It seems to favour Residual Network style topologies.


Is there a way of automatically drawing bounding boxes around interested objects?

How many classes do you want to annotate it?

We can get the bounding boxes and class names coarsely by using the pre-trained models such as YOLO-V3 darknet, SSD and others.

Then, we can load those annotations using the tool labelImg and manually correct it. It reduces the lot of work and it is called the Human-AI labeling technique.


What does the number of required expert demonstrations in Imitation Learning depend on?

The answer to your question can be found in the original paper that introduced the max-margin and projection imitation learning (IL) algorithms: Apprenticeship Learning via Inverse Reinforcement Learning (by Abbel and Ng, 2004, ICML). Specifically, theorem 1 (section 4, page 4) states

Let an $\text{MDP} \setminus R$, features $ \phi : S \rightarrow [0, 1]^k$, and any $\epsilon > 0$ be given. Then the apprenticeship learning algorithm (both max-margin and projection versions) will terminate with $t^{(i)} \leq \epsilon$ after at most
$$n=O\left(\frac{k}{(1-\gamma)^{2} \epsilon^{2}} \log \frac{k}{(1-\gamma) \epsilon}\right)$$
iterations.

Here $k$ is the dimension of the feature vectors, so it's clear that the number of iterations needed for these algorithms to terminate scales with $k$. The proof of this theorem can be found in appendix A of the same paper (and all other terms are defined in the paper, which you should read to understand all the details). Of course, this result holds (only) for these specific IL algorithms (which are the algorithms the author of your slides, Abbel, is referring to). See also theorem 2 and the experiments section (in particular, figure 4, which shows the performance as a function of the number of trajectories) of the same paper.
These slides provide a nice overview of the contents of this paper, so I suggest that you read them too.


Hyper-plane in logistic regression vs linear regression for same number of features

Mm, I understand what you want to say, but I think you're slightly wrong with terminology. If you consider a d-dimensional vector space, people usually a hyper-plane (d - 1)-dimensional, because it's a subspace of lower dimensionality. So for your example d = 2 the logistic regression would fit a 1-dimentional line (which would separate 2 classes) and the linear regression would fit a 2-dimentional plane (for every combination of 2 features the prediction would lay on some plane in 3-D space). So the general rule would be logreg -> (d-1)-dimensional hyperplane in d-dimensional vector space, linear regression -> d-dimensional hyperplane in (d+1)-dimensional vector space.


Can non-Markov environments also be deterministic?

Markov Environment is not about deterministic or stochastic. "Depends only on the current state and your action" does not mean you know what will happen(deterministic).

We can have Markov + deterministic, Markov + Stochastic, Non-Markov + deterministic, and Non-Markov + stochastic.

The definition you have is not a definition of deterministic. It is a definition of Markov property.

Refer to Wikipedia.

  A stochastic process has the Markov property if the conditional
  probability distribution of future states of the process (conditional
  on both past and present values) depends only upon the present state;
  that is, given the present, the future does not depend on the past. A
  process with this property is said to be Markovian or a Markov
  process. The most famous Markov process is a Markov chain. Brownian
  motion is another well-known Markov process.

Markov property is assumed mostly in stochastic problems.
Brownian motion is the motion of molecules of ink in the water and used to model the movement of a stock price, which is stochastic.

Deterministic means when you are in the same state and choose the same action your next state will be always the same.

Stochastic means even you are in the same state and choose the same action, you next state can be different than the previous time.

Example) You toss a coin and roll a die. Every time you roll a die you get pennies as many. If the coin gets head, you get a chance to roll a die twice next time.
Your state can be (money you collect so far, coin head/tail in the previous time).

In this problem, your next state will not be affected by the past. the only thing you need to know is the current state, the money you got and head or tail. It has a Markov process/environment. However, still, it is stochastic because you don't know what will be the next state.


Do I need to use a pre-processed dataset to classify comments?

Here's a list of some of the best python libraries for natural language processing.

Natural Language Toolkit (nltk)
Covers all the basic functions and NLP tools such as tokenization etc.
TextBolb
This is a good library of beginners, it provides the nltk toolkit in a simplified format.
Spacy
It is an advanced library and can be used in production code.

You can preprocess textual data in a number of ways. It depends on the type of task at hand and the size of the data.

From your question, I think you are referring to converting the words to a vector form word2vec. Here is a massive word2vect list from Google.

Also have a look at preprocessing techniques in NLP such as tf-idf etc.


What are the mathematical prerequisites needed to understand research papers on neural networks?

Knowing you want to focus on the theory, I think that a good choice is Deep Learning Book from Ian Goodfellow et al., which is publicly available. It has three main parts. On the first one the author presents the math/ statistic tools that will be needed to understand the following parts. On the second part, the author explains the current state of the art in Deep Learning and on the last part more advanced topics are introduced. The author also uses references so as to facilitate extra resources to dive more into the theory.
On the other hand I strongly recommend you Google Scholar, there you have plenty of information given by articles/ papers of the current state of the art techniques in the field. There you can also find papers related on what that you mentioned about the Hahn-Banach Theorem.


What is the current state of the art in animal facial recognition?

In case of animal classification - trying to specify the animal, trying to tell cat from a dog - it should be similar to human faces. An algorithm such as SVM could be used in that case. Trying to tell two breeds of dog apart would be more challenging. I have come across this article explaining how they use facial recognition software for animals news. And I have also come across  this article where they write about a novel method for identifying individual cetaceans using facial features and symmetry on dolphins.


What is the impact of the number of features on the prediction power of a neural network?

Based on the clarifications given in the comments on the original question, i will try my best to give an answer.
If the number of features increases, does the approximation capability of an ANN improve theoretically speaking?
It depends on whether this additional feature adds 'usefulness' to the data that was already supported. If you add a feature that is already in the dataset, it obviously will not increase the approximation capabilities. If you add a feature that is not yet in the current set of features and does influence the thing you are trying to approximate, then yes! There are some exceptions in this case of course, such as what if features are multicorrelated etc. It is not super important or measurable in ANNs, but statistics has a bunch of theory on this stuff. Simple statistical multiple regression has 4 assumptions and also checks for multicollinearity to see whether it is possible get 'useful' results from the data etc. As ANNs is super general, this is not applicable, but you could argue that similar practices could be looked into when seeking the best possible 'theoretical' validity of the features that you are using.
Does adding more features make your ANNs produce more accurate results (in practice)?
Again, it depends. If you indeed added a useful feature, then the possibility exists that your ANN will in practice be able to produce more accurate results. But if your ANN is not able to converge to a solution, then your results will be gibberish. It is more likely that a network is unable to converge if you just throw data at it. More data requires bigger networks etc. So it is highly dependent on your training method. AKA, if your added feature is useful and the method you use is sufficient, then I'd argue that, yes, adding a feature will most likely make your ANNs produce more accurate results.


Why would an AI need to 'wipe out the human race'?

It's a possible side effect

Any goal-oriented agent might, well, simply do things that achieve its goals while disregarding side effects that don't matter for these goals.

If my goals include a tidy living space, I may transform my yard to a nice, flat lawn or pavement while wiping out the complex ecosystem of life that was there before, because I don't particulary care about that.

If the goals of a particular powerful AI happen to include doing anything on a large scale, and somehow don't particularly care about the current complex ecosystem, then that ecosystem might get wiped out in the process. It doesn't need to want or need to wipe out us. If we are simply not relevant to its goals, then we are made of materials and occupy space that it might want to use for something else.

We are a threat to most goals

Any goal-oriented agent might want to ensure that they can fulfill their goals. Any smart agent will try to anticipate the actions of other agents that may prevent them from achieving those goals, and take steps to ensure that they succeed anyway. In many cases it is simpler to eliminate those other agents rather than ensure that their efforts fail.

For example, my goals may include storing a bag of sugar in a country house so that I can make pancakes when visiting without bringing all ingredients every time. However, if I leave it there, it is likely to get eaten by rats during winter. I may take all kinds of precautions to store it better, but rats are smart and crafty, and there's clearly a nontrivial chance that they will still succeed in achieving their goal anyway, so an effective extra precaution is killing the rats before they get a chance to try.

If the goals of a particular powerful AI are to do X; it may come to an understanding that (some?) humans might actually not want X but Y instead. It can also easily deduce that some of those humans might actively do things that prevent X and/or try to turn off the AI. Doing things that ensure that the goal gets achieved is pretty much what a goal-seeking agent does; in this case if existence of humans isn't strictly necessary for goal X, then eliminating them becomes a solid risk reduction strategy. It's not strictly necessary and it may take all kinds of other precautions as well, but just like in my example of rats, humans are smart and crafty and there's clearly a nontrivial chance that they will still succeed in achieving their goals (so that X doesn't happen as AI intends) so an effective extra precaution could be killing them before they get a chance to try.


I have 5000 html files (structured text), how can I generate a new one that "resembles" those?

You could try to train a recurrent neural net on char level. Basically, you took GRU or LSTM and use a sequence of characters, not tags or words. In the blogpost "The Unreasonable Effectiveness of Recurrent Neural Networks" there are examples for Shakespeare, Linux source code on C and for papers in latex code, and the results are quite valid, produced from similar to your size training sets. The good thing in case of html pages is that modern browsers are very good at handling a slightly broken HTML, so this approach could work for both of your tasks.


In deep learning, the error function is sometimes known as loss function or cost function (but I do not exclude the possibility that these terms/expressions have also been used to refer to different although related functions, so you should take into account your context).
In statistical learning theory, the function that you want to minimize is known as expected risk
\begin{align}
R(\theta)
&=
\int L(y, f_{\theta}(x)) d P(x, y) \\
&=
\mathbb{E}_{P(x, y)} \left[ L(y, f_{\theta}(x)) \right],
\end{align}
where $\theta$ are the parameters of your model $f$, and the function that you actually minimize is the empirical risk given the dataset $D = \{ (x_i, y_i) \}_{i=1}^n$,
$$
R_{\mathrm{emp}}(\theta)=\frac{1}{n} \sum_{i=1}^{n}L(y_i, f_{\theta}(x_i)),
$$
which is a generalization of the commonly used cost functions, such as mean squared error (MSE) or binary cross-entropy. For example, in the case of MSE, $L = \left(y_{i}-f_\theta(x_i)\right)^{2}$, which can be called the loss function (though this terminology may not be standard).
You optimize the empirical risk, a proxy for the expected risk, because the expected risk is incomputable (given that $P(x, y)$ is generally unknown), so, in this sense, the expected risk is unknown.
If you use the term error function to refer to the expected risk, then, yes, the error function is typically unknown, but error function is typically used to refer to an instance of the empirical risk, so, in that case, it is known and computable.
Note that I purposely used the term loss function above to refer to $L$ and the term cost function to refer to the empirical risks, such as the MSE (i.e., in this case, I did not use loss function and cost function as synonyms), which shows that terminology is not always used consistently, so take into account your context.


What are examples of approaches to dimensionality reduction of feature vectors?

Dimensionality reduction could be achieved by using an Autoencoder Network, which learns a representation (or Encoding) for the input data. While training, the reduction side (Encoder) reduces the data to a lower-dimension and a reconstructing side (Decoder) tries to reconstruct the original input from the intermediate reduced encoding.

You could assign the encoder layer output ($L_i$) to a desired dimension (lower than that of the input). Once trained, $L_i$ could be used as a alternative representation of your input data in a lower feature-space, and can be used for further computations.


Detect named entities inside words using spaCy

You could use a tool to decompose composite word. There are many open source softwares for various languages. For example : german-word-splitter


Can we use transformers for audio classification tasks?

Yes, Transformers can be used to work with audio data, such as audio processing (audio classification, speaker identification, etc) (Audio ALBERT), speech-to-text (Streaming Automatic Speech Recognition with the Transformer Model), and text-to-speech (Neural Speech Synthesis with Transformer Network).


Is it fair to compare AlphaGo with a Human player?

Depends on the purpose of the comparison.

If we are comparing ability to win a game of Go, then yes.

If we are comparing learning ability, then maybe. It depends on the task. AlphaGo and systems like it are capable of learning only in well-described limited domains. There may be an analogy with sensory learning (it might even be possible in theory to take a small piece of brain tissue and run an algorithm similar to AlphaGo's learning process on it).

In general, the approach used by AlphaGo and other reinforcement learning successes is "trial-and-error plus function approximation". It seems analogous to perception and motor skills, such as object recognition or riding a bike, as opposed to reasoning skills and games as humans play them, which goes through many more cognitive and conscious layers that have no real analog in a RL system like AlphaGo.

  A human player plays limited games compared to a system that undergoes millions of iterations

This is an advantage of a machine to learn this kind of task. It would equally apply in other simulated environments with simple rules. If your goal is to have the most skilled and optimal navigation of such a domain, the implication now is that you would not train a human expert through years of study, but to write the simulator and train an AlphaGo-like machine.

This is no different a comparison than deciding cars and roads are better solutions to long distance travel for the general population than walking or horses and carts. It doesn't matter what underlies the advantage of one over the other, the assessment is cost/benefit, which resolves to a single comparable number.

It would, however, be wrong to assess AlphaGo as a better general-purpose learning engine than a human. The fact that humans do not have to work fully through millions of simulations in full detail is important. It means that something about how humans learn is still not covered by learning machines. Some of these things are understood and being discussed - such as the ability to focus intuitively on important aspects of what to learn, the ability to reason about the environment, learning analogously or transfer learning from other domains.


Which linear algebra book should I read to understand vectorized operations?

If you already have two years of a bachelor's of mathematics, I recommend part I of the book that you're mentioning. That part of the book reviews the main mathematics used in the optimization of neural nets (in part 1), and then actually goes through the various models in detail in the later parts. The review is done at a level that is suitable for someone who has already studied these topics, but needs a refresher.
The book Matrix Differential Calculus with Applications in Statistics and Econometrics covers more advanced topics, which might also be what you are looking for. There is also the related Wikipedia article.


Are there any ongoing projects which use the Stack Exchange for machine learning?

There certainly appear to have been research projects involving some form of text mining / information retrieval /etc. and StackExchange sites.

Some examples I was able to find through google/google scholar (unlikely to be anywhere near an exhaustive list):

TACIT: An open-source text analysis, crawling, and interpretation tool describes numerous text-crawlers for a variety of sites (including Stack Exchange sites, but also Twitter, Reddit, etc.). At first glance, this appears to be primarily about crawling, not about doing anything else with the data afterwards. Searching for other papers that cite this one on Google Scholar may yield interesting results though, it may lead to papers that used this for crawling and did more with the data afterwards.
Chaff from the Wheat : Characterization and Modeling of Deleted Questions on Stack Overflow describes research into the quality of Stack Overflow questions in some sense (specifically, predicting whether questions will get deleted for example). I'm not 100% sure if this is also the kind of stuff you're interested in; it is Stack Exchange + Machine Learning as implied by the title of your question, but not necessarily about retaining information from answers as implied by the text in your question.
Text mining stackoverflow: An insight into challenges and subject-related difficulties faced by computer science learners also describes text mining in StackOverflow questions and answers, though at a very quick glance it appears to be primarily about topic detection etc., not necessarily about automated question answering for example.
Different Facets of Text Based Automated Question Answering System appears to be a relatively recent survey on the topic of Automated Question Answering research. Stack Exchange is mentioned a few times as an example of a source of data for such systems, but doesn't appear to be used otherwise.
Extending PythonQA with Knowledge from StackOverflow is specifically about incorporating Questions and Answers from StackOverflow in an automated Question and Answering system for questions about the Python programming language. The paper provides a link to more details (http://pythonqas2.epl.di.uminho.pt), but that link appears to be down. I suppose you could always try contacting the authors directly if you're interested in more information on this.

More generally, Automated Question Answering systems appears to be a rather active area of research still, not a trivial / "solved" problem. StackExchange can be one source of data for such systems, but there are plenty of other sources of data too (Wikipedia, Quora, etc.).


Does the quality of training images affect the accuracy of the neural network?

For most of the current use cases, where NNs are used in conjunction with images, the image quality (resolution, color depth) can be low.

Consider image classification for example. The CNN extracts features from the image to tell different types of objects apart. Those features are pretty independent from the quality of the image (in reasonable bounds). Compare it with your own visual experience. Try to reduce the resolution of an image of a car step by step to figure out how little details you need until you can no longer distinguish it from a plane. This is similar to modern CNNs, which can even outperform human vision in some regards.

This changes when small details start to matter. Maybe you need to be able to detect small differences in fur patterns to tell different cat breeds apart. As soon as you lose those details, the detection rate will drop significantly.

So the answer to your question is, it depends. As long as you do not lose the important features of the image, you'll be fine with low resolution.

In case you care about the reason for the low quality of images used in machine learning - The resolution is an easy factor you can manipulate to scale the speed of your NN. Decreasing resolution will reduce the computational demands significantly.

Many CNNs even include pooling layers in their architecture, which artificially reduce the resolution further after certain processing steps. This is usually a good idea as long as you are fine with loosing positional information. You shouldn't do this when teaching the CNN to play a game, because location is highly important, but for image classification this has become an established method to increase performance.