What I notice is that the network's fitness keeps climbing up and falling down again. It seems that my current approach only evolves certain patterns on placing signs on the board and once random mutation interrupts current pattern new one emerges. My network goes in circles without ever evolving actual strategy. I suspect solution for this would be to pit network against tic-tac-toe AI, but is there any way to evolve actual strategy just by making it to playing against itself?

The likely cause of this phenomenon is that your fitness function involves evaluating the fitness of an agent by letting it play lots of other agents (the entire population), many of which are likely very poor agents.

Because Tic-Tac-Toe is such a simple game, we know that optimal play from both sides leads to a draw. Suppose we have a population of the following three strategies:

$\pi_1$: an optimal player
$\pi_2$: a sub-optimal player
$\pi_3$: a different sub-optimal player

There can easily be situations where the optimal player $\pi_1$ gets a draw against both of the sub-optimal players (if they're not extremely bad), because right from the get-go an optimal player will play "safe" enough such that it can still guarantee a draw against an optimal opponent, which may not be the fastest way to win against a sub-optimal player.

In the same situation, the sub-optimal player $\pi_2$ for example may be able to consistently win against a slightly worse sub-optimal player $\pi_3$. In this example situation, your fitness function ranks the agents $f(\pi_2) > f(\pi_1) > f(\pi_3)$, which is wrong.

As you already suggested yourself, the most straightforward way to address this problem would be to simply evaluate the fitness of strategies by having them play against an optimal minimax-agent, rather than a population of many strategies (including poor ones).

If you really want to use only evolving, no tree search, you'll have to find a way to fix the fitness function such that the problem described above can no longer occur. One way you could try to do that (not 100% sure it would work, but imagine that it might) would be to set up some larger tournament bracket where agents progress through the brackets if they're able to beat others which they were paired up with. Getting further in the tournament would then increase the fitness of an agent. Very bad sub-optimal players should not be able to progress far in the tournament if they get beaten by other (sub-)optimal agents, but sub-optimal agents which manage to advance still shouldn't be able to get more than a draw against an optimal agent. Some things to keep in mind with this idea:

You'll likely want to repeat the entire tournament many times with different, randomized initial pairings in the bracket, and compute fitness based on average (or maybe median or max) ranking across all repetitions of the tournament. This would be to filter out members of the population getting particularly lucky or unlucky with the opponents they encounter in the brackets.
You'll have to think about which actions the agents select in such a tournament. Do they deterministically play the action given by the $\arg\max$ of the softmax outputs (in which case you could also just use the linear outputs rather than softmax outputs, since the softmax function does not change ranking of outputs). Or do they nondeterministically sample actions according to the softmax distribution? Sampling actions according to the softmax distribution seems attractive because it leads to more variance in game state situations encountered, and it is important to be robust and be capable of handling many different game states. On the other hand, it does introduce noise and may make an optimal agent accidentally lose. I suppose I would lead towards deterministic play with the $\arg\max$ over the softmax outputs. In a large tournament, there will still be sufficient variety due to encountering many different opponents.
You'll have to put thought into handling the situation where two agents may infinitely keep drawing against each other. Which agent advances? If this happens, I think I would gradually shift from deterministcally playing $\arg\max$ actions to playing according to the softmax distribution. This guarantees that someone eventually loses. Averaging results over multiple different games like this, plus averaging over multiple different repetitions of the complete tournament, should yield accurate results.
You'll want to think about how to handle illegal actions. You describe including punishments for illegal actions in the fitness function. This introduces extra noise / variance in the fitness function which is already difficult enough to get right to begin with, so I wouldn't do this. I'd recommend having a manual post-processing step in which you manually set probabilities of illegal actions being played to $0$, and normalizing to make sure the probabilities over the remaining actions add up to $1$ again. This is also what's done (on a much larger scale) by DeepMind in their state-of-the-art Go/Chess/Shogi agents for example. If you really don't want to do this, if you really want evolved strategies to automatically learn not to ever have high outputs for illegal actions, I would recommend immediately making them lose the game if they do suggest an illegal action. That way you still have a "clean" fitness function based only on wins/draws/losses.


Which is the most successful AI project so far?

I'd say the most successful are the ones so commonly used that we don't even notice them:

The mail systems that automatically decipher handwritten addresses on your packages, they use machine vision and have probably been doing it since mid-90s.
Algorithmic trading bots on stock markets - they handle something like 85% of all trades.
Many modern CPUs use AI techniques, including neural networks, to guess what your program is going to do next and optimize branch prediction and memory fetches.
Most good modern fraud and spam detectors use some combination of AI techniques (clustering, decision trees, SVMs, even some machine vision to check out attached pictures) - and, in the opposing BlackHat camp, the latest automatic CAPTCHA breakers use all the latest advancements in deep learning too).
And of course there's Google, Facebook and US DoD who try to put AI into anything they can think of.


How does the math behind heat map filters work?

About your question concerning ColorMaps:
A cv2 ColorMap is basically just a lookup table which directly maps the intensity values of the input image to a predefined RGB color. In its essences it is exactly what you did by categorizing and associating with a specific color value.

Most of the cv2 ColorMaps just have a little more detail, most of them have either 64( "Rainbow", "Hot", ...) or 256 ("Jet", "Magma", ...) steps, some like "Winter" have 11. They come from the GNU Octave or Matlab color palettes.

If you really want to build this lookup table yourself the easiest way to achieve a high level of granularity for the individual steps is to use the HSV color space

import numpy as np
import cv2

N_STEPS = 20
h = np.linspace(0, 180, N_STEPS, endpoint=False) # cv2 Hue range is [0, 179]
s = np.ones_like(h)*255   # cv2 saturation range is from [0, 255]; adjust it to your liking
v = np.ones_like(h)*255   # cv2 value range is from [0, 255]; adjust it to your liking
hsv_colormap = np.dstack([h,s,v])

rgb_colormap = cv2.cvtColor(np.uint8(hsv_colormap), cv2.COLOR_HSV2RGB)

IMG_SIZE = 100
#intensities = np.random.random((IMG_SIZE,IMG_SIZE)) # from 0 to 1
intensities = np.array([np.linspace(0,1, IMG_SIZE, endpoint=False), ]*IMG_SIZE) # from 0 to 1
intensity_indices = np.uint8(N_STEPS*intensities) # map insity ranges to discretized intervals

color_mapped_intensities = rgb_colormap[0,intensity_indices,:]
cv2.imshow('colormap', color_mapped_intensities)

cv2.waitKey(0)

you can play around with the Hue values ranges to feed into the color map

Happy color mapping to you !


Power efficiency of human brains vs. neural networks

126 million artificial neurons at 12.6 Watts, with IBM's True North

Back in 2014, IBM's True North chip was pushing 1 million neurons at less than 100mW.

So that's roughly 126 million artificial neurons at 12.6 Watts.

A mouse has 70 million neurons.

IBM believes they can build a human-brain scale True North mainframe at a "mere" 4kW.

Once 3D transistors come to market, I think we'll catch up to animal brain efficiency pretty fast.


Do neurons of a neural network model a linear relationship?

In a neural network (NN), a neuron can act as a linear operator, but it usually acts as a non-linear one. The usual equation of a neuron $i$ in layer $l$ of an NN is

$$o_i^l = \sigma(\mathbf{x}_i^l \cdot  \mathbf{w}_i^l + b_i^l),$$

where $\sigma$ is a so-called activation function, which is usually a non-linearity, but it can also be the identity function, $\mathbf{x}_i^l$ and $\mathbf{w}_i^l$ are the vectors that respectively contain the inputs and the weights for neuron $i$ in layer $l$, and $b_i^l \in \mathbb{R}$ is a bias. Similarly, the output of a layer of a feed-forward neural network (FFNN) is computed as

$$\mathbf{o}^l = \sigma(\mathbf{X}^l \mathbf{W}^l + \mathbf{b}^l).$$

In your specific example, you set the new weight to $0$, if the output of the linear combination is less than $0$, else you use the output of the linear combination. This is the definition of the ReLU activation function, which is a non-linear function.


Is it possible to build an AI that learns humanity, morally?

I'm going to refer you to one of my favorite AI philosophers, Phillip K. Dick, who thought deeply on this subject and wrote about in some detail in Do Androids Dream of Electric Sheep.

Essentially, replicants (artificial humans) had a design flaw--they lacked empathy.  This flaw was allowed to persist because it had a useful side-effect in that replicants couldn't cooperate to resist their human overlords, and persisted in a state of chattel-slavery.

But the new Nexus models, which include Roy Baty and Pris, have become intelligent enough to start developing empathy, allowing them to band together and return to earth, seeking some kind of salvation, with often deadly results for humans.

Underlying this plot device, which pre-figures the formalization of evolutionary game theory by a few years (my guess is Dick attended a lecture at Berkely where the ideas underlying the formal field were discussed), is the idea that empathy is a function of sufficiently strong intelligence.

It's important to recognize that Dick's philosophy is heavily influenced by Christian philosophy, with an Old Testament emphasis on the golden rule *"Love the other as the self" (Leviticus 19:18), but evolutionary game theory demonstrates a natural basis for cooperation, which extends into algorithmic contexts.

The legitimate concerns expressed by Musk and Hawking are more concrete: that a human created alien* superintelligence could wipe us out inadvertently in pursuit of some goal we humans don't even understand.

Thus, value alignment is an issue of critical concern in the strictly hypothetical (as of today) field of superintelligence/AGI/ultraintelligent machines.

Stuart Russell called this the "Value Alignment Problem" referencing human vs. AI values.

From a Game Theory standpoint, I like to think about minimax as the "Iron Rule", and superrationality as the "Golden Rule".

The Iron Rule dictates that, in a condition of uncertainty, a rational agent must make the safest guess--that which limits the maximum potential harm to the agent, even if the result is not optimal in the sense of benefit.

"Renormalized rationality" is the term used to connote giving other agents the "benefit of the doubt" that they will be superrational also, and choose cooperation over betrayal or competition.

Generally, this concept is termed "reciprocal altruism", but it's not clear to me that this is entirely distinct from Leviticus 19:18 in the sense that the passage does not specifically exclude a result of mutual, greater benefit.

Reality may necessitate non-cooperation if one of the agents is irrationally adversarial:

Take a game of iterated Dilemma called "Turn the Other Cheek":

Iteration 1: A defects / B cooperates
Iteration 2: A defects / B cooperates (turns the other cheek)
Iteration 3: A defects / B defects

A's first choice is rational in a condition of uncertainty.  A's second choice shows a degree of paranoia.  A's thirds choice is irrational, as A could have cooperated, gaining more benefit, with only limited downside, which, in the worst case, still leaves A ahead of B.

B is superrational but not irrational.  B will not keep cooperating with an irrationally adversarial agent (this is sometimes termed "tough love").  B is willing to take not just one, but two "hits" out of goodwill, where goodwill is willingness to make a potential sacrifice in service of a more optimal potential result.  Nevertheless, B is still superrational and will always "forgive"--if A ever renormalizes their rationality, they will take a hit on a single iteration by cooperating, and B will cooperate on the next, and each subsequent, iteration, so long as A does not switch back to defection.

(There's a convoluted argument against this behavior, with the idea that the merely rational agent will always want to be ahead, and this will want to defect on the last iteration, which leads back up the chain to defecting on every iteration, but this is not rational as, if A defects initially then renormalizes their rationality, A will always be slightly ahead.)

Dilemma is an excellent analog for practical application of ethics in that, the only way the agents have to communicate is thought their actions.  The choice of cooperate/defect is information in a binary format.  Ultimately people are judged by their actions, not their words.

Philosophically speaking, we can't ignore the Iron Rule unless we're going for sainthood, but that doesn't mean we can't strive for the Golden Rule.

Mythologically, based on the work of recent narrative philosophers such as Stross and Rajaniemi, the dystopian aspect of the hypothetical Singularity derives from superintelligences solely focused on minimax, to the exclusion of all else.

George Bernard Shaw, in his play Mrs. Warren's Profession, casts the purely economic consideration of people as dehumanization (reduction of human bodies and minds to resources only.)  In Shaw's example, it is cast as the dehumanization of laborers in pursuit of marginally greater returns.

"Humanizing" AI's may require making sure they can see the superrationality of the Golden Rule, even with rational limitations for survival against an irrational foe (uncooperative in all conditions.)  Rajaniemi's name for this nemesis is "the All-defector"

See Also:

God's Algorithm as a minimax function.

Divine Move as an inspired, counter-intuitive choice which, in the most generalized sense, leads to a more optimal outcome. In the context of the game of Go, it's a choice that leads to victory for a single player, but in the context of Dilemma games, this would be the more optimal Nash equilibrium. (Note the etymology of inspired)


How have robots developed and how sophisticated might they be in the future?

The future of robotics

Companies like Google are doing restless research in robotics whereby machine learning algorithms are applied to empower self driving cars well equipped with computer vision, etc.

The future is so exciting in that it will be an era where artificially intelligence machines can rewrite their own source code and constantly improve upon themselves at an exponential rate -- ASI: artificial super intelligence.

Right now, robots like Sophia are using machine learning algorithms to answer or respond to human interactions.


If the reward function of an environment depends on some initial conditions, should I create a separate environment for each condition?

In this scenario, is it preferable to create two separate environments based on the priority (input parameter) of the alert?

It is difficult to make a hard rule here.
If the resulting environments can be cleanly sorted into a few different categories, and the ideal behaviour and/or the states visited are radically different in each scenario, then maybe a few different agents optimised for each scenario could work well.
A more general approach however, is to include the episode start data as part of the state that the agent observes on each time step. A single agent can then in theory learn the different behaviours required depending on the initial values, plus still generalise from anything shared between the multiple scenarios.

The alert has two priorities "HIGH" and "LOW", when the priority is "HIGH" the stopping condition is a reward of "100" and when the priority is "LOW", the stopping condition is a reward of "1000".

This may work against you. RL agents do not respond to the absolute values of rewards, other than how they compare to other rewards also available within the same episode (or continuing environment).
If there is only ever one issue to solve at a time, and no conflict between solving either of the "HIGH" or "LOW" priority problems (such as splittig resources or effort between them), the different reward system seems redundant. Solved is solved. You might rate the usefulness of an agent that solves the "LOW" priority issue well higher, but it seems to me that this describes what you should work on first, not the goals of the agent. To influence the goals of the agent, both rewards would need to be available within the same episode or continuing environment, requiring the agent to make a choice between them.


Is it possible to implement reinforcement learning using a neural network?

Andrej Karpathy's blog has a tutorial on getting a neural network to learn pong with reinforcement learning. His commentary on the current state of the field is interesting.

He also provides a whole bunch of links (David Silver's course catches my eye). Here is a working link to the lecture videos.

Here are demos of DeepMinds game playing.
Get links to the papers at Andrej Karpathy's blog above
- rat fps

nice demos at 19 minutes into this


Why does estimation error increase with $|H|$ and decrease with $m$ in PAC learning?

Definitely, you can find the proof in different resources (for example, in these notes or in the paper that originally proposed PAC learnability, A Theory of the Learnable). However, the intuition behind your question is when the size of the hypothesis increases, if you do not change anything, you can't see more part of the space. Hence, the estimation error will increase. Moreover, when you increase the number of samples, you have more chance to see more part of the hypothesis space, hence, the estimation error decrease.
Also, you can see some lemma about the relation of the PAC learnability and other similar concepts in the Wikipedia article Probably approximately correct learning:

Under some regularity conditions these three conditions are equivalent:

The concept class $C$ is PAC learnable.
The VC dimension of $C$ is finite.
$C$ is a uniform Glivenko-Cantelli class.


If I understand your query correctly, you want to create a latent space that groups similar objects. You should then probably look for Siamese networks. However, your loss function will need another term to increase dissimilarity between different labels. Otherwise, as pointed out by Mike NZ, the net would collapse(yes, it is possible). Perhaps this will give some insights.
Note that the above method is not completely unsupervised. There are, in fact, a few claims of unsupervised classifications via clustering, although your objective function would look very different. You could go through this paper(called SCAN) for more details.
Hope it helps.
[Edit]
If you want a (lower-dimensional)representation of objects themselves, browsing through this could help. For a complex problem, linear reductions like PCA, although helpful, aren't probably what you're looking for. Here you can try training autoencoders. The loss function would work, along with some regularization term.


How data augmentation like rotation affects the quality of detection?

The problem is that this kind of data augmentation seems (in theory) to hamper the network to gain precision on bounding boxes location as it loosens the frame.

Yes, it is clear from your examples that the bounding boxes become wider. Generally, including large amounts of data like this in your training data will mean that your network will also have a tendency to learn slightly larger bounding boxes. Of course, if the majority of your training data still has tight boxes, it should stell tend towards learning those... but likely slightly wider ones than if the training data did not include these kinds of rotations.

  Are there some studies dealing with the effect of data augmentation on the precision of detection networks? Are there systems that prevent this kind of thing?

  (Obviously, it seems advisable to use small rotation angles)

I do not personally work directly in the area of computer vision really, so I'm not sufficiently familiar with the literature to point you to any references on this particular issue. Based on my own intuition, I can recommend:

Using relatively small rotation angles, as you also already suggested yourself. The bounding boxes will become a little bit wider than in the original dataset, but not by too much.
Using rotation angles that are a multiple of $90^\circ$. Note that if you rotate a bounding box by a multiple of $90^\circ$, the rotated bounding boxes become axis-aligned and your problem disappears again, they'll become just as tight as the bounding boxes in the unrotated image. Of course, you can also combine this suggestion with the previous one, and use rotation angles in, for example, $[85^\circ, 95^\circ]$.
Apply larger rotations primarily in images that only have bounding boxes that are approximately "square". From looking at your image, I get the impression that the problem of bounding boxes becoming wider after rotations is much more severe when you have extremely wide or thin bounding boxes (with one dimension much greater than the other). When the original bounding box is square, there still will be some widening after rotation, but not nearly as much, so the problem may be more acceptable in such cases.


Why do small datasets require more samples, while big datasets require fewer samples in negative sampling?

He likely found this to a be a best practice to avoid over fitting, with a small data set if you only use small and easy to learn (less words -> less degrees of freedom) sequences then you open your model to the risk of over fitting that data set where as on a large data set that has alot more total information you can train on small sequences without being at risk of over fitting because although the smaller sequences will be easier to learn the variance of sequences will be much higher.


Can I have different rewards for a single action based on which state it transitions to?

The reward function can be a function of the current state, current action, and next state: $R(s_t, a_t, s_{t+1})$. It's valid to use the Bellman operator in this setting because it's still a contraction and will yield the optimal value function.

NOTE: I'm assuming that you will be solving the MDP with the Bellman equation.


What exactly are the "parameters" in GPT-3's 175 billion parameters and how are they chosen/generated?

Parameters is a synonym for weights, which is the term most people use for a neural networks parameters (and indeed in my experience it is a term that machine learners will use in general whereas parameters is more often found in statistics literature). Batch size, learning rate etc. are hyper-parameters which basically means they are user specified, whereas weights are what the learning algorithm will learn through training.


How can I develop a genetic algorithm with a constraint on the sum of alleles?

There are multiple ways to handle 'illegal' individuals, each one with pros and cons:

Abortive methods: The individuals that violate constraints are eliminated as soon as discovered (i.e. after crossover or mutation) and new individuals are generated in order to keep the population stable. This usually implies a slower creation of new generations, as some individuals are discarded.
Contraceptive methods: The crossover and mutation are written in such a way to make it impossible for a newly generated individual to violate any constraints. This way to act is usually more efficient, as no individuals are discarded, but it might not be possible.
Penalization function: The fitness function gives a huge penalization to the individuals that violate constraints (usually proportionally to the constraints it violates). In this way they usually do not get to reproduce and their genes are eventually lost. You'd go this way if illegal individuals are very rare and there is no possibility for them to take over the full population (causing the algorithm to fail).

By experience, try to act on the crossover algorithm first (contraceptive way), it is the best way to exclude constraints violations. If this is not possible, pick one of the other two methods, depending on how often the illegal individuals are generated (not so often -> penalization, very ofter -> abortive)  and on how easy it is to penalize individuals that violate constraints (easy -> penalization, not easy -> abortive).

A contraceptive way to handle your example is writing the crossover as following:

bool[] crossover(bool[] p1, bool[] p2)
{
    // classical binary crossover
    bool[] child = p2.copy()
    child[rndSection] = p1[rndSection]

    // contraceptive part
    while (CountTrue(child) > 3) child[RandomIndexOfTrue(child) = false

    return child
}

But this is a fairly complex problem and each scenario has its own specifications.


Does balancing the training data set distribution for a neural network affect its understanding of the original distribution of data?

This is a very good question. Your problem is the classic classification problem of Neural Networks. In this problem the main objective of the Neural Network is to transform the data by some non-linear (in general) transformation so that the data becomes linearly separable for the final layer to perform classification.

Point to Note: This is not a regression problem, so that you are trying to fit a curve. Whenever there is a regression problem logically you can use a PDF to write some kind of information about new data. You can express mathematically the probability of your data falling within a certain range of error, since this is a optimising continuous function problem (generally RMSE).

This is not in the case of classifier. Classifier follow Bernoulli probability (though we represent the cost function as continuous). So current event is independent of past events. This makes a classifier harder to train for unbalanced class. So if we write:

func foo(data):
    return True

It pretty much has 98% accuracy, but you can understand we do not want this type of classifier.

In general we want both classes to have good accuracy scores, sometimes this is measured by $F1_{score}$ but I like to think it in terms of proportionality. If we have $n$ examples with $m$ in one class, then and the classifier $a_1$ and $a_2$ correct predictions in both classes respectively, then I would check both the metric $ \frac {a_1}{m}$ and $ \frac {a_2}{n-m}$, which gives you the general idea.

Also in practical the identification 2% is sometimes far more important than the rest 98% (air-plane defects, cancer detection). So we use special ML algorithm called Anomaly Detectors for such type of problems.


Do the rows of the design matrix refer to the observations or predictors?

Based on my understanding, in common machine learning, we formulate our matrices, from vectors, as rows to be observations, columns to be predictors.

The rows (or, in general, the first dimension of your tensor) are typically the observations. For example, in TensorFlow, the first dimension of the input tensor typically refers to the batch size, i.e. the number of observations. If you are using Pandas (a Python library to manipulate data), the rows are typically the observations and the columns are the predictors.
However, in general, it does not really matter which convention you use, provided that you use one of the conventions consistently in your implementation (i.e. you choose one of the conventions and you stick with it throughout all your code, to avoid complexity), and make it clear in the documentation. So, you can have a matrix where either the rows or columns are observations and, consequently, the columns or, respectively, rows are the features (aka predictors or independent variables).
Anyway, it is probably a good idea to be consistent with existing literature and implementations/libraries, so you should probably use the rows for the observations.


In variational autoencoders, why do people use MSE for the loss?

On page 5 of the VAE paper, it's clearly stated

We let $p_{\boldsymbol{\theta}}(\mathbf{x} \mid \mathbf{z})$ be a multivariate Gaussian (in case of real-valued data) or Bernoulli (in case of binary data) whose distribution parameters are computed from $\mathbf{z}$ with a MLP (a fully-connected neural network with a single hidden layer, see appendix $\mathrm{C}$ ).
...
As explained above and in appendix $\mathrm{C}$, the decoding term $\log p_{\boldsymbol{\theta}}\left(\mathbf{x}^{(i)} \mid \mathbf{z}^{(i, l)}\right)$ is a Bernoulli or Gaussian MLP, depending on the type of data we are modelling.

So, if you are trying to predict e.g. floating-point numbers (in the case of images, these can be the RGB values in the range $[0, 1]$), then you can assume $p_{\boldsymbol{\theta}}(\mathbf{x} \mid \mathbf{z})$ is a Gaussian, then you can equivalently minimise the MSE between the prediction of the decoder and the real image in order to maximise the likelihood. You can easily show this: just replace $p_{\boldsymbol{\theta}}(\mathbf{x} \mid \mathbf{z})$ with the Gaussian pdf, then maximise that wrt the parameters, and you should end up with something that resembles the MSE.
So, to answer your question more directly: yes, minimizing the MSE is theoretically founded, provided that you're trying to predict some floating-point number.


Loss randomly changing, incorrect output (even for low loss) when trying to overfit on a single set of input and output

Softmax activation always adds up to 1, because it's designed to deal with probabilities (in problems of classification, those probabilities represent how likely the network thinks an object belongs to a specific class). You can verify that by summing up the numbers of your output layer. So currently your network is trying to do the impossible, to produce output that sums up to 5, instead of 1. Therefore, loss will never become stable. Since you want your output layer to produce all ones, you need to use some other activation, for example, linear. Linear activation does not have the same constraint that Softmax does.


Trained a regression network and getting EXACT same result on validation set, on every epoch

The answer is I was too tired to see I was using the same model for validation every epoch, instead of that epoch's model.

Write good code, and this won't happen to you.


When labelled data is not available, what are some common unsupervised learning algorithms for pattern recognition that can be used?

There are some unsupervised learning algorithms that can be used for pattern recognition (i.e. the discovery of patterns in data). The most notable one is probably k-means, which is a clustering algorithm. In k-means, you cluster your unlabeled data into groups (or clusters) based on the distance (or similarity) between them. When a new data point arrives, you'll associate it with the most similar cluster. In this sense, you are performing pattern recognition in an unsupervised way.

Here's an excerpt from the famous book Pattern Recognition and Machine Learning (2006) by C. Bishop

  In other pattern recognition problems, the training data consists of a set of input vectors $x$ without any corresponding target values. The goal in such unsupervised learning problems may be to discover groups of similar examples within the data, where it is called clustering, or to determine the distribution of data within the input space, known as density estimation, or to project the data from a high-dimensional space down to two or three dimensions for the purpose of visualization.

So, there are other problems, apart from the problem of clustering, that you may want to solve with unsupervised learning algorithms for the purpose of pattern recognition, such as density estimation (see e.g. mixture models) or dimensionality reduction (see e.g. PCA).


Will we be able to build an artificial intelligence that feels empathy?

Let us describe a very simple system that does something we could label as empathyc.

A chatbot answers "I am sorry to hear that. What happened?" when we type "I feel bad", and it replies "I am glad to hear that. Fancy some music?" when we type "I feel good".

Somehow, it perceives a human emotion, and acts accordingly.

Planes fly but they do not fly as birds. Similarly, we can expect artificial empathy from an AI, not necessarily natural empathy as we feel it.


What is the difference between memetic algorithms and genetic algorithms?

A genetic algorithm is an algorithm, based on natural selection (the process that drives biological evolution), for solving both constrained and unconstrained optimization problems.
A memetic algorithm is an extension of the concept of a genetic algorithm that uses a local search technique to reduce the likelihood of premature convergence.
The paper A Comparison between Memetic algorithm and Genetic algorithm for the cryptanalysis of Simplified Data Encryption Standard algorithm compares both approaches.
To answer your last question, yes, an individual's lifetime still plays a part in memetic algorithms because the objective here is to avoid premature convergence.


From an implementation point of view, what are the main differences between an RNN and a CNN?

In Convolutional Neural Networks (CNNs) you have small kernels (or filters) that you slide over an input (e.g. image). The value resulting from the convolution of the filter with a subset of the image over which the filter is currently positioned is then put into its respective cell in the output of that layer. Essentially, training CNNs boils down to training small filters, for example for detecting edges or corners etc. in input data, which most frequently happens to be images indeed. The assumption here is that features can be detected locally in the input volume, which entails that the nature of the input data shall be coherent over the entire volume of input data.
Recurrent Neural Networks (RNNs) do not work locally, but are applied to sequences of arbitrary input data, where one input node may receive sensor readings, while the next node receives the date on which the sensor reading was measured. Of such arbitrary data, you feed a sequence through the RNN, which always keeps its own internal state from processing the previous instance/sample in the sequence in memory while processing the next data point/sample in the sequence. Depending on the kind of recurrent cell type that is employed to construct a RNN layer, the memory of the previous internal state then affects the computation of the next internal state and/or output computed when working on the next data sample. So, information of past data points/samples is carried forward while iterating though a sequence.
In short, CNNs are meant to detect local features in volume data, while RNNs preserve information over their previous internal state while processing the next data sample.
Probably one of the best online resources walking you through all the related concepts step by step is the following lecture series offered by the Stanford University.


How do I solve this optimal control problem with reinforcement learning?

I have tried to use DQN in Keras, but I am not sure that I am using correct state variables/reward.

You have a wide range of choices that are all valid. As it is a simple control and learning scenario, provided you cover basics (described in a moment), then the difference in your choices are about how easy you make it for the agent to learn. You may actually want to set things up so that there is a certain difficultly level, so you get to compare different agent designs. You don't necessarily need the "best" state representation and reward function designs here. Just some that work.

State representation

The most important factor is that your state has the Markov property - that it contains enough information that the agent really can predict expected future rewards.

It is OK for there to be randomness, but generally not OK for there to be important hidden variables that affect the outcome in a major way. If there are such variables in the environment, but not in the state representation, then a simple agent like DQN will struggle.

Reward function

The reward function should capture the end goals that you are interested in, and ideally as simply and purely as possible given that. The purpose of most RL methods is to maximise the expected sum of reward, and they are built to deal with sparsity - you don't need to help or signal the same thing twice or give rewards for "getting close" etc, although sometimes this can improve learning speed if done well.

I have also written a more comprehensive answer about reward function design as an answer to Deciding on a reward per each action in a given state (Q-learning)

  Currently, the state variables are the velocity vector of the wolf, the distance vector from the wolf to the rabbit.

This seems reasonable. Do consider that for a neural network you will want to keep the scale of these to within a nice range e.g. -1 to +1 for all features. So you may want to scale from whatever you are using for the values in the environment.

You can make this problem easier to learn by using polar co-ordinates. That removes the need for the neural network to solve the inherent trigonometry problem when deciding if turning left or right is better. I have solved a very similar pursuit RL problem and compared between cartesian and polar co-ordinates, and the difference is very large between the two representations for a DQN based agent.

Specifically for the easiest learning you want the difference vector between your agent's current heading and the target, expressed as a difference in heading plus distance to go. If you do this, sometimes the initial randomised neural network will already be efficient at solving the problem.

However, it may be interesting for you to see how well different agents do solve that trigonometry problem of converting a difference in vector coordinates and velocity, into a direction to turn. You could even make it harder and give absolute location coordinates for both agent and the target, plus the agent velocity (3 vectors instead of 2), requiring the neural network to approximate even more complex maths before it solves the problem.

  The reward at each time point is the negative current time. When the wolf catches the rabbit, the reward is 1000 - current time (the wolf is penalized for running too long).

That should work, but seems more complex than you need. You are already penalising the wolf for each time step during the chase, so there is no real need to have any extra formula for the final reward. In fact you can just have a final reward of 0 for reaching a satisfactory goal (providing there are no unsatisfactory end points that you want it to avoid - perhaps yo umay make the environment trickier later on?). The agent will simply try to resolve the episode quickly because that will still maximise the (negative total) return.

An alternative might be to reward some value for catching the rabbit e.g. +10, and zero on each time step. Then to encourage a fast resolution you would need to use discounting, so that the agent values being nearer to the rabbit higher than being further away because the potential reward is fewer time steps away.

There is also no need to use large values. Values are relative, so if you have some minor issue to solve plus a main goal, sometimes it is worth having a wide range of values. Here you don't need it. Having a value of 1000 may challenge your neural network to learn properly (because error for the first time the wolf catches a rabbit would be so large it may cause a large step in weights - enough to destabilise learning in the NN), for no real benefit.

For your first reward scheme I recommend a fixed reward of e.g. -1 per time step, and to end the episode when the wolf catches the rabbit. No multiplications or offsets.

If you have a very large area for the wolf to explore compared to the distance that the wolf needs to be from the rabbit in order to catch it, it may help to give some reward for proximity to the rabbit. The simplest to implement and learn would just be the difference in distance between the wolf and rabbit between $t$ and $t+1$ multiplied some small factor to put this into a similar range to the penalty per time step. Note if you extend this reward out to any significant distance that it should make the learning problem far easier (combine polar coordinates and this reward system will make it very easy for the DQN to learn - it should take only a few episodes to get near optimal behaviour, if your learning hyperparameters are good for the DQN).

Without any proximity reward, you will rely on the wolf literally bumping into the rabbit through random behaviour, before it will have any data example that getting the vector between itself and the rabbit close to (0,0) is a good thing. You may need to have a relatively large capture radius, plus limit the area that the wolf (and eventually rabbit) can explore, in order to avoid very long sequences of random behaviour where nothing is learned initially.


What's the difference between content-based attention and dot-product attention?

The Attention is All you Need has this footnote at the passage motivating the introduction of the $1/\sqrt{d_k}$ factor:

To illustrate why the dot products get large, assume that the components of $q$ and $k$ are independent random variables with mean 0 and variance 1. Then their dot product, $q \cdot k = \sum^{d_k}_{i=1}q_ik_i$ has mean 0 and variance $d_k$.

I suspect that it hints on the cosine-vs-dot difference intuition. The cosine similarity ignores magnitudes of the input vectors - you can scale $h^{enc}$ and $h^{dec}$ by arbitrary factors and still get the same value of the cosine distance.
The footnote talks about vectors with normally distributed components, clearly implying that their magnitudes are important. This suggests that the dot product attention is preferable, since it takes into account magnitudes of input vectors.  And the magnitude might contain some useful information about the "absolute relevance" of the $Q$ and $K$ embeddings.
Edit after more digging: Note that transformer architecture has the Add & Norm blocks after each
attention and FF block. At first I thought that it settles your question: since
every input vector is normalized then cosine distance should be equal to the
dot product. That's incorrect though - the "Norm" here means Layer
Normalization - analogously to batch normalization it has trainable mean and
scale parameters, so my point above about the vector norms still holds.
Finally, since apparently we don't really know why the BatchNorm works
same thing holds for the LayerNorm. Interestingly, it seems like (1) BatchNorm
(2) LayerNorm and (3) your question about normalization in the attention
mechanism - all of it look like different ways at looking at the same, yet
undiscovered and clearly stated thing.


Is there any other rotated object detection datasets?

Multiscale Rotated Bounding Box-Based Deep Learning Method
Here's a link for a reference


Count number of objects in image using CNN

You can run YOLO, then count the number of occurrences.


What are the popular approaches to estimating the Q-function?

There's are some solutions to calculating q-values; find the exact values:

Brute-force the action sequence to find max (not pratical)
Do recursion on Bellman equation to get max (the same like action sequence brute-force, not pratical)

Estimate the q-values:

Based on different problems to solve, apply some classic algorithms or human logics to estimate; during estimation, some heuristic tactics may be used
Do a lot of randomisation to find max, including Monte Carlo tree search

Gradually optimise the q-values:
(1) Do optimisation based on Bellman equation (Q-learning):
$$ q_t(s_t,a_t) = q_t(s_t,a_t) + \alpha(r + \gamma\times\max(q_{t+1}(s_{t+1},a_{t+1})) - q_t(s_t,a_t))$$
Bellman equation is true when the temporal difference (the part multiplied with $\alpha$) reaches zero, which means the max of time t+1 reaches exact value.
(2) Do optimisation based on Bellman equation (Q-network), fit the neural network to expected value:
$$r + \gamma\times\max(q_{t+1}(s_{t+1},a_{t+1}))$$


Video summarization similar to Summe's TextRank

From "Deep learning-based video summarization — A detailed exploration" by Surya Remanan:
Video summarization can be considered as the process of distilling a raw video into a more compact form without losing much information. In a general video summarization system, image features of video frames are extracted, and then the most representative frames are selected through analyzing the visual variations among visual features. This is done either by taking a holistic view of the entire video or by identifying the local differentiation among the adjacent frames. Most of those attempts rely on global features such as colour, texture, motion information, etc. Clustering techniques are also used for summarization. Video summarization can be categorized into two forms:

Static video summarization (keyframing) and
Dynamic video summarization (video skimming)

Static video summaries are composed of a set of keyframes extracted from the original video, while dynamic video summaries are composed of a set of shots and are produced taking into account the similarity or domain-specific relationships among all video shots.
Following is an attention-based video summarization model - PyTorch implementation of the ACCV 2018-AIU2018 paper Video Summarization with Attention
There is a video summarisation focused models using reinforcement learning -  Unsupervised video summarization with deep reinforcement learning (Theano)
There is an LSTM - GAN based approach to video summarization - video summarization lstm-gan pytorch implementation
Microsoft Bing Search has come up with a video summarization technique using thumbnails. Intelligent Search: Video summarization using machine learning


Is my proof of equation 0.6 in the book "Reinforcement Learning: Theory and Algorithms" correct?

The expectation of a sum is equal to the sum of the expectation this just follows from the linearity property of expectations
$$ \begin{aligned} E[\sum_{t} f(s_t,a_t)] &= \sum_{\tau} p(\tau)\left(\sum_t f(s_t,a_t)\right)
 \\ &= \sum_\tau\sum_{t}p(\tau)f(s_t,a_t)
 \\ &= \sum_t\sum_\tau p(\tau)f(s_{t,\tau},a_{t,\tau})
 \\ &= \sum_tE[f(s_t,a_t)]
\end{aligned} $$
note that in the penultimate line I swapped the sums around which can be understood as looking at all trajectories for a single time-step instead of looking at time-steps within  a single trajectory (the second expression). I append subscript for variables in the penultimate line for clarity since they also depend on which trajectory they were drawn from.

As mentioned by nboro in the comments, the linearity of property only holds for infinite sums if
$$\sum_{i=0}^\infty E[|X|]< \infty  \quad or \quad E\left[\sum_{i=0}^{\infty}|X|\right]. <\infty$$
More details can be found here. Using this theory alongside knowledge that since we know that value functions in RL are always bounded (in continuing tasks discounting factors are introduced) then we can say there exists an $M$ such that
$$ |f(s_{t,\tau},a_{t,\tau})| \leq M \qquad \forall t,\tau $$
Then
$$ \begin{aligned}\sum_t\sum_\tau \lambda^t p(\tau)|f(s_{t,\tau},a_{t,\tau})| &\leq \sum_t\sum_\tau \lambda^t p(\tau)M
\\ &=M\sum_t\lambda^t\sum p(\tau)
\\ &=M\sum\lambda^t
\\ &< \infty
\end{aligned}$$
As for the rest of the proof is there anything else you need clarity on? it looks good to me. From that step to the next step where you change distribution from $p(\tau)$ to $p(s,a)$ is just because of marginalisation but I think you got that.


How do I choose the search algorithm for a particular task?

The choice of the most appropriate search algorithm for a particular task is often based (but not exclusively) on its time complexity, space complexity, termination (if the algorithm always halts), optimality guarantees (if the algorithm is guaranteed to find the optimal solution), available implementations (as software libraries) and (if known) the actual performance (for such particular task). There are algorithms that have the same time or space complexities, but that, in practice, have different performance, which also depends on the problem being solved and implementations (which might or not be optimised).

For example, consider a search tree where you have a finite number of nodes and paths. Suppose that solutions lie at the leaves of this tree and that you are interested in one solution, but not necessarily the optimal one. In that case, in practice, DFS might find solutions faster than BFS, because of its nature and the task being solved. However, in the case you have a tree with infinite-length paths, then DFS might not terminate. In that case, proceeding layer by layer (BFS) might be a more appropriate strategy.


Which algorithm for scheduling race grid?

What you're looking for is most likely constraint programming (CP). Essentially, with a CP model you declare a set of variables and a set of constraints (and an objective, optionally). Then a solver will solve your model and find a solution if one exists (if you had specified an objective, it will find the best solution).

Here is an example which is very silimar to your problem (a MiniZinc model which solves the problem is included). You can download MiniZinc and try it out for yourself.


How does crossover work in a genetic algorithm?

As far as I know, there isn't a "specified correct way". The whole idea is that you want the population to converge and increase the sample rate in that more optimal looking place. What works best all depends upon your fitness landscape.

You could also crossover by doing something like

crossover_point = random_number_size_genome
child[:] = parent_a[:crossover_point] + parent_b[crossover_point:]

Or have multiple points of crossover or more exotic types of crossover.
As far as I know, the impact on your algorithm because of the different crossover algorithms shouldn't be that different.

Fitness function and how you select the fittest has a way bigger impact.
You have understood the crossover step correctly, looking at your example.


Can we prove that a piece of code in AI software will cause it to never turn against us?

Unfortunately, this is extremely unlikely.

It is nearly impossible to make statements about the behaviour of software in general. This is due to the Halting problem, which shows that it is impossible to prove whether a program will stop for any given input. From this result, many other things have been shown to be unprovable.

The question whether a piece of code is friendly, can very likely be reduced to a variant of the halting problem.
An AI that operates in the real world, which is a requirement for "friendliness" to have a meaning, would need to be Turing complete. Input from the real world cannot be reliably interpreted using regular or context-free languages.

Proofs of correctness work for small code snippets, with clearly defined inputs and outputs. They show that an algorithm produces the mathematically right output, given the right input.
But these are about situations that can be defined with mathematical rigour.

"Friendliness" isn't a rigidly defined concept, which already makes it difficult to prove anything about it. On top of that, "friendliness" is about how the AI relates to the real world, which is an environment whose input to the AI is highly unpredictable.

The best we can hope for, is that an AI can be programmed to have safeguards, and that the code will raise warning flags if unethical behaviour becomes likely - that AI's are programmed defensively.


I guess you are using linear activation functions,  maybe you are not initializing your weights, or you are regularizing your model enough.

Initialize weights with glorot, insert dropout layers in between, use Relu as your activation function, stop the training process based on Early Stopping. and just experiment with one hidden layer.

side note: if you use Adam, don't mess with the learning rate.
in SGD optimizer you could use decay because there is a single learning rate for all weight updates and the learning rate does not change during training.
In Adam, the learning rate is maintained for each network weight (parameter) and separately adopted as learning unfolds.


How to prove the formula of eligibility traces operator in reinforcement learning?

I will refer to $\mathcal T^{\pi} $as $\mathcal T$ and $P^{\pi}$ as $P$ for notational simplicity
\begin{align}
(\mathcal{T})^{n+1} Q &= \mathcal{T}(\mathcal{T}(...(\mathcal{T}(Q))))\\
&= r + \gamma P(r + \gamma P(...(r + \gamma P Q)))\\
&= r + r\sum_{i=1}^{n} \gamma^i P^i + \gamma^{n+1} P^{n+1} Q
\end{align}
\begin{align}
\mathcal{T}_{\lambda}Q &= (1-\lambda) \sum_{n=0}^{\infty} \lambda^n (\mathcal{T})^{n+1} Q\\
&=(1-\lambda)\{\lambda^0 (\mathcal T)^1Q + \lambda^1 (\mathcal T)^2Q + \lambda^2 (\mathcal T)^3Q + \ldots   \}
\end{align}
when you plug in expression for $(\mathcal T)^i Q$ inside this sum and rearrange you get 3 sums
\begin{equation}
\mathcal{T}_{\lambda}Q = (1-\lambda) \sum_{n=0}^{\infty} \lambda^n r + (1-\lambda)\sum_{n=1}^{\infty} \lambda^n \gamma^n P^n r + (1-\lambda)\sum_{n=0}^{\infty} \lambda^n \gamma^{n+1} P^{n+1} Q
\end{equation}

sum:
\begin{equation}
(1-\lambda) \sum_{n=0}^{\infty} \lambda^n r = r
\end{equation}
sum:
\begin{equation}
(1-\lambda)\sum_{n=1}^{k} \lambda^n \gamma^n P^n r = (1-\lambda\gamma P)^{-1}(1 - \lambda^k \gamma^k P^k)\lambda\gamma P r
\end{equation}
As $k \rightarrow \infty$ and since $\gamma < 1$ this is in the limit equal to
\begin{equation}
(1-\lambda)\sum_{n=1}^{\infty} \lambda^n \gamma^n P^n r = (1-\lambda\gamma P)^{-1}\lambda\gamma P r
\end{equation}
sum:
\begin{equation}
(1-\lambda)\sum_{n=0}^{\infty} \lambda^n \gamma^{n+1} P^{n+1} Q = (1 - \gamma\lambda P)^{-1}(1-\lambda)\gamma P Q
\end{equation}
If you combine all 3 you get
\begin{align}
\mathcal{T}_{\lambda}Q &= r + (1-\lambda\gamma P)^{-1}\lambda\gamma P r + (1 - \gamma\lambda P)^{-1}(1-\lambda)\gamma P Q\\
&= r+ (1-\lambda\gamma P)^{-1}(\lambda \gamma P r + \gamma PQ - \lambda\gamma PQ)\\
&= r+ (1-\lambda\gamma P)^{-1}(\lambda \gamma P r + (\mathcal T)Q - r - \lambda\gamma PQ)\\
&= (1-\lambda\gamma P)^{-1}(r - \lambda \gamma P r + \lambda \gamma P r + (\mathcal T)Q - r - \lambda\gamma PQ)\\
&= (1-\lambda\gamma P)^{-1}( (\mathcal T)Q  - \lambda\gamma PQ + Q - Q)\\
&= Q + (1-\lambda\gamma P)^{-1}( (\mathcal T)Q  - Q)
\end{align}


Is my 57% sports betting accuracy correct?

My question is, can I rely on my Accuracy (mean & standard deviation) for future games even though my Testing Accuracy is lower than 52.5%?

If by Accuracy you mean training accuracy, then absolutely you should not trust those values. For almost all machine learning algorithms there is a problem with overfitting to training data, which will result in reporting over-estimates for metrics against the training data. This is why you should always have a test data set of unseen data, because the performance on the training data is not what you truly care about - it is the performance on new unseen data, or how the model generalises, that matters when you want to use it to predict new values.
The test data set gives you a measure of how well your model will generalise, because it simulates the performance of the model against unseen data.
However, the test data set is not perfect for all uses. Random chance will cause your test measurements to vary. If you use your test results to select a model from a list of models based on performance, then:

The model you selected does have the highest likelihood of being the best one, as you want. It is not guaranteed to be the best though.

The test result is likely to be an over-estimate of general performance. That is because you cannot separate the random fluctuations in test results from real performance improvements, you only see the combination.

The more tests you run, the more likely you will have an inflated view of the performance of your best model.
The usual fix for this is to use cross validation. Cross validation uses yet another data set to help you with the first step of choosing the best model. After you have chosen your best performing model using cross validation, then you can use a test set that you have kept in reserve to measure the performance. Because you have not used that test set to select a model, then it will give you an unbiased measure of performance. You should still bear in mind that this measure still comes with implied error bars (and with other caveats, such as any inherent sampling bias).
When predicting future results from past data, you do also need to be concerned about population drift and non-stationary problems. This is a common issue in any data set that includes complex behaviour that can evolve over time. This is very likely to affect results from sports teams where many conditions affecting performance evolve over the same timescales that you are trying to predict. In practice this means you will want to feed in new data ad re-train your models constantly, and despite this your models will tend to lag behind reality. It is unlikely you will achieve even the test result accuracy in the long term.
You can add one more thing to your testing routines to help assess the impact of non-stationarity - when reserving data for the test or cross-validation sets, don't do it at random, instead reserve all the latest results (e.g. last four weeks of data) for test only. You are likely to see a reduction in the metrics when you do this for a problem domain like sport, but that should give you a more realistic assessment of the model you are building.


How do I create an AI for a two-players board game?

Assuming it is a turn-based game and, for each turn, there's an optimal choice that will lead to the winning state (zero-sum), you can basically simplify the question to "What is the optimal sequences of moves for me to win, considering the current situation that is presented on the board?". So you will need to perform your algorithm every turn as the optimal sequence will change when the board changed.

There's a relatively reliable AI algorithm that's being implemented to win games like chess, backgammon, etc. The technique is called minimax. In summary, minimax is simply a search algorithm to minimize the other player's score, while maximizing your score.

One of the problems you will encounter is that the search tree becomes so wide as we search through deeper parts of the tree, so it is essential to also implement alpha-beta pruning to cut down the amount of search. In general, alpha-beta pruning is simply eliminating branches that are very unlikely to be the optimal choice (in terms of score) thus reducing the number of searches to speed up the algorithm.


What caused the resurgence of AI since the early 2000s?

To vastly oversimplify a lot of the progress of modern research in AI/neural nets, the recent advances stems from applicational improvements of the back propagation algorithm.

ex: our recent possession of big data (it turns out that back-prop works very well if you have lots of data to feed into the network), increased computing power to harness that data, and creative network designs to leverage that computing power (deep learning).

So far I haven't found a good source that integrates the knowledge of current practice of neural networks into the classical study of AI nicely. But if you are still interested you can read this modern classic textbook on neural networks and make your own judgements.

http://www.deeplearningbook.org


How to resolve lexical ambiguity in natural language processing?

In general, how algorithm should distinguish the word meaning and recognise the word within the context?

I don't think anybody knows how to answer this for the general case. If they did, they'd have basically solved AGI.  But we can certainly talk about techniques that get part-of-the-way there, and approaches that could work.

One thing I would consider trying (and I don't know off-hand if anybody has tried this exact approach) is to model the disambiguation of each word as a discrete problem for a Bayesian Belief Network where your priors (for any given word) are based on both stored "knowledge" as well as the previously encountered words in the (sentence|paragraph|document|whatever).  So if you "know", for example, that "Reading is a city in the UK" and that "place names are usually capitalized", your network should be strongly biased towards interpreting "Reading" as the city, since nothing in the word position in the sentence strongly contradicts that.

Of course I'm hand-waving around some tricky problems in saying that, as knowledge representation isn't exactly a solved problem either.  But there are at least approaches out there that you could use.  For example, you could use the RDF / triple based approach from the Semantic Web world.  Finding a good way to merge that stuff with a Bayesian framework could yield some interesting results.

There has been a bit of research on "probabilistic RDF" that you could possibly use as a starting point.  For example:

http://om.umiacs.umd.edu/material/papers/prdf.pdf

http://ceur-ws.org/Vol-173/pos_paper5.pdf

https://www.w3.org/2005/03/07-yoshio-UMBC/

http://ebiquity.umbc.edu/paper/html/id/271/BayesOWL-Uncertainty-Modeling-in-Semantic-Web-Ontologies

http://www.pr-owl.org/


How do I calculate the partial derivative with respect to $x$?

While this may not be the answer you were looking for, I hope this explanation will help you to understand applying backpropagation to a CNN. Fundamentally, convolutional layers are no different than dense layers, however there are restrictions. The key one is weight-sharing which allows a CNN to be much more efficient than a regular dense layer (as well as it being sparse due to locality). Imagine we are transforming a 4x4 image into a 2x2 image. Since we are inputting a 16-vector, and outputting a 4-vector, we need a weights matrix of 4x16:

This has 64 parameters. In a convolutional layer, we can accomplish this by convolving a 3x3 kernel over the image:

$$ K=
\begin{bmatrix}
k_{1,1} & k_{1,2} & k_{1,3} \\
k_{2,1} & k_{2,2} & k_{2,3} \\
k_{3,1} & k_{3,2} & k_{3,3}
\end{bmatrix} $$

This convolution is equivalent to multiplying by the weights matrix:

As you can see, this only requires 9 parameters and backpropagation can be applied to update these parameters.

Image Source: https://towardsdatascience.com/intuitively-understanding-convolutions-for-deep-learning-1f6f42faee1


Can singularity result into the extinction of the third world?

Since we're in the Artificial Intelligence StackExchange, I'll approach the answer from that perspective. The singularity (in this case superintelligent artificial general intelligence (AGI)) tends to have global consequences. In Superintelligence: Paths, Dangers, Strategies Bostrom elaborates on this idea. Essentially, there are three potential outcomes (keep in mind these are from a human perspective.):

Global positive
Global negative
Uneven outcomes

Global positive assumes the AGI has goals which align with human values (likely because we sorted out the control problem/alignment). The outcome here ranges between humans simply continuing to exist (maybe the AGI simply leaves) and humans receiving a cosmic endowment (utopian singularity stuff).

Global negative assumes the AGI does not have goals which align with human values (likely because we didn't sort out the control problem/alignment).The outcome here ranges from humans simply continuing to exist (again, the AGI may simply leave), to humans no longer existing (perhaps we are more useful to the unaligned AGI as paper clips).

The global outcomes appear to be significantly more likely due to the way we are approaching the control problem/alignment. Aligning an AGI with human goals in general is extremely difficult. Aligning an AGI with specific human's goals implies total control over it, which is significantly more difficult. With that said, it may happen, which could lead to the third potential outcome.

Uneven outcomes assume the control problem has been completely solved, and the AGI is perfectly aligned with a specific group of human's goals (to reiterate, this is incredibly difficult to do, and is unlikely to happen.) In this scenario, the controlling group would have a decisive strategic advantage, receive a cosmic endowment, and would likely form a singleton. Perhaps they would share with others, perhaps they would turn everybody else into paper clips, who knows. On this path, groups which do not pursue AGI (perhaps for the reasons you listed) are taking an existential risk.

So, in general, it is unlikely things will go particularly well for one group while going badly for another, but if it does it may be quite bad for the losing groups.


Best ways of leveraging AI for stock market trading

Your list is complete for what is considered 'popular' by most practitioners who apply AI for stock trading.  Supervised learning and rule learning are at the top for accuracy.  There are more academic papers published on classifiers than on regression approaches;
classifiers are typically more accurate than regressors.


Can a vanilla neural network theoretically achieve the same performance as CNN?

All CNNs can be represented as vanilla networks on the flattened image data. Just to do so, you would need A LOT of parameters (most of which would be 0) for what CNNs do freely. You can think of a CNN as reusing a filter on a masked input (whichever receptive field it's looking at whatever point during the convolution) repetitively.

In other words, fully connected layers use all the information, so it can still learn spatial dependence as a CNN does, while CNNs for each neuron only look at a specific receptive field and will reuse that filter for all neurons in that channel. This constraint saves computation and allows wider and deeper models under some budget.

This is nice because the hypothesis of why CNN's work are, is that at each point in the network we care about looking at localized features rather than global ones and that creating a composition of these makes it so even if each neuron only relates to a handful of neurons in the previous layer, the receptive field from the initial image can still be quite large if not the whole thing.

Take away: CNNs are an efficient implementation of a vanilla NN, given the locality constraint that each neuron only looks at a small localized subset of neurons from the previous layer.


What kind of algorithm is the Levenberg–Marquardt algorithm?

In the context of Neural Networks, Backpropagation (with Gradient Descent, to use its full name) and Levengerg Marquardt are both members of the broader family of gradient descent algorithms. Backpropagation itself is not gradient descent, but it does the gradient climbing portion of a broader gradient descent algorithm.

You can imagine the function of a Neural Network as a function from its inputs to its outputs. If you were trying to solve, for example, a regression problem you can imagine this problem in multidimensional space with each training point corresponding to the coordinate provided by the values of its inputs and outputs (each of which represent a dimension). Then the entire training set you have for learning this regression problem become a set of points in this multidimensional space, and the function that your neural network performs is a curve in this multidimensional space. The closer this curve is to the points on the training set, the better it performs at the regression problem which essentially means we can generalize the task of training the neural network to a curve fitting problem.


Is neural networks training done one-by-one?

Should I be changing the weights/biases on every single sample before moving on to the next sample,

You can do this, it is called stochastic gradient descent (SGD) and typically you will shuffle the dataset before working through it each time.

  or should I first calculate the desired changes for the entire lot of 1,000 samples, and only then start applying them to the network?

You can do this, it is called batch gradient descent, or in some cases (especially in older resources) just assumed as the normal approach and called gradient descent.

Each approach offers advantages and disadvantages. In general:

SGD makes each update sooner in terms of amount of data that has been processed. So you may need less epochs before converging on reasonable values.
SGD does more processing per sample (because it updates more frequently), so is also slower in the sense that it will take longer to process each sample.
SGD can take less advantage of parallelisation, as the update steps mean you have to run each data item serially (as the weights have changed and error/gradient results are calculated for a specific set of weights).
SGD individual steps make typically only very rough guesses at the correct gradients to change weights in. This is both a disadvantage (performance of the NN against the objective on the training set can decrease as well as increase) and an advantage (there is less likelihood of getting stuck in a local stationary point due the "jitter" these random differences cause).

What happens in practice is that most software allows you to compromise between batch processing and single-sample processing, to try and get the best performance and update characteristics. This is called mini-batch processing, which involves:

Shuffling the dataset at the start of each epoch.
Working through the shuffled data, N items per time where N might vary from maybe 10 to 1000, depending on the problem and any constraints on the hardware. A common decision is to process the largest batch size that the GPU acceleration allows to run in parallel.
Calculate the update required for each small batch, then apply it.

This is nowadays the most common update method that most neural network libraries assume, and they almost universally will accept a batch size parameter in the training API. Most of the libraries will still call simple optimisers that do that SGD; technically it is true, the gradients calculated are still somewhat randomised due to not using the full batch, but you may find this called mini-batch gradient descent in some older papers.


Is it realistic to train a transformer-based model (e.g. GPT) in a self-supervised way directly on the Mel spectrogram?

The reason most music-generation models use discrete representations is because the long-term structures of music are very challenging to model. Note that MAESTRO (the dataset behind the two papers you linked) contains performances, not scores, so they include timing and accents of real performers--but are still sequences of discrete events, not audio.
There's been some work on learning discrete representations directly from audio, such as with vector-quantized variational autoencoders (VQ-VAEs). Typically an autoregressive model is trained on top of the learned representation; Jukebox used a transformer for that. By the way, I'd highly recommend reading through the "related work" section of the Jukebox paper for an overview of work on the audio/speech synthesis task.
wav2vec is probably closest to what you're describing. They train a transformer on raw audio, self-supervised, in order to learn good representations of human speech for the speech-to-text task.
As far as training directly on spectograms, there's MelNet, a somewhat exotic RNN trained for a variety of audio synthesis tasks, including music.
Hope this helps!


What is the reason for taking tuples as vectors rather than points?

They are equivalent. When we consider a particular instance as a vector, we are not literally imagining it as an arrow with it's head at the point coordinates and tail at the origin. It's just when you are working with a tuple of numbers in a mathematical context, it is conventional to call it a vector. This language follows into machine learning which is usually based on associated linear algebra.


Is there a mathematical theory behind why MLP can classify handwritten digits?

The approximation theorem says you can approximate anything. But this is kind of meaningless in so far as you can do KNN and get an arbitrary approximation of your training data also.

Proving CNN correctly extract features is, I don't think possible. Or if it is, something involving VC theory is probably the best you can do.


How do you program fear into a neural network?

There are a lot of approaches you could take for this. Creating a realistic artificial analog for fear as implemented biologically in animals might be possible, but there is quite a lot involved in a real animal's fear response that would not apply in simpler AI bots available now. For instance, an animal entering a state of fear will typically use hormones to signal changes throughout its body, favouring resource expenditure and risk taking ("fight or flight").

In basic reinforcement learning, the neural network would not need to directly decide switch on a "fear mode". Instead, you can make use of some design in the agent and learning algorithm to help learn from rare but significant events. Here are a few ideas:

Experience replay. You may already be doing this in the Pacman scenario, if you are using DQN or something similar. Storing the state transition and reward that caused a large positive or negative reward, and repeatedly learning from it should offset your concern
Prioritised sweeping. You can use larger differences experienced between predicted and actual reward to bias sampling from your replay memory towards significant events and those linked closely to them.
Planning. With a predictive model - maybe based on sampled transitions (you can re-use the experience replay memory for this), or maybe a trained state transition prediction network - then you can look multiple steps ahead by simulating. There is a strong relation between RL and look-ahead planning too, they are very similar algorithm. The difference is which states and actions are being considered, and whether they are being simulated or experienced. Experience replay blurs the line here - it can be framed as learning from memory, or improving predictions for planning. Planning helps by optimising decisions without needing to repeat experiences as much - a combination of planning and learning can be far more powerful than either in isolation.
Smarter exploratory action selection. Epsilon-greedy, where you either take a greedy action or take a completely random action, completely ignores how much you may have already learned about alternative actions and their relative merit. You can use something like Upper Confidence Bound with a value-based agent.
In a deterministic world, increase the batch size for learning and planning, as you can trust that when a transition is learned once, that you know everything about it.

You will need to experiment in each environment. You can make learning agents that are more conservative about exploring near low reward areas. However, if the environment is such that it is necessary to take risks in order to get to the best rewards (which is often the case in games) then it may not be optimal in terms of learning time to have a "timid" agent. For instance in your example of Pacman, sometimes the ghosts should be avoided, sometimes they should be chased. If the agent learned strong aversion initially, it might take a long time to overcome this and learn to chase them after eating a power-up.

For your example of the spider, as the constructor of the experiment then you know that the bite is bad every time and that the agent must avoid it as much as possible. To most RL algorithms, there is no such knowledge, except gained through experience. An MDP world model does not need to match common sense, it may be that a spider bite is bad (-10 reward) 90% of the time and good 10% of the time (+1000 reward). The agent can only discover this by being bitten multiple times . . . RL typically does not start with any system to make assumptions about this sort of thing, and it is impossible to come up with a general rule about all possible MDPs. Instead, for a basic RL system, you can consider modifying hyperparameters or focusing on key events as suggested above. Outside of a basic RL system there could be merit in replicating other things, such as "instinctive" fear.


Too small gradient on large neural network

I changed the layer from tf.contrib.rnn.LSTMBlockCell to tf.contrib.rnn.LayerNormBasicLSTMCell. Then the gradients become large enough to influence the network.


How do I solve the knapsack problem using the hill climbing algorithm?

I think there are at least three points that you need to think before implement Hill-Climbing (HC) algorithm:

First, the initial state. In HC, people usually use a "temporary solution" for the initial state. You can use an empty knapscak, but I prefer to randomly pick items and put it in the knapsack as the initial state. You can use a binary array to save the "pickup status" of each item such that if $a_i=1$ then you pick the $i$-th item and if $a_i=0$ then you don't pick the $i$-th item.

The next is the heuristic function. A function that evaluate your current state. Actually, you can define it freely (like the initial state) as long as the function can evaluate which state is better than the others. You can use a simple one:

$$
h =
    \begin{cases}
      -1, & \text{if}\ \sum (w_i \times a_i) > \text{MaxWeight} \\
      \sum (v_i \times a_i), & \text{otherwise}
    \end{cases}
$$

with $w_i$ and $v_i$ is the weight and the value of the $i$-th item respectively. One of the disadvantage of this function is if you have more than one invalid state (a state with the total weight is more than the limit), they have the same value, which is $-1$.

The last is the method to generate the neighbors of the current state. Once again, you are also free to define the method that you will use. I think your proposed actions are good enough:

reverse the status of an item (1 to 0 or 0 to 1). That means I add or reduce the item on the current state.
swap the status two items that have different value. Formally swap $a_i$ and $a_j$ with $ i \neq j$ and $ a_i \neq a_j$. That means I swap the item that I picked before with the item that I didn't pick before

Then, after you define everything you can run the algorithm as explained in Wikipedia:

currentNode = startNode;
loop do
    L = NEIGHBORS(currentNode);
    nextEval = -INF;
    nextNode = NULL;
    for all x in L
        if (EVAL(x) > nextEval)
            nextNode = x;
            nextEval = EVAL(x);
    if nextEval <= EVAL(currentNode)
        //Return current node since no better neighbors exist
        return currentNode;
    currentNode = nextNode;


Why is chess still a benchmark for Artificial Intelligence

Chess isn't really a benchmark per say.

The method developed in AlphaGo to play Go should in principle generalize quite nicely to other games of this sort, such as chess. Since Stockfish is quite dominantly the strongest Chess AI, the natural question would be to see how well AlphaGo's method compares to Stockfish.

Being one of the most well developed AI agents of all time, the situation concerning the defeat of Stockfish (AlphaZero was trained for only 4 hours entirely via self-play, without access to historical data) signifies the complete dominance of modern neural-network methods over classic methods (hard coded evaluation functions).

Also as @DukeZhou♦ mentioned in the comments, while Chess bots can regularly beat human players, it's still a useful metric to evaluate bots against each other via "games" of this sort.


How to approach this handwritten digit recognition?

I think one approach you can try to segment the digits and Connected Components Labeling (https://en.wikipedia.org/wiki/Connected-component_labeling).
With it, you'll end up with a label for each letter and then you can try to find the convex hull of the letter.
After that, just crop a square for each convex hull and input it to your CNN.
Notice that it will only if there is at least one pixel between the letters...


Can AI help summarize article or abstract sentence keyword?

Yes. Text summarisation has been a research topic in (computational) linguistics for literally decades. Have a look at the Wikipedia page on Automatic Summarisation for an overview.

There are basically various different approaches: either, selecting salient sentences (or parts of sentences) which represent the gist of the text, or, trying to 'understand' the text and generating new sentences. The former is generally easier, and works on any text, while the latter would probably be able to produce better results, but is more complicated and would not work on any text, as it would be specific to a particular topic.


How does best-first search differ from hill-climbing?

Best-first search
BFS is a search approach and not just a single algorithm, so there are many best-first (BFS) algorithms, such as greedy BFS, A* and B*. BFS algorithms are informed search algorithms, as opposed to uninformed search algorithms (such as breadth-first search, depth-first search, etc.), i.e. BFS algorithms make use of domain knowledge that can be encoded into a so-called heuristic function (that's why they are informed!).
Every BFS algorithm defines a so-called evaluation function $f$ of the following form
$$f(n) = g(n) + h(n)$$
for all nodes $n \in V$ (where $V$ is the set of nodes or states of the search space), where

$g(n)$ is the cost from the start node (of the search) to the node $n$, and

$h(n)$ (the so-called heuristic, which is the way of including domain knowledge to solve the search problem) is an estimate of the cost of the cheapest path from $n$ to the goal node.

Depending on how you define $f$ and, in particular, the heuristic function $h$, you get different BFS algorithms. For instance, A* is a BFS algorithm where $h$ is an admissible heuristic (i.e. it never overestimates the cost to the goal node). Because of this admissibility property, A* is guaranteed to find the globally optimal solution (i.e. the cheapest path from a start node to the goal node among all paths), but you can ignore this detail.
To apply a BFS algorithm, you need to define the evaluation function and the search space, i.e. the states (or nodes) and their connections. For example, if you want to find the cheapest path from Paris to Madrid, you need to define that Paris is the start node, Madrid the goal node and then you need to define all intermediate nodes between Paris and Madrid, but you also need to define $g$ and $h$.
Hill climbing
Hill climbing (HC) is a general search strategy (so it's also not just an algorithm!). HC algorithms are greedy local search algorithms, i.e. they typically only find local optima (as opposed to global optima) and they do that greedily (i.e. they do not look ahead). The idea behind HC algorithms is that of moving (or climbing) in the direction of increasing value. HC algorithms can be used to solve optimization problems and not just well-defined search problems, i.e. you start from some solution and you move to the best neighboring solution, and then loop.
Best-first search vs hill climbing

BFS algorithms are informed search algorithms (as opposed to uninformed)
BFS algorithms need to define the search space and the evaluation function
Some BFS algorithms (such as A*) are guaranteed to find the best global solution
HC algorithms are general (i.e. widely applicable) but local and greedy search and optimization algorithms; consequently, they are not generally guaranteed to find the global optimum (but, in practice, they may work well, depending also on the problem).
HC algorithms do not need to define the search space explicitly (i.e. you do not need to define the start and goal nodes, and so on), but you just need to a way of determining the best neighboring solution

Further reading
The book Artificial Intelligence: A Modern Approach by Stuart Russell and Peter Norvig provides more details about these two search approaches. (You can find free copies of this book online).


What are some (deep) reinforcement learning books for beginners?

Reinforcement Learning: An Introduction by Richard Sutton and Andrew Barto is undoubtedly one of the best books, to begin with. Despite its age, the book is still the canonical introduction to reinforcement learning. It does require some patience, but I think it's very approachable and rigorous at the same time!


Recognize carp and give them a unique id

I'm not sure I understood your question entirely so please correct me if i'm wrong. You're having all the carp tagged, so if I give you a picture of any of them you know exactly which one is in the picture right? If that is the case then you're dealing with a classic classification problem. One simple way of solving such problem will be to use CNN on the input image to extract features, and at the end of the network have N neurons, where each neuron matches one carp. Just apply a softmax over all the inputs to have a probability distribution, and select the highest value (or if the values is highest then a threshold). Regarding the number of labeled examples per class, having only one might be a problem. I would suggest you to look into few shot learning, which are trying to solve this exact problem (training decent models with limited training data).


Is it possible to state an outliers detection problem as a reinforcement learning problem?

It is quite often possible to frame a problem as a Reinforcement Learning (RL) problem at some level. However, this may turn out to be for no benefit, or a net cost towards solving the problem. Casting parameter or hyperparameter searches as RL can be adding a layer of complexity and reduce efficiency.

One key thing to bear in mind is that any classification or regression that occurs within a RL framework will end up using effectively the same models and approaches that could solve the same problems directly. These models would either appear directly as the function approximators in RL that implement policies or value functions, or they would be an implied part of them. If you have labeled data for classification - even delayed until some time after you collect data, then you are usually going to be better off using supervised learning directly.

For hyperparameter searches (e.g. cutoffs for anomaly detection) then you may not need labelled data, but just need a good way to test the model offline.

The first point at which supervised learning or classic anomaly detection might fail for you is if you never receive any feedback about individual records, only a measure of overall performance. In other words, if you can measure consequences of good performance, but never measure or check  correctness.

Your characterisation

  about 7-10% of data do not make any sense.

does not appear to fit that. It looks like you could detect this, maybe manually labelling a few thousand records, and train a classifier using supervised learning techniques. That is likely a much better use of your time than trying to restructure the problem at a higher level and trusting a trail-and-error approach to discover the same rules.

Putting that to one side, assuming you do have a problem where

data to be classified is arriving as a stream, and needs to be processed online, item by item or in small batches
you have reason to think that an accept/reject stage before processing further would be useful
you have no way to label training data for accept/reject
you have a way to measure performance of the remaining system after the accept/reject phase

then you could use RL to frame the accept/reject phase as an action. There are some challenges there, but essentially you would use RL along with measurement feedback to sample errors or gradients - typically using TD Error or policy gradients. This could wrap almost any model that does classification or anomaly detection etc, provided it could be trained using those gradients.

From comments, if the underlying distributions for accept/reject are non-stationary, this may point you more towards a RL solution. However, that may come with a cost to performance - you will need to balance exploration rate (which will reduce the performance of the model against stationary data) versus speed of learning new distributions. This is a problem for all online learners; the main advantage of a RL approach here is that it will not require generating new labelled data. If you can use a recency-weighted anomaly detection algorithm instead, then you won't need the labeled data either - whether that is better requires testing, personally I'd take a working anomaly detection as the baseline and only use RL if it proved itself better.

The specific items that you turn into states and rewards are not clear to me from the question, and you would need to work on these things carefully. It is possible you will need more than the current data item in order to define state, and that will depend a lot on how the feedback loop works that establishes reward.


Will attention based networks prevail over RNN and LSTM?

I think there are two parts to answering this question.

First, about the specific paper that has been mentioned.

The paper's title is hyperbolic, and probably written that way to get more people to read it. The paper itself does not make the claim that attention-based networks will supplant existing recurrent network architectures. Instead, it makes a more modest claim that the new attention-based architecture can outperform existing modes on a machine translation task (which is supported with experiments), and that, maybe, in the future, attention-based models could work well in other domains, or provide advantages in interpretability.

However, there's also a second way to interpret this question. Attention-based networks are getting a lot of buzz in the AI community this year and to some extent last year. It seems likely that there are a lot of research efforts underway to apply them to other kinds of problems. If these efforts pan out, then they could indeed become the tool of choice.

At this point however, the answer to whether they will has to be "Maybe". There is no good underlying theory for why some NN architectures work better than others on a given kind of problem. We have post-hoc explanations for some (like, for instance, CNNs), but in the absence of a general theory, we're better off waiting for the empirical results before making a call. In particular, the No Free Lunch Theorem tells us that if Attention-based networks work very well on one kind of problem, we should actually expect them to work poorly on others! I suspect we'll have a better answer to this question in 1-2 more years however.


Why are not validation accuracy and loss as smooth as train accuracy and loss?

You are training your model on the train set and only validating your model on CV set, thus your weights are getting exclusively optimised according to the loss of Training Set (in a continuous manner) and thus always decreasing. We do not have such guarantees with the CV set, which is the entire purpose of Cross Validation in the first place. Ideally it gives you an unbiased measure of how well your model and its trained weights will perform in the real world. Thus even if it performs well in Training Set, the loss can still go up in CV set which is what you are seeing in your graph.

Speaking in layman terms, even if you do all the sums of a single exercise given in a book, your performance might not be the same in a model paper. You do another exercise from another book, there is no guarantee your previous concepts will stick and you may do even worse in the model paper. Same thing is happening here where exercises are your training set and model papers are to evaluate your learning.


Is doing a master's in natural language processing combined with machine learning worth it?

The combination of NLP and ML is an interesting one. The most interesting work is not the mainstream systems employing feed forward, convolution kernel, and LSTM networks with word-to-vector input normalization or straight audio samples as input data.

The kinds that are most interesting are the kinds that more closely mimic the systems in which natural language developed, which are those of the language networks in the brain, including auditory, visual, and motor pathways and the kinds of cognitive abilities that support conversations beyond chat with no underlying comprehension or belief.

That some balanced rise and fall of beliefs and semantic associations are part of real conversation has been known for a few decades. The AI research directions along these lines include semantic nets, recursive artificial networks (an advanced type of RNN just entering the scene), and fuzzy belief models. Some of the current perspectives centered on letter, word, and grammar are obsolete from a linguistics point of view. Contemporary linguistics do not assume words are the elementary objects and formal grammar rules describe legal combinations of them. Smaller elements such as word roots, prefixes, suffixes, conjugations, phonetics, adjacency, and the dynamic aspects of language are more accurate modeling elements of real talk and writing.

Looking up some of these terms in an academic article search engine can pull up some of this research over the last two or three decades so that a historical background is clearer. Many of those articles have proposed systems, mathematics to describe the perspective, and sometimes algorithms and results of testing the modality. After spending a few days looking around, it will be easier to pick a master's thesis or a group of them to run past those at a university.

Applications for deeper forms of natural language capability beyond idiot talk chatbots is so profoundly needed, there will be little need to investigate the career options for most of these lines of research, provided the feasibility of developing components that fit into products and services is high for that line of research.


Why AI is (or not) a good option for the generation of random numbers?

AI approaches are generally not good for generating random numbers, for these reasons:

Similar to why they are not good for adding numbers, there already exist many strong pseudo-random and "true" random sources, possible without using any AI approach, and demonstrably good enough for purposes such as simulations and cryptography. These existing algorithms and devices perform much more efficiently at the task than any AI could.
Generating random numbers with good statistical properties is a very precise task, that the approximations typically used in machine learning cannot cope with (a pseudo-random function is inherently noisy and difficult to learn). AI search techniques are not helpful either, because the amount of output that needs to be generated in order to validate a random data source is very large, making things like tree searches for solutions intractable.

One possible relationship between AI and random number generation that could be explored is in code generation and code analysis for random number generators. In other words, an AI that is sophisticated enough to understand the complexity of code used in a typical random number generator (RNG) might help with making random number generation more efficient, or less easy to hack, or expose flaws in existing implementations. This is beyond the capabilities of AI techniques at the moment - or put another way, human intelligence has taken RNG technology quite far already, so there is very little practical that current AIs can do to improve on it.

  Would GANs be suited for this purpose?

No. The input to a GAN is typically a vector of random numbers that have been generated, by an already-existing RNG. The purpose the the GAN is convert this simply-distributed random vector input $\mathbf{x}$ - which might be suitable for many other purposes, such as running simulations of stochastic processes - into a $\mathbf{y}$ sampled from a distribution of some target population $\mathbb{Y}$, which is typically being expressed as a subset of a large space $\mathbb{R}^d$. The values that $\mathbf{y}$ will take are biased to a subspace (or manifold) of all possible space within the representations possible in that vector space. For instance, you might input a random vector with 100 variables and the GAN will generate a picture of a dog. It cannot generate anything other than attempts at pictures of dogs, whilst the space of all possible images that it might create if completely random is far larger (and usually not interesting to look at).

So a GAN would take an already-good random source and turn it into a biased set of numbers that would be generally unusable as a RNG without further processing. The kind of processing you would do (e.g. "noise whitening") is something that some current "secure" random generators do - meaning to use the GAN to generate quality random numbers you would literally put existing RNG code on its input and output layers. The GAN would add nothing but wasted computation.

You cannot even train a GAN to do this task, since a truly random space contains all possible outputs, there will be no negative examples for the discriminator. Although a random picture most often looks like static noise, and a picture of the Mona Lisa is clearly considered "not random" by most people, in fact it is just as likely to be seen as any other image.