Can a second network take as input the weights of a first network and help training the first network?

That is similar to a chicken-and-egg problem; If the second network could figure out the direction and learn faster, why couldn't the first network do that too and learn even faster.

However, I would recommend looking up Actor Critic Methods. Here there are two networks but one (the critic) evaluates the other (the actor) to head it in the right direction.


Alphazero policy head loss not decreasing

The loss of the policy head here is really quite different from losses in, for instance, more "conventional" Supervised Learning approaches (where we typically expect/hope to see a relatively steady decrease in loss function).

In this AlphaZero setup, the target that we're updating the policy head towards is itself changing during the training process. When we improve our policy, we expect the MCTS "expert" to also be improved, which may lead to a different distribution of MCTS visit counts, which in turn may lead to a different update target for the policy head from previous update targets. So it's perfectly fine if our "loss" increases sometimes, we may still actually be performing better. The loss is useful for the computation of our gradient, but otherwise it doesn't have much use -- it certainly cannot be used as an accurate indicator of performance / learning progress.

  but only 80 at max are legal at any single state (think chess - a lot of possibles but very few are actually legal at any given time). Also, if MCTS has 20 simulations, then the improved probabilities vector (against which we train our policy loss) will have at most 20 non-zero entries.

This can be a problem yes. The fact that the majority of moves are not legal at any point in time is not a problem, but if you only have 20 MCTS simulations for a branching factor of 80... that's certainly a problem. The easiest fix would be to simply keep MCTS running for longer, but obviously that's going to take more computation time. If you cannot afford to do this for every turn of self-play, you could try:

using only a low MCTS iteration count for some moves, not adding these distributions to the training data for the policy head
using a larger MCTS iteration count for some other moves, and only using the distributions of these moves as training data for the policy head

This idea is described in more detail in Subsection 6.1 of Accelerating Self-Play Learning in Go.


How does high entropy targets relate to less variance of the gradient between training cases?

Since it is a trained network already, when you run an example through it, the gradient will not have a very high variance.
The gradient varies a lot when you are training a network from the scratch but then it stops varying much since it understands the pattern.


Is data leakage relevant when scaling across samples?

Considering that you are making a minmax scaling, the only time in which there would be no risk of data leakage is if the minimum value on your training set equals the minimum value of the test set, and your maximum value on the training set equals the maximum value on the test set.
In that circumstance the result of your scaling would be exactly the same as fitting the scaler to the training set, and applying it to the test set.
There is basically no advantage in scaling them separately, and you take the risk of data leakage if their value ranges differ. I don't know much about genomic data, but even if your assumption was correct I would recommend not to do that.


Aside from specific training sets, what distinguishes the capabilities of different AI implementations?

If I understand what you mean correctly then the answer is basically nothing. Fundamentlaly all ML algorithms are ding the same thing, which is to optimize some weights for a certain output (this is true even for non-parametric methods in an implict way, but lets not dive too deep here). The only differences are:

Dataset the model is trained on
Specific dimensionality of inputs and outputs to make them compatible with the input data and the output labels.
Complexity that can be expressed by the model (the number of weights and their structure in layers in the case of Neural Networks).
Changes to the optimization process (gradient clipping, regularization, ecc.)
Structural changes to the architecture that embed assumptions about the specific problem setting (e.g. 2D-Convulutions embed assumptions for images, softmax activations embeds the assumption for classification with probabilities, hidden states embed the assumptions about memory and "fogetting", ecc...)

Therefore, if you have a "similar" problem setting where you can assume that 3, 4, 5 can be the same without problems, then you can just make the appropriate changes in 2 and change the dataset (1) to get a model that works on something different.
Of course, being able to tell how "similar" a problem is to another and what possible things could be different is quite tricky and requires a lot of knowledge about the domain of Machine Learning and how every algorithm works and why.
In essence, I'm saying that in principle you could take a model and train it for a new setting with very minor and simple changes that don't require a lot of knowledge. However, without the wider knowledge on the field of ML/AI you won't be able to tell if what you are doing is ok and how well can it work in general.


What is the target output for updating a Deep Q Network

As you say, the output of a $Q$ network is typically a value for all actions of the given state. Let us call this output $\mathbf{x} \in \mathbb{R}^{|\mathcal{A}|}$. To train your network using the squared bellman error you need first calculate the scalar target $y = r(s, a) + \max_a Q(s', a)$. Then, to train the network we take a vector $\mathbf{x'} = \mathbf{x}$ and change the $a$th element of it to be equal to $y$, where $a$ is the action you took in state $s$; call this modified vector $\mathbf{x'}_a$. We calculate the loss $\mathcal{L}(\mathbf{x}, \mathbf{x'}_a)$ and back propagate through this to update the parameters of our network.
Note that when we use $Q$ to calculate $y$ we typically use some form of target network; this can be a copy of $Q$ where the parameters are only updated every $i$th update or a network whose weights are updated using a polyak average with the main networks weights after every update.
Judging by your code it looks as though your action selection is what might be causing you some problems. As far as I can tell you're always acting greedily with respect to your $Q$-function. You should be looking to act $\epsilon$-greedily, i.e. with probability $\epsilon$ take a random action and act greedily otherwise. Typically you start with $\epsilon=1$ and decay it each time a random action is taken down to some small value such as 0.05.


Siri and Cortana are AI to some extent. The usual label is "weak AI" (also called "narrow" or "soft" AI). It turns out the Wikipedia article on Weak AI explicitly refers to Siri:

Siri is a good example of narrow intelligence. Siri operates within a limited pre-defined range, there is no genuine intelligence, no self-awareness, no life despite being a sophisticated example of weak AI. In Forbes (2011), Ted Greenwald wrote: "The iPhone/Siri marriage represents the arrival of hybrid AI, combining several narrow AI techniques plus access to massive data in the cloud." AI researcher Ben Goertzel, on his blog in 2010, stated Siri was "VERY narrow and brittle" evidenced by annoying results if you ask questions outside the limits of the application.

Important to note that "mixing" weak AIs does not make a "stronger" AI, by some arguments (see Searle's Chinese Room argument), but there is no consensus on this topic yet.


I'd call any system that receives input to be "perceiving", and that can constitute a basic definition of "awareness" or even "consciousness", distinct from "self-awareness" or consciousness in the human sense, which we tend to think of in terms of sentience.

What the Chinese Room seems to argue is that sentience is based on qualia, which may be difficult or impossible to validate in another entity.  (With humans, we can assume another individual is sentient because we perceive ourselves to be sentient, and share the same medium of human brains and bodies.)

It seems increasingly likely that "algorithms sufficiently advanced" will one day be able to model human cognition, but the real question will be is it actual cognition in the sense of understanding, or mere imitation? ("Semantics" vs. "Syntactics")

Cognition is derived from the Latin cognosco, itself related to the Greek words such as γνωστός (a root of "gnostic")

The question and definition of sentience is less clear cut, but the argument re: algorithms relates to cognition thus:

"The lights may be on, but is anybody home?"


A good machine learning object detector, would be helpful in finding the logs but classical image processing would be required to accurately find the log profiles.  Neural networks are generally not used for computing geometric dimensions.  In addition, you will need to have a calibration method so that you can convert pixels to some distance unit (e.g. inches, centimeters, etc.).

The Timbeter app seems to do what you need (see below).

This app (see below)  is interesting although it doesn't solve your problem.  It appears good at counting but finding the profiles to within an inch is a bigger problem.

This blog "HOW MANY LOGS CAN A WOODCHUCK COUNT?" goes over the image processing challenges of counting logs.  The author doesn't completely solve the problem.

There is a post on Stackoverflow, How to count white object on Binary Image?
 which makes an attempt.


I think this is an interesting approach, but it's hard to tell if it can make better or faster learning networks.

If I understand correctly, this could be looked at as using a function returning a tuple as activation function. The number of elements in the tuple must be equal to the number of neurons in the next layer.

For example you could use an activation function like f(x) = (ReLU(x), sigmoid(x)) if the next layer contains two neurons.

I think the biggest problem is to determine why one value should be passed to exactly THAT neuron and another value to another neuron - why not opposite? This must either be determined randomly or by doing some advanced analysis in beforehand.

Anyway, I think it's an interesting idea which should be tested out, but it's hard to say whether it will give better or worse results before trying it out.


Is this a correct visual representation of a recurrent neural network (RNN)?

I will answer this question, leaving it open to challenge by anyone more knowledgeable.

The equations to update each layer of an RNN are

$y_t = \sigma(W_x x_t + b_x )$

and

$h_t = \sigma(W_hx_t + b_h)$

Where $h_t$ is the hidden layer (in blue in the picture), and $y_t$ is the output layer (red in picture). This equations says that every single component in the hidden layer vector, i.e. every unit in the hidden layer, is a function of the linear combination of the $x_t$ vector, which is the first yellow row along the depth axis. In other words, all the first yellow input nodes on the bottom left in the picture.

Thus, technically this picture is not correct, all the yellow nodes should have points to all the blue nodes. Also, by similar reasoning, all the blue nodes in the subsequent steps of the hidden layer should be connected to all the blue nodes of the previous layer.

Of course, that would make for a much uglier/harder picture to make, so I don't blame the authors, although this has given me a few hours of confused research, which I guess still meets their educational goal.


Restoration of localized damaged areas (time signals, but guess also applicable to images)

So , you want to remove the "noise" (damaged signal) from the signal, by giving the perfect signal as output. This model is called as a "denoising autoencoder". If you have enough training data, the noise is not at all a problem for the neural network , if you really feel that the network is capturing the noise , then you have to increase the training epochs , or sometimes it can be a bad initialized network.


How can I train a RL agent to play board games successfully without human play?

The approach will vary depending on some features of the game:

How many players (two for tic tac toe and many classic games).
Whether it is a "perfect information" game (yes for chess and tic tac toe), or whether there is significant hidden information (true for most forms of poker)
Whether the game is zero sum. Any game with simple win/loss/draw results can be considered zero sum.
Branching complexity of the game. Tic tac toe has relatively low branching factor, at worst 9, and reduces by one each turn. The game of Go has branching factoraround 250.
Action complexity of the game. A card game like Magic the Gathering has huge complexity even if the number of possible actions at each turn is quite small.

The above traits and others make a large impact on the details of which algorithms and approaches will work best (or work at all). As that is very broad, I will not go into that, and I would suggest you ask separate questions about specific games if you take this further and try to implement some self-playing learning agents.

It is possible to outline a general approach that would work for many games:

1. Implement Game Rules

You need to have a programmable API for running the game, which allows for code representing a player (called an "agent") to receive current state of the game, to pass in a chosen action, and to return results of that action. The results should include whether any of the players has won or lost, and to update internal state of the game ready for next player.

2. Choose a Learning Agent Approach

There are several kinds of learning algorithms that are suitable for controlling agents and learning through experience. One popular choice would be Reinforcement Learning (RL), but also Genetic Algorithms (GA) can be used for example.

Key here is how different types of agent solve the issues of self-play:

GA does this very naturally. You create a population of agents, and have them play each other, selecting winners to create the next generation. The main issue with GA approaches is how well they scale with complexity - in general not as well as RL.
With RL you can have the current agent play best agent(s) you have created so far - which is most general approach. Or for some games you may be able to have it play both sides at once using the same predictive model - this works because predicting moves by the opposition is a significant part of game play.

2a. How self-play works in practice

Without going into lines of code, what you need for self-play is:

The game API for automation and scoring
One or more agents that can use the API
A system that takes the results of games between agents and feeds them back so that learning can occur:

In a GA, with tournament selection, this could simply be saving a reference - the id of the winner - into a list, until the list has grown large enough to be the parents for the next generation. Other selection methods are also possible, but the general approach is the same - the games are played to help populate the next generation of agents.
In RL, typically each game state is saved to memory along with next resulting state or the result win/draw/lose (converted into a reward value e.g. +1/0/-1). This memory is used to calculate and update estimates of future results from any given state (or in some variants used directly to decide best action to take from any given state). Over time, this link between current and next states provides a way for the agent to learn better early game moves that eventually lead to a win for each player.
An important "trick" in RL is to figure out a way for the model and update rules to reflect opposing goals of players. This is not a consideration in agents that solve optimal control in single agent environments. You either need to make the prediction algorithm predict future reward as seen from perspective of the current agent, or use a global scoring system and re-define one of the agents as trying to minimise the total reward - this latter approach works nicely for two player zero sum games, as both players can then direcly share the same estimator, just using it to different ends.

A lot of repetition in a loop. GA's unit of repetition is usually a generation - one complete assessment of all existing agents (although again there are variants). RL's unit of repetition is often smaller, individual episodes, and the learning part of the loop can be called on every single turn if desired. The basic iteration in all cases is:

Play anything from one move to multiple games, with automated agents taking all roles in the game and storing results.
Use results to update learned parameters for the agents.
Use the updated agents for the next stages of self-play learning.

3. Planning

A purely reactive learning agent can do well in simple games, but typically you also want the agent to look ahead to predict future results more directly. You can combine the outputs from a learning model like RL with a forward-planning approach to get the best of both worlds.

Forward search methods include minimax/negamax and Monte Carlo Tree Search. These can be combined with learning agents in multiple ways - for instance as well as in planning stages using the RL model, they can also be used to help train a RL model (this is how it is used in Alpha Zero).


What could I do to this CNN to achieve a higher accuracy on the cifar10 dataset?

You results show signs of overfitting at around epoch 40. In order to overcome this you can either simplify the model somewhat or increase regularization. You do not share what values you are using for dropout regularization so you can try increasing that.

But to be honest, I am not sure if that is going to help. You are using dropout in a pure CNN architecture, which I do not see that often in recent models. When they exists, they are usually at the very end of the network, or after densely connected layers. There are claims that regular dropout does not work as intended for convolutional layers, so the idea of spatial dropout is developed.

But you are using them after pooling layers and such a design may not be as bad as using them after convolutional layers. In any case, if you check the top performers on this dataset, you can see there are some who are trying to come up with new regularization techniques for CNN's, such as shake-shake, shakedrop and cutout.

However, given you already have an overfitting model, removing regularization might not be the best idea. But if you replace the layers after (and including) the last convolution with some fully connected layers, you can use dropout regularization on them instead. This would remove the largest convolutional layer of the model and you can add dense layers without increasing model size.

For image recognition tasks of any kind, easiest way to achieve high accuracy is via transfer learning.


What is the input to the left most LSTM cell c(t-1) and h(t-1)?

Your question is related to the initial states of LSTM, where c(t-1) is the cell state (memory) and h(t-1) is the previous LSTM block output.
As pointed out here, it is reasonable to assume that those are random values.


Question classification according to chapters

You have got text which have words. Now each of the words have dependency with words that are occurring previously or/and with the ones occurring later. So to capture the "context" in which a particular word occurs is done by sequence models in deep learning. Sequence models that are quite used often are Recurrent Neural Networks (RNN), Gated Recurrent Units (GRU) and Long Short Term Memory (LSTM) listed in the order of their functionality and complexity. To know more about them visit Jonathan Hui's blog.

Your problem is a classification problem i.e. given a sentence the classifier should tell which chapter and section it belongs to. You can use one of the sequence models to "encode" the sentence as a vector (based on the context of words in it and meaning of sentence) and pass that vector to a fully-connected neural network that has a softmax layer at the end to tell which chapter and section the text belongs to. The number of classes you are having are C * S (one for each chapter-section pair). From the output of softmax layer you can pick up the top 3 classes that got the highest probability for a sentence.

Doc2Vec converts a paragraph (can also be a sentence) into a vector representation. The Doc2Vec model has been trained over a huge corpus of text to be able to capture the context of a text and represent it as a high dimensional vector.

The corpus of text that you have, is from a probability distribution is quite different from that used for training Doc2Vec model. Sequence models (mentioned above) can learn that probability distribution for your text corpus exclusively. In other words, you have a tailor-made representation for your text so that the context can be understood much better.

That is why I suggested you to use sequence models. You can also compare the performance obtained by using pretrained Doc2Vec as text representation with that obtained by using one of the sequence models.

To know about the effectiveness of RNN visit Andrej Karpathy's blog.


What are the latest methods to train a chat bot?

Your question is incredibly broad -- so in response, two broad frameworks I'd encourage you to look at are:

For cutting edge chatbot conversation development http://rasa.ai is an open source framework that is more adaptable than more traditional rule-based systems
For speech recognition check out https://discourse.mozilla.org/c/deep-speech that is also open source.


Is it possible to pre-train a CNN in a self-supervised way so that it can later be used to solve an instance segmentation task?

Is it possible to use SSL to pre-train e.g. a faster R-CNN on a pretext task (for example, rotation), then use this pre-trained model for instance segmentation with the aim to get better accuracy?

Yes, it's possible and this has already been done. I don't know the details (because I have not yet read those papers), but I will provide you with some links to some potentially useful papers (based on their titles and abstracts) and associated code.

Learning Instance Segmentation by Interaction (code, 2018)
Self-supervised Transfer Learning for Instance Segmentation through Physical Interaction (code, 2019)
White Matter Tract Segmentation with Self-supervised Learning (code, 2020)
Self-supervised Depth Estimation to Regularise Semantic Segmentation in Knee Arthroscopy (code, 2020)
Simulation of Brain Resection for Cavity Segmentation Using Self-supervised and Semi-supervised Learning (code, 2020)

You can probably find more relevant papers here, where I also found some of the just cited papers.
The pre-text tasks designed in these papers could be useful in your case, but it may also turn out that you need to develop other pre-text tasks or combine multiple of them.
Maybe you can start from some pre-trained faster R-CNN or some appropriate model for instance segmentation (that you can find on the web, for example, here), which has been pre-trained on some imagery data similar to yours (either with SSL or by other means), then try to fine-tune this model with your labeled dataset for instance segmentation, and see if you get better results than just training a faster R-CNN from scratch. Eventually, if this pre-trained model does not lead to higher performance, you could pre-train it yourself with some SSL technique that you can come up with or one that is described in the literature. Of course, you should probably use a pre-trained model that has been pre-trained with data that is relevant for your downstream task (i.e. the instance segmentation task). You didn't describe the details of your unlabelled and labeled data, so I cannot be more specific (and I wouldn't currently be able to, in any case, because I didn't fully read those papers, and my experience with SSL techniques is mostly theoretical).
For more info about SSL, take a look at this and this answers.


Is any classifier not subject (or less susceptible) to fooling?

I would say this is not necessarily a duplicate but quite similar to some other questions. However, I will answer the question posed here.

At a theoretical level, what you are asking is there any algorithm that cannot be tricked into predicting the wrong class?

The answer is: No

It is analogous to asking whether there is a perfect architecture for an arbitrary classification problem, and that answer is quite obviously not. It would also not likely to be terribly difficult to show this is provably the case, at least for a class of algorithms(i.e connectionist models).


How to create vector representation of roadmap like scans

It turned out that my intuition was not far off. The skeletonization is a good step. The Hough transform though is not a good way to create a graph of the roadmap. It seems that the Ramer–Douglas–Peucker algorithm can help out here. This algorithm first takes all the skeleton pixels as input, and sees this as a starting graph. The algorithm then proceeds to remove intermediary pixels that do not add information to the shape of the graph.


Are there any easy ways to create annotated training images for object detection?

From what I read in the papers and my experience, the trend lately is to generate synthetic data using a photorealistic 3D graphics engine for automating the image labelling. The process would be:

Get a visually realistic 3D engine such as Unreal, Unity... any videogame engine would work
Build 3D models of the objects you want to detect (or ask for help to a 3D artist)
Build a 3D environment for placing the objects 3D models
Generate random instances of the objects in the environments with different environmental conditions: lighting, brightness, position, rotation...

Since you have the 3D model of the object and the whole control in the 3D engine you can annotate images, bounding boxes, pixels... or whatever stuff you want in an automated way.

This is an approach for automatic image labeling with synthetic data. Automating the labeling of real data is a challenge yet to be tackled.


Would Google's self-driving-car stop when it sees somebody with a T-shirt with a stop sign printed on it?

Google’s self-driving car most likely uses mapping of traffic signs using google street view images for roadway inventory management. If traffic signs are not in its database, it can still “see” and detect moving objects which can be distinguished from the presence of certain stationary objects, like traffic lights. So its software can classify objects based on the size, shape and movement patterns. Therefore it is highly unlikely that a person would be mistaken for a traffic sign. See: How does Google's self-driving car identify pedestrians?

Image: Technology Review

To support such a claim, Illah Nourbakhsh, a professor of robotics at Carnegie Mellon University, gave an interview to the New York Times magazine cover story on autonomous driving cars, and includes this hypothetical scenario, saying:

  If they’re outside walking, and the sun is at just the right glare level, and there’s a mirrored truck stopped next to you, and the sun bounces off that truck and hits the guy so that you can’t see his face anymore — well, now your car just sees a stop sign. The chances of all that happening are diminishingly small — it’s very, very unlikely — but the problem is we will have millions of these cars. The very unlikely will happen all the time.

Even so, the risk would be minimal, since the car is always looking out for traffic, pedestrians and other obstacles.

Sources:

How Google's self-driving cars see the world
The Dream Life of Driverless Cars at The New York Times


Why do most deep learning papers not include an implementation?

The paper's authors needed to implement their models anyway in order to conduct their experimentations, so why not publish the implementation?

Some papers and authors actually provide a link to their own implementation, but most of the papers (that I have read) don't provide it, although some third-party implementations may already be available on Github (or other code-hosting sites) when you are reading the paper. There may be different reasons why the author(s) of a paper don't provide a reference implementation

They use some closed-source software or maybe it makes use of other resources that cannot be shared
Their implementation is a mess and, so, from a pedagogical point of view, it's quite useless
This may encourage other people to try to reproduce their results with different implementations, so it may indirectly encourage people to do research on the same topic (but maybe not providing a reference implementation could actually have the opposite effect!)

Plus, if the implementation is not added and there's no reproducibility, what prevents authors from forging results?

Actually, that's a good question that only researchers that have already published something or some reviewers can say something about it. I would also like to know their take on this!
Nevertheless, from some reviews of papers I have read (e.g. on OpenReview), in most cases, the reviewers are interested in the consistency of the results, the novelty of the work, the clarity and structure of the paper, etc. I think that, in most cases, they probably trust the provided results, also because, often, for reproducibility, researchers are expected to describe their models and parameters in detail, provide plots, etc., but I don't exclude that there are cases of people that try to fool the reviewers. For example, watch this video where Phil Tabor comments on ridiculous attempts to fool people and plagiarism by Siraj Raval.


Auto-regression - Reduce error in prediction

The predictions tend to move towards the mean of the series as one predicts for longer horizons. Also, in general, optimal long range forecast is the process mean.

In other words, the past of the process contains no information on the development of the process in the distant future.

And, this might be the reason that you are getting poor forecasts.

  What changes should be done in the approach? or Do I have to change the model?

You might want to move to ARIMA models. See how they perform.

If you have some other time series that might act as explanatory variables then you might want to augment the dataset and use the ARIMAX model.

Third option would be to try out RNN if you have lots of data. You can also try hybrid models like ARIMA and RNN together.


Convolutional neural nets and reduction of the layers

The reason why you go from 16 to 3 channels is that by convention filters span the entire depth of the input. Therefore, your filters would actually be 7x7x16 in order to cover all channels of the input.

Detailed procedure

The output of the convolution automatically has a depth equal to the number of filters (so in your case this is 3) because you have an (m x k) filter matrix, where m is the number of filters and k is the number of elements in the unrolled filter (in your case, m = 3 and k = 7 x 7 x 16 = 784, so the filter matrix is 3 x 784).

The input is usually unrolled according to the im2col procedure, where each tile corresponding to a single filter location is stretched into a column equal to the unrolled filter size. This is repeated for each filter location, so you end up with very large matrix of size (k x n), where k is the same as k above in the filter matrix, and n depends on your padding and stride.

Multiplying the (m x k) filter matrix with the (k x n) input matrix gives you an (m x n) output matrix, where m is the number of filters.

Further reading

You can find some very nice visual explanations of the convolution procedure here and here.


Choosing an AI method to recreate a given binary 2D image

This sounds like a problem that's best solved with a simple non-AI algorithm. If you just enumerate the coordinates in a regular order (rows, colums, zig-zag, hilbert curve) and emit the coordinates where the image has a '1' you're meeting your requirements. Is there any specific reason you want to use an AI algorithm which is most likely worse than this?


I don't know if there is an existing pretrained NN that does this but it wouldn't be very hard to modify one to do this.

First, I'd take a pretrained image classification NN (e.g. VGG, ResNet), drop its final layer and replace it with one with 4 neurons, representing the 4 orientations (so that you know which way to rotate it).

Then I'd take again a dataset of regular images (e.g. a subset of ImageNet) and assume that they are correctly oriented. I'd make three more duplicate datasets with the same images rotates by 90, 180 and 270 degrees respectively. These 4 datasets would be the 4 classes I'd fine tune the model on.

By training your model on this dataset, you'll be training it to recognize which side your image is facing. Since it's a pretrained net and its a fairly simple task, I think that after a few iterations, your model will have converged. Then you could write a script that uses this model to predict an image's orientation and rotate it accordingly.


How are training hyperparameters determined for large models?

In general, it is definitely very computationally expensive, so an exhaustive search is not performed in practice. However, there are some recent approaches for determining whether the architecture is "fine" without training the neural network first - by looking at the covariance matrix after forwarding the data, for example, in a recent paper Neural Architecture Search without Training. However, such an approach is very limited.


Backpropagation of neural nets with shared weight

If we write $ H^2 = W^{h}H1 + W^{x}X^{2} $ then it will be better to understand the backward propagation step.

Now,

$\frac{\partial}{\partial W^{x}} W^{h}W^{x}X^{1}$ can be written as:
$\frac{\partial H^2}{\partial H^1}\frac{\partial H^1}{\partial W^{x}} $

$\frac{\partial H^2}{\partial H^1} =  (W^h)^T$ and
$\frac{\partial H^1}{\partial W^{x}} = (X^{1})^T $

Therefore,

$\frac{\partial}{\partial W^{x}} W^{h}W^{x}X^{1} = (W^h)^T(X^{1})^T  $

I hope it has solved your problem.


What size of neural networks can be trained on current consumer grade GPUs? (1060,1070,1080)

Usually the problem is to fit the model into video RAM.
If it does not, you cannot train your model at all without big efforts (like training parts of the model separately).
If it does, time is your only problem. But the difference in training time between consumer GPUs like the Nvidia 1080 and much more expensive GPU accelerators like the Nvidia K80 are not very large. Actually the best consumer cards are faster than GPU accelerators, but lack other properties like VRAM. Random comparisons and benchmarks: Which GPU(s) to Get for Deep Learning: My Experience and Advice for Using GPUs in Deep Learning, A Comparison between NVIDIA’s GeForce GTX 1080 and Tesla P100 for Deep Learning.
To calculate if your models fits into VRAM, you just approximate how much data and which hyperparameters you have (inputs, outputs, weights, layers, batch size, which datatype and so on).


What should I do when I have a variable-length sequence when instantiating an LSTM in Keras?

In Keras, what you specify is the hidden layer size. So :

LSTM(128)

gives you a Keras layer representing a LSTM with a hidden layer size of 128.

As you said :

  From the LSTM/RNN unfolding image or description, I found that each RNN cell take one time step at a time

So if you picture your RNN for one time step, it will look like this :

And if you unfold it in time, it look like this :

You are not limited in your sequence size, this is one of the feature of RNN : since you input your sequence element by element, the size of the sequence can be variable.

That number, 128, represent just the size of the hidden layer of your LSTM. You can see the hidden layer of the LSTM as the memory of the RNN.

Of course the goal is not for the LSTM to remember everything of the sequence, just link between elements. That's why the size of the hidden layer can be smaller than the size of your sequence.

Sources :

Keras documentation
This blog

Edit

From this blog :

  The larger the network, the more powerful, but it’s also easier to overfit. Don’t want to try to learn a million parameters from 10,000 examples – parameters > examples = trouble.

So the consequence of reducing the size of hidden state of LSTM is that it will be simpler. Might not be able to get the links between the element of the sequence. But if you put a too big size, your network will overfit ! And you absolutely don't want that.

Another really good blog on LSTM : this link


How to calculate the confidence of a classifier's output?

Obvious answer for a binary (2 classes)classification is .5. Beyond that the earlier comment is correct. One of the things I have seen done is to run your model on the test set and save the prediction probability results.  Then create a threshold variable call it thresh. Then increment thresh from 0 to 1 in a loop. On each iteration compare thresh with the highest predicted probability prediction call it P. If  P>thresh declare that as the selected prediction, then compare that with the true prediction. Keep track of the errors for each value of thresh. At the end select the value of thresh that has the least errors.
There are also some  more sophisticated methods for example "top 2 accuracy" where thresh is selected based on  having the true class within either the prediction with the highest probability or the second highest probability . You can construct a weighted error function and select the value of thresh that has the net lowest error over the test set.
For example an error function might be as follows. If neither P(highest) or P(second highest) = True class, error=1. If P(second highest) = true class, error=.5. If p(highest)=true class error=0. I have never tried this myself so I am not sure how well this works. When I get some time will try it on a model with 100 classes and see how well it does. I know in the Imagenet competition they evaluate not just the top accuracy but also the "Top 3" and "Top 5' accuracy. In that competition there are 1000 classes.
I never thought of this before but I assume you could train your model specifically to optimize say the Top 2 accuracy by constructing a loss function used during training that forces the network to minimize this loss.


Machine learning with graph as input and output

You can flatten the graph into a matrix and then train it like a normal neural network input. Perhaps an adjacency graph or maybe simply a series of linear equations representing the nodes and convert it into matrix form.


Difference in UCB performance when scaling the rewards

Epsilon greedy is unaffected by scaling of rewards, it always selects a random action with a probability of epsilon.
On the other hand, if we look at the formulation of UCB (Section 2.7 of Reinforcement Learning, Sutton and Barto):
$$A_t \doteq  \underset{a}{\operatorname{argmax}} [\mathcal{Q}_t(a) + c \sqrt{\frac{\ln t}{N_t(a)}}]$$
Where $Q_t(a)= \frac{R_1 + R_2 + \dots + R_{n-1}}{n-1}$ is the average of rewards seen so far for that action. By scaling the rewards, for example by 1/10, you are essentially scaling $Q_t(a)$ by 1/10 which will change UCBs behaviour to favour the exploratory right hand term $c\sqrt{\frac{\ln t}{N_t(a)}}$. If you want to offset your scaling of the rewards you should scale c by the same amount. Theoretically, I believe this should result in performance the same as that achieved pre-scaling.
In fact, another way of looking at it is that when using UCB, you could dispense with the $c$ constant altogether and simply scale the rewards to tune the exploitation/exploration trade-off. Smaller rewards favour exploration, larger rewards favour exploitation.


Could artificial general intelligence harm humanity?

This is a very, very controversial topic. But here's my opinion on the topic:

AI won't harm humanity (at least, to the level that a lot of people think it will).

As Kartik mentioned in his answer, AGI (Artificial General Intelligence) is the concept of AI that's become so "generally" intelligent, that it can do everything we as humans can, and more, but do it better, more efficiently, more accurately, you name it.

I'd say that AGI is (so far) a fictional scenario. Why? Because of the physical limitation of computing that wouldn't let us get there. In order to simulate natural, physical intelligence, you wouldn't be able to use gradient descent, you would need to simulate physical interactions of neurons. Simulating even a single molecule on the fastest supercomputers takes a very long time, because the overhead introduced with taking the real, physical world, transposing it onto mathematics, and running those mathematics on electronics, are too much. So simulating the complete working of just a small cluster of neurons would be almost infeasible.

Then you might ask: "what if we were to get enough computing power in the future?". It's a valid question, however, there's one problem: we're already (possibly) quite close to the physical limit of how small we can get our transistors before Quantum Tunnelling kicks in.

So I'd say it's impossible because of a few key reasons:

Current algorithms are intrinsically guided by human-generated data (even reinforcement learning algorithms need human-engineered rewards to understand what's good and what's bad). Therefore, there's no such thing as a self-guided objective for machine learning today.
Computing power isn't there yet, and most probably, will never get near the level required for AGI.
Humans have an innate bias to "humanize" non-living entities. Language, for example, is a great representation of the way our minds work on a fundamental, subconscious level. When we say "my computer isn't co-operating with me", or something along those lines, we don't realize that computers are big calculators with a screen and lots of predefined operations to run.

One thing I don't disagree with: Machine Learning will replace a lot of jobs - but not nearly as many as you may think! It'll take time - if Tesla eventually comes out with a fully-autonomous semi-truck, it's not like the whole industry will adopt to it overnight. Humans, and our genome, will have time to adapt and evolve overtime. We'll free up more people, over time, to get into fields like healthcare, education, and entertainment. When we don't have the option to get jobs like cashiers, drivers, etc., we'll be forced to get more advanced jobs.

This is just how technology has worked, and will continue to work for as long as humanity exists.

People, however, can use AI/ML for negative purposes and bad things. Face swapping, automated social engineering, etc., are all great examples of this. But with powerful unregulated/illegal technology, comes even more powerful regulated/legal technology.

As pointed out in the comments, transistors are not the only solution. There are alternatives like Quantum Computing, that enable you to remove that mathematical overhead, and directly run your operations on qubits. However, there's still a lot of work to be done to see if technology like AGI or self-guided machine learning is possible even with quantum computers.

None of this is certain - anything I've just mentioned can change in the next few weeks, months, or years.


How to evaluate the performance of an autoencoder trained on image data?

I will answer my own question to try and provide some insights.
My research supervisor suggested that I should use the SSIM metric or some other well-known image processing metric (see the book "Modern Image Quality Assessment" by Wang and Bovik) for assessing the visual similarity of an images.
Another way I evaluate the performance of an autoencoder is by simply visually comparing the input and output images taken from the test set. This is by no means very scientific, but it gives a good idea whether an autoencoder is able to reconstruct the input images. One thing I would add here is that even if an autoencoder can reconstruct images perfectly, it doesn't mean that the encoding it learned is useful. For example, when I wanted similar images to be mapped to similar encodings, the autoencoder that was able to do that better was outputting more blurred reconstructed images in comparison to the autoencoder that wasn't achieving this similarity preservation (but was outputting better reconstructions).


How to make deepfake video without a fancy PC?

Yes. There are services that provide free environment to run jupyter notebooks for research purposes (with GPU included, which is crucial for neural networks) - such as Google Colaboratory and Kaggle Kernels. Although they limit how long your computation may run (12 and 6 hours accordingly), which adds some difficulties to the process, although I think it is possible to bypass these restrictions.


How do I determine which relevant features have been learned during training in a CNN?

There are methods called "scoring systems" where you give a image scores such as "0.9 stripes, 0.0 red, 0.8 hair, ..." and use those scores to classify objects. It's an older idea, not used to determine if the network is learning. It's not in a standard CNN.

To determine if relevant information is being learned or not, it's standard to use the testing accuracy, training accuracy, confusion matrix, or AUC.

Determining what exactly a CNN is learning is a complicated research problem that's ongoing. In short - you can't really know. For a basic network, you can tell that it is learning something but not what it's actually using to make determinations.


Is there a general file type associated with AI projects?

No, there is no file type associated with AI projects in general.

Your examples of Photoshop and Excel are specific corporate branded products. These store bespoke data that only works with those products (plus maybe a few converters that can read the files for competitor products).

Even more general examples such as .jpg for images or .txt for text documents are not a good match to AI in general. AI is such a broad field, it is next to impossible to define a standard set of components of an "AI project" in order to build a single file format that could handle the contents of all AI projects.

That said, practical work in AI is likely to include use of specific file formats and extensions, depending on what you do. For instance if you work with Python and TensorFlow you are likely to use .py files for your source code, .ckpt for neural network training checkpoints, .pb for saved models. The last two extensions - ckpt and pb - are only semi-formal naming conventions though.

This lack of a single file extension, and use of maybe a dozen ones that you would be familiar with on a project, is true of most coding and software development work. If you work with an integrated devlopment environment (IDE), then you might have a single "master" file that allows you to load up all the resources in the project to work on them. That is entirely optional though, and may have any of a number of extensions depending on which IDE you use.

As a quick example, the Transformers project on github is a popular resource for natural language processing, a topic often considered as part of AI. A quick look at that project shows:

Various development tool configuration files either starting with . or with various extensions - .cfg, .yaml
.py files for Python source code
Script files with .sh extension or with extension removed for convenience of using on command line
Dcoumentation written with .txt, .md and .rst extensions.
Some .ipynb files used for Jupyter notebooks - a format that bundles documentation, Python scripts and their output for sharing work.


How to make spacy lemmatization process fast?

https://spacy.io/api/lemmatizer just uses lookup tables and the only upstream task it relies on is POS tagging, so it should be relatively fast. For large amounts of text, SpaCy recommends using nlp.pipe, which can work in batches and has built in support for multiprocessing (with the n_process keyword), rather than than simply nlp.
Also, make sure you disable any pipeline elements that you don't plan to use, as they'll just waste processing time. If you're only doing lemmatization, you'll pass disable=["parser", "ner"] to the nlp.pipe call.
Example code that takes all of the above into account is below.
import spacy
nlp = spacy.load("en_core_web_sm")

docs = ["We've been running all day.", "Let's be better."]

for doc in nlp.pipe(docs, batch_size=32, n_process=3, disable=["parser", "ner"]):
    print([tok.lemma_ for tok in doc])

# ['-PRON-', 'have', 'be', 'run', 'all', 'day', '.']
# ['let', '-PRON-', 'be', 'well', '.']
```


What is the purpose of storing the action $a$ within an experience tuple?

We need to store the action $a$ as it tells us the action that we took in the state that we are backing up.
Suppose we are in state $s$ and we take action $a$, then we will receive a reward $r$ and next state $s'$. The goal of RL, and in particular DQN (I mention DQN as it is the first algorithm that comes to mind when I think of a replay buffer but it is of course not the only algorithm to make use of one), is that we are trying to learn optimal state-action value functions $Q(s, a)$. We thus want our value function to be able to predict $y = r + \gamma \max_{a'}Q(s', a')$, i.e. given $s$ and $a$ we want to be able to predict $y$. As you can see, we need to know which action we took in state $s$ so that we can train our value function to approximate $y$, and the value function clearly depends on $a$, hence we need to also store $a$ in our experience tuple.


What does the term "closed expression" mean?

A closed expression refers to a formula which has no free variables [1]. This is also called sentence. In a logic system you have a set of axioms which are sentences and rules which state how to derive a sentence from this [2]. If a sentence can be derived from the axioms, this means that the axioms entail this sentence. If a sentence is not derivable, it is not entailed by the axioms.


Network doesn't converge with ReLU or Leaky ReLU, but works well with sigmoid/tanh

I don't think it is dead ReLU units as a main cause, although they may be happening as part of the NN failing.

The NN architecture is too complex for the given task (too deep, too many neurons) and that means that any problems you have with other design choices will tend to get amplified. It could be that your NN is close to diverging on the given data and architecture, and that sigmoid is more resilient to it.

I'd suggest the following changes:

Dropping the learning rate, try 0.01 or even 0.001
Normalising the two input features. NNs like to work with data that is mean 0, standard deviation 1, although there is some flexibility here, your values ranging from 0 to 6 are probably starting to cause minor problems
Look at standard initialisation routines for weights, available within TensorFlow framework, such as Glorot uniform. Your random -1 to +1 is probably too high a range for the given network, and NNs - especially "deep" NNs with 3+ hidden layers - are very sensitive to how initial weights are set.
Simplify the network architecture a little. Five hidden layers and 200 neurons seems a bit much for your goal of over-fitting to this small data set. Try something like 3 hidden layers and 50 neurons.
Your output layer and loss function are designed for a regression task, but you mention that the goal is to identify classes. You need a softmax layer and multiclass log loss for predicting exclusive classes.


Can we use a messenger that does not alter the AI to solve the control problem?

I would say no due to the possibility of psychological manipulation of the messenger by the AI. Also, the LED communication constraints place severe limitations on the capabilities of the AI, as the usefulness of AI is likely predicated on its ability to learn quickly from a vast amount of information (e.g. using the internet).

In some sense you may successfully control an AI using techniques like this but the nuance of the control problem is controlling an AI without restricting its ability to solve our greatest problems.

We already knew that we could keep an AI safe inside a computer isolated from the rest of the word, but the problem fundamentally is that we if had a truly general AI, we would never want to keep it isolated. Is there some way to unleash it so that it is fully capable of solving our problems while simultaneously making sure that it is safe?


Does this $\max$ mean that we need to maximize the regret in this regret formula?

Yes, you're interpreting the $\max$ there wrongly. In your second formula
$$
\operatorname{Regret}_{T}(\mathcal{H})=\max _{h^{\star} \in \mathcal{H}} \operatorname{Regret}_{T}\left(h^{\star}\right) \label{1}\tag{1}
$$
The sign $=$ means "is defined as", so maybe the following notation is less confusing
$$
\operatorname{Regret}_{T}(\mathcal{H}) \triangleq \max _{h^{\star} \in \mathcal{H}} \operatorname{Regret}_{T}\left(h^{\star}\right)
$$
In fact, in section 2.1 of the same paper, there's a similar but more detailed formula that should clarify the meaning of $\max$
$$
\operatorname{Regret}_{T}(\mathcal{H})=\max _{h \in \mathcal{H}}\left(\sum_{t=1}^{T}\left|p_{t}-y_{t}\right|-\sum_{t=1}^{T}\left|h\left(\mathbf{x}_{t}\right)-y_{t}\right|\right)
$$
Note that $\operatorname{Regret}_{T}(\mathcal{H})$ is defined for $\mathcal{H}$, while your first formula is defined for $x$. So, \ref{1} is the regret for the whole hypotheses class $\mathcal{H}$, which is thus the maximum regret that you can have across all possible hypotheses $h \in \mathcal{H}$. This should make sense.


What kind of word embedding is used in the original transformer?

No, neither Word2Vec nor GloVe is used as Transformers are a newer class of algorithms.  Word2Vec and GloVe are based on static word embeddings while Transformers are based on dynamic word embeddings.
The embeddings are trained from scratch.


Maximum Q value for new state in Q-Learning never exists

Your problem is with how you have defined $s'$.

The next state for an agent is not the state that the agent's action immediately puts the environment into. It is the state when it next takes an action. For some, more passive, environments, these are the same things. But for many environments they are not. For instance, a robot that is navigating a maze may take an action to move forward. The next state does not happen immediately at the point that it starts to take the action (when it would still be in the same position), but at a later time, after the action has been processed by the environment for a while (and the robot is in a new position), and the robot is ready to take another action.

So in your 2-player game example using regular Q learning, the next state $s'$ for P2 is not the state immediately after P2's move, but the state after P1 has also played its move in reaction. From P2's perspective, P1 is part of the environment and the situation is no different to having a stochastic environment.

Once you take this perspective on what $s'$ is, then Q learning will work as normal.

However, you should note that optimal behaviour against a specific opponent - such as a random opponent - is not the same as optimal play in a game. There are other ways to apply Reinforcement Learning ideas to 2-player games. Some of them can use the same approach as above - e.g. train two agents, one for P1 and one for P2, with each treating the other as if it were part of the environment. Others use different ways of reversing the view of the agent so that it can play versus itself more directly - in those cases you can treat each player's immediate output as $s'$, but you need to modify the learning agent. A simple modification to Q learning is to alternate between taking $\text{max}_{a'}$ and $\text{min}_{a'}$ depending on whose turn you are evaluating (and assuming P1's goal is to maximise their score while P2's goal is to minimise P1's score - and by extension maximise their own score in any zero-sum game)


Will parameter sweeping on one split of data followed by cross validation discover the right hyperparameters?

Depending on the number of features you have you might want to try reducing features in order to reduce overfitting and speed up your model. I assume you are using proper regularization as well. PCA may be an option to help if time is the main issue, medium PCA. As the medium article states PCA can be used to reduce the dimensionality of the data while keeping retaining variance. If you run a random forest on the different sets and the feature importance ranking is at all different that is a big issue and the number of features should be reduced. Ideally, the features should be cropped till they have the same (structure/feature importance) but slightly different weights associated with those features. Here is a article demonstrating feature selection.
Here are some publications I believe are relevant to your problem. More in line with your desire to try to validate after one sweep. You may want to look up applications of LSTM or Yolo to your model I have a feeling these technologies will help direct your final decisions on what to do.
Subspace regularization method for the single-trial estimation of evoked potentials (1999) by P.A. Karjalainen, J.P. Kaipio, A.S. Koistinen, M. Vauhkonen
On Quadratic Penalties in Elastic Weight Consolidation (2017) by Ferenc Huszár
Parameter space exploration with Gaussian process trees (2014) by  Robert B. Gramacy   University of California, Santa Cruz, CA
Herbert K. H. Lee   University of California, Santa Cruz, CA
William G. Macready
Optimal Hyperparameters for Deep LSTM-Networks for Sequence Labeling Tasks (2017) by Nils Reimers, Iryna Gurevych
Evaluating Hospital Case Cost Prediction Models Using Azure Machine Learning Studio (2018) by Alexei Botchkarev -evaluation of regression performance


Finding an optimum back propagation algorithm

When you are training a neural network, you use an algorithm called back propagation. This algorithm uses partial derivatives to determine the optimal values for weights. Partial derivatives are a calculus based method which tell you how far you need to adjust the weights in order to get to an optimum value. However, when you have neural networks with many different weights you need to be able to decide which ones to update in order to achieve a more optimal network. This is where stochastic gradient descent comes in. This algorithm is able to take many different gradients and traverse through many dimensional space into a lower error rate. Also, from your diagram of your neural network, I do not see any hidden neurons, which are neurons between the input and output layers which do most of the actual work. While you might be able to train your neural network optimally without hidden neurons, it might improve your accuracy depending upon your dataset. Finally, implementing a back propagation algorithm is a lot of work which is why many people use pre built libraries. Unless you are doing research, I recommend you do the same. I think for a problem like this, pybrain would be perfect, and if you want to use deep neural networks, tensorflow would be good for that.


What are the minimum requirements to call something artificial neural network?

If you pick up a textbook on Neural Networks, you'll find that the simplest examples shown are ones that just implement an AND gate or something.  They're trivial, probably fewer lines of code than what you have there.  The bar to be an "artificial neural network" is pretty low... it certainly isn't the case that ANN's must be incredibly complicated with thousands of lines of code, and many layers, or even many "neurons" total.

Basically, if something is setting up at least one "neuron" with multiple inputs, and using some kind of weighting function to generate an output from those inputs, it's a valid ANN. It might be a really simple example of an ANN, but it's still an ANN.

Remember what Geoffrey Hinton says in his Coursera class - (paraphrased) "We don't pretend that the things we're building really work the way the brain does, we're just taking the brain as loose inspiration for an approach that we've found works".


Are deep learning models suitable for training with sparse data?

The problem isn't the GAN but the implementation of its discriminator which is typically a convolutional neural network (CNN).  CNNs have trouble with sparse data.  They require dense data to learn well.  There are ways to work around this.  See the following for some ideas:

Sparse and Dense Data with CNNs: Depth Completion and Semantic Segmentation
Sparse data in deep learning
SECOND: Sparsely Embedded Convolutional Detection


How does the weight update formula for logistic regression work?

I think i found out how that works, so i made a short article about it .
https://medium.com/@kourloskostas/python-spam-filter-86b21d7d1564
I hope it helps!


How does Google's self-driving car identify pedestrians?

The AI of the car uses sensor data to process all the data and classifies objects based on the size, shape and movement patterns. It can recognize surroundings from a 360 degree perspective by making predictions about vehicles, people and objects around it will move.

It can detect pedestrians, but as moving, column-shaped blurs of pixels, so it really cannot tell whether it's a rock or a crumpled piece of paper.

However it is programmed to determine certain patterns when a police officer has halted traffic or the car is being signaled to move forward.

It also recognizes cyclists as objects outlined in red and can slow down to let the cyclist enter into a lane.

Above images are provided by Chris Urmson who heads up Google's driverless car program.

Sources:

How Google's self-driving cars see the world
Hidden Obstacles for Google’s Self-Driving Cars
(video) Chris Urmson: How a driverless car sees the road


Is there a document with a list of conjectures or research problems regarding reinforcement learning (like the Millennium Prize Problems)?

It's not really an exhaustive list, but Hutter maintains a small list of problems (click on the bullet point "Universal AI Book" here) related to AIXI (a reinforcement learning agent), some of which have already been solved. The money awards are in the range of 50-500 euros, so they are not as financially important as the Millennium Prize Problems.


How to design a fitness function for a problem where there are 2 objectives?

If we assume that each laptop requires 1 component of each type, so 1 screen, 1 hard drive, 1 RAM, etc. (i.e. the company has no preference for the type of component), then the company, to maximize the number of laptops it can build (which is supposedly the ultimate goal of the company), it should

maximize the number of instances of the least available component in the offer, and

minimize the cost of the offer

So, one possible fitness function would then need to take these 2 objectives into account, so this would be a multi-objective optimization problem.
Recall that a fitness function evaluates individuals in the population. If we assume that the individuals are the offers and the goal would be to find the best offer in the space of offers, then we can devise some fitness function $f$ of the form
$$
f(o) = (1 - \alpha)\frac{1}{1 + p(o)} + \alpha \min(o), \label{1}\tag{1}
$$
where

$o$ is an offer, so $f(o)$ is the fitness of the offer $o$, i.e. how much it is desirable (i.e. we want the fitness to be high, usually).
$p(o)$ is the price/cost of the offer $o$
if we assume that $o$ is an array of the form $o = [1000, 800, 2000, \dots]$, where $o_i$ is the number of items of type $i$ (e.g. $i$ is RAM, then $o_i$ would be the number of RAMs in the offer $o$), then $\min(o)$ would be the number associated with the least available item in the offer (e.g. if the indidivudal was $[12, 4, 5]$, then $\min(o) = 4$)
$\alpha \in [0, 1]$ is a hyper-parameter that trades off the two objectives (i.e. prices and the smallest number)

The fitness function $f$ in equation \ref{1} will be maximal when $p(o) = 0$ and $\min(o) = N$, where $N$ is some maximum threshold of possible number of items of the same type that an offer can have.
I have just come up with this fitness function. I don't know whether it will work in practice or not, but this is the idea of what you have to do.  You can design other similar fitness functions (for example, how would you design a fitness function that takes into account that the number of components of each type should be more or less the same?). You may want to try to implement this e.g. with DEAP and see how it behaves. You probably also want to make sure that $o$ are arrays of integers (and not floating-point numbers). You probably also want to read this answer.


This is conditioning in the sense of conditional probability. The idea is that the authors have some "standard physically-inspired features". They are splitting the data up into bins based on the values of these features, and then training a model for each bin. They are then examining the differences between the models. Usually this is done to learn something about the benefits of using the different features, and about the relationships between features and outputs.


Why does (not) the distribution of states depend on the policy parameters that induce it?

The reason you are confused is because this is not the full derivation of the Policy Gradient Theorem. You are correct in thinking that $\mu(s)$ depends on the policy $\pi$ which in turn depends on the policy parameters $\theta$, and so there should be a derivative of $\mu$ wrt $\theta$, however the Policy Gradient Theorem doesn't require you to take this derivative.
In fact, the great thing about the Policy Gradient Theorem is that the final result does not require you to take a derivative of the state distribution with respect to the policy parameters. I would encourage you to read and go through the derivation of the Policy Gradient Theorem from e.g. Sutton and Barto to see why you don't need to take the derivative.

Above is an image of the Policy Gradient Theorem proof from the Sutton and Barto book. If you carefully go through this line by line you will see that you are not required to take a derivative of the state distribution anywhere in the proof.


How is the performance of a model affected by adding a ReLU to fully connected layers?

ReLU is piecewise linear function that outputs the received input directly if it's positive, or outputs a zero. i.e., $max(0, x)$

How significant is adding relu to full connected layers?
ReLU, being an activation function, will determine what the output of the nodes in your FCs are. Since it's a non-linear function, one significance is it will allow the nodes in your model to learn  complex mappings between the inputs and the outputs. Compared to using a linear function, it will allow back-propagation since it has a derivative, allowing the neural network to have the advantage of stacking of multiple FC layers.
Is it necessary?
ReLU (and non-linear activation functions in general) will introduce non-linear properties in a neural network that enables it to learn more complex arbitrary structures in the inputs. Without activation functions between the layers, your neural network will simply be a linear function, regardless of the number of layers it has. Why? A linear function + linear function gives a linear function. Additionally, see this answer.
How is the performance affected by adding ReLU?
Compared to no activation function at all, it will be slower to train but will only behave like a linear regression model. So ReLU will increase the power of the model by making it non-linear.
However, compared to other non-linear activation functions like tanH, ReLU will speed up training as (1) its computation step is cheaper i.e., $0.0$ or $x$ without additional operations (2) its gradient just depends on the sign of the input $x$. See this answer


When exactly am I overfitting -- contradicting metrics

Generally, a model is considered to be overfitted when there's a huge gap between training and test/validation performances.
So during the training, you monitor the loss on validation data, and training data, and stop training if validation loss stagnates/increases given the training loss keeps decreasing.
In your scenario, I'm not sure what the total loss metric corresponds to. As I said, you've to measure loss on a held-out data other than training data to detect, and prevent overfitting.


What are examples of simple problems and applications that can be solved with AI techniques?

This is fairly boilerplate advice, but, since you're brand new to AI, I'd personally suggest writing a classical Tic-Tac-Toe AI, ideally using minimax.

I suggest this because minimax is fundamental to AI, and there are many webpages devoted to this subject, such as How to make your Tic Tac Toe game unbeatable by using the minimax algorithm and Tic Tac Toe: Understanding the Minimax Algorithm.  (Google search for "Tic-tac-toe" and "minimax" will yield a plethora of other sites.  I'd also recommend looking at this minimax page from Stanford: "Strategies and Tactics for Intelligent Search".)

I recommend this approach as a good basic primer.  The real cutting-edge work is being done in Machine Learning and Neural Networks, and for that reason, it's probably more important than ever to have some basic grounding in classical AI before you start dipping your toe in that pond.


How to fill in missing transitions when sampling an MDP transition table?

I assume you use the 12 discrete features as state variables, and, for each of these variables, you will have at least two values. So, the minimum number of states will be $2^{12} = 4096$, which gives $(2^{12})^{2} = 16777216$ possible transitions. In order to reach this, you will need a huge amount of simulations, also taking into account that this number is a minimum since you might have more values per state variable, and you probably have more than one action per state transition.
How to fill the gaps depends on your problem, in a problem where I did this, I filled the gaps using a uniformly random transition to its neighbor states. However, since I had to fill in such a a large amount, there was no significant difference between using this estimated transitions probabilities with a predefined transition table.
In your case, it might be better to use Q-Learning, which is a model-free method that does not require the transition probabilities and uses directly the rewards and states obtained.


Are there any downsides of using a fixed seed for a neural network's weight initialization?

When you use a particular seed, it actually ceases to become a random initialization and is instead fixed. I believe the only reason to actually do this would be for reliable reproduction in research and not as a method of training production models.


Can Convolutional Neural Networks be applied in domains other than image recognition?

Yes, Convolutional Neural Networks(CNNs) can and have been applied to non-image problems. Arguably, any problem in which the location of a feature(s) is relevant can be attempted via CNNs. CNNs works under the assumption that points close to each other in the data share some correlations/relationship whist points further apart don't share as much information. So, theoretically, if you can phrase your problem so that it meets this requirement(s), it can be attempted by a convolutional neural network. Here are few applications of CNNss that are dont involve images:

Speech recognition
Character-level Convolutional Networks for Text Classification
Text classification using CNNs
Natural Language Processing
Time Series Forecasting
Sentence modeling
I've used 1-D CNNs on spectral data. Here are some examples of CNNs applied to spectral data :

Spectral Convolutional Neural Network for music classification
Convolutional neural network based classification for hyperspectral data


Has the Lovelace Test 2.0 been successfully used in an academic setting?

No.

TL;DR: The Lovelace Test 2.0 is very vague, making it ill-suited for  evaluation of intelligence. It is also generally ignored by researchers of Computational Creativity, who already have their own tests to evaluate creativity.

Longer Answer:
According to Google Scholar, there are 10 references to the "Lovelace Test 2.0" paper. All of those references exist merely to point out that the Lovelace Test 2.0 exists. In fact, at least two of articles I consulted (A novel approach for identifying a human-like self-conscious behavior and FraMoTEC: A Framework for Modular Task-Environment Construction for Evaluating Adaptive Control Systems) proposed their own tests instead.

One of the authors who wrote the FraMoTEC paper also wrote his thesis on FraMoTEC, and indirectly critiqued the Lovelace Test 2.0 and other similar such tests:

  The Piaget-MacGyver Room problem [Bringsjord and Licato, 2012], Lovelace Test 2.0 [Riedl, 2014] and Toy Box problem [Johnston, 2010] all come with the caveat of being defined very vaguely — these evaluation methods may be likely to come up with a reasonable evaluation for intelligence, but it is very difficult to compare two different agents (or controllers) that partake in the their own domain-specific evaluations, which is what frequently happens when agents are tailored to pass specific evaluations.

Another major issue with the Lovelace Test 2.0 is that there is a proliferation of other tests to "measure" the creativity of AI. Evaluating Evaluation: Assessing Progress in Computational Creativity Research, published by
Anna Jordanous in 2011 (3 years before the invention of the Lovelace Test 2.0) analyzed research papers about AI creativity and wrote:

  Of the 18 papers that applied creativity evaluation methodologies to evaluate their system’s creativity, no one methodology emerged as standard across the community. Colton’s creative tripod framework (Colton 2008) was used most often (6 uses), with 4 papers using Ritchie’s empirical criteria (Ritchie 2007).

That leaves 10 papers with miscellaneous creativity evaluation methods.

The goal of "Evaluating Evaluation" was to standardize the process of evaluating creativity, to avoid the possibility of the field stagnating due to the proliferation of so many creativity tests.  Anna Jordanous still remained interested in evaluating creativity tests, publishing articles such as "Stepping Back to Progress Forwards: Setting Standards for Meta-Evaluation of Computational Creativity" and Four PPPPerspectives on Computational Creativity.

"Evaluating Evaluation" does provide some commentary to explain the proliferation of systems to evaluate creativity:

  Evaluation standards are not easy to define. It is difficult to evaluate creativity and even more difficult to describe how we evaluate creativity, in human creativity as well as in computational creativity. In fact, even the very definition of creativity is problematic (Plucker, Beghetto, and Dow 2004). It is hard to identify what ’being creative’ entails, so there are no benchmarks or ground truths to measure against.

The fact that so many tests of creativity already exist (to the extent that Jordanous can make an academic career in studying them) means that it's very difficult for any new test (such as the Lovelace Test 2.0) to even be noticed (much less cited). Why would you want to use something like the Lovelace Test 2.0 when there's so many other tests you could use instead?


If I am interested in theoretical computer science, is AI a bad choice?

There are certainly results in theoretical computer science / pure math with deep implications for AI. But to my knowledge these results typically aren't labeled as results of artificial intelligence, but as something more congruent in that particular field (For example in CS, we might say "agent with unbounded computational power"; in math might say some statement is "decidable/undecidable" with respect to some system). Of course they still matter in the field of AI, but you need to know what you are looking for.

See my question What are some implications of Gödel's theorems on AI research? for some examples. Or you can look up MIRI's research guide for a better idea of what existing work is out there that links formal math / CS to AI research.

Another point to raise is that there is no good definition of AI in fields outside of normal discourse (or even within, perhaps), so its difficult to decide what discussions pertains to the study of AI. Questions like whether ZFC with/without choice is expressive enough might not be on the mind of most AI researchers, but could still have some implications.

So to answer you question more directly, there is certainly a field of study regarding theoretical AI. Regarding whether or not its a good choice is something for you to decide, but it is (in my humble and not-very-well-educated opinion) very difficult field that isn't very popular, and has not seen major progress in many years.


What is the meaning of these equations in Noise2Noise paper?

The equation you are referring to is called Mean Squared Error (or $L_2$ loss) and it is used for regression tasks, where the goal is to predict a real value given some input.
In your case, the inputs are measurements of temperature $y$, either at a certain point in time or point in space or both or none, this is not clear from the image. Now, the goal would be to predict the temperature at a new point in space, time, or both, where we don't have access to a measurement. That is we would like to find a function $f$ (e.g. a simple linear function) which we can use for prediction. But how can we measure which function is "best"? We introduce a loss function $L(f,y)$, another function which tells us how good our proposed function is.
Visually it looks like this (image source):

Red crossed are measurements, the black line is our function we use for prediction and the green dotted lines are the errors (the distance from our prediction to the real measurement). In this example salary depends on experience.
Now, the paper introduces the constant mean of all measurements as $y$, $z = \mathbb{E}_y\{y\} = \frac{1}{N}\sum_i^N y_i$, as our function $f$, which is known to be the minimizer for the $L_2$ loss in the case where there is no dependence on other variables (e.g. time or space).


What are the differences between an agent that thinks rationally and an agent that acts rationally?

Maybe a good example to think about would be something like the Sphyx story. The wasp in the story appears to behave like a rational being: it seems to have a plan of action, it seems to be able to do advanced operations like counting, and it seems to execute the plan well. However, if you disrupt the wasp's plan, it becomes apparent that it is not thinking rationally. Instead, it has evolved a very complex behaviour that appears to include rational components, but is still just an instinct.
In the context of AI, consider a GOFAI system for planning like GraphPlan as opposed to a machine learning system for generating plans. While the former is a general-purpose algorithm for reasoning about planning problems, the latter is an input/output mapping that may mimic reasoning, or may be more "instinctual". Some AI'ers would say that the latter system is not really engaged in rational thought, while the former is. Both systems exhibit rational action, however.
Instinctive systems often work very well, and I'd say that the increasing effectiveness of machine learning approaches since that edition of R&N was published (almost 10 years ago now?) makes this a fuzzier distinction in practice than the book might suggest.


Has Google's view of AI risen to the status of universal truth?

No, "Google's view of AI" (which in and of itself already is kind of vague, there are plenty of people within Google who have very different ideas) has not risen to the status of universal truth.

  It sometimes seems that, when people don't understand something in a paper with the word Google anywhere in the authorship block, it is automatically assumed that the reader is wrong because the paper is of universal authority. They seem to jump to the assumption that their mind must adjust to whatever the paper states.

It does not seem like that to me at all.

Yes, there has been research (from Google actually) indicating that there is some bias in paper review for example. When reviewers can see the author names and their affiliations, there is some measurable bias towards accepting them more often when they are from famous / well-known than when they are not famous. That includes Google, but also other companies (like Facebook, OpenAI) and universities and simply individuals who happen to be well-known even if their affiliations aren't. The same thing probably also happens occasionally even when double-blind reviewing is used, because... well, it's not difficult to guess where the anonymous authors may be coming from when the experiments described in their paper use a huge collection of TPUs.

It's nothing more than that though, just a bias. They get their papers accepted (and probably also cited) a bit more often. They definitely do not always get their papers accepted. Plenty of papers from Google have been rejected too (which means the reviewers did not simply accept whatever was written without critical thinking), and that will continue to happen too. It also happens more often than not that I see/hear others being critical or confused by things they write. There is still plenty of critical thinking going around.

Of course, the fact of the matter still is that they have managed to attract many world-class researchers (who were also already known as such before joining Google). It is no surprise that such researchers do have many good ideas, do write solid papers, and with Google they also do have the resources (in terms of computation, finances, development support from highly skilled software engineers, etc.) to execute more ideas, execute them more quickly, and more thoroughly. This naturally gives them an important voice, but they are not immune to criticism.


Is there a methodology for splitting up annotated orthophotos into smaller photos that retain the original bounding boxes?

You can reduce your photo size and scale the corresponding boxes to the new dimensions (416x416).
Or if you want to go with your technique, you can slice the image and then, check if the bounding box lies in the slice, then, reorient it according to the slice you took.
Take a look at albumentations library for this.


Unix timestamps for Recurrent Neural Networks

I would say offsets from previous N-event is necessary in this case. You can also encode their max and avg as other features in order to compress the irregular sampling times into a vector.


What is the difference between environment states and agent states in terms of Markov property?

$\mathcal S$ is just a set of all possible states. It doesn't matter if it's agents perceived state or true environment state, they are within the same set of states. Agent cannot perceive itself to be in some "middle" state that's not in $\mathcal S$, it might think that's in the state that's not the actual environment state but that state is also in set of all states.
To give an example, if the car can be blue or red, then the agent might think that the state is blue car or red car, but it cannot think that car is purple because that's not one of the possible states. It might wrongly think that the car is blue when the actual car is red, but that's ok because blue is one of the possible car states. Of course, it might also correctly think that the car state is red.


Why does the DQN not converge when the start or goal states can change dynamically?

Your problem is not that the environment is stochastic or dynamic. In fact you are using the terms slightly incorrectly. These terms do not usually refer to the fact that starting state can differ or goal locations can move episode-by-episode. They typically refer to behaviour of state transitions.

Although in your case you could view the initial state as stochastic, this is not a big deal, and not likely to be the cause of your problems.

From your questions, it seems to me that you are not really running a DQN algorithm yet. It is not 100% clear what your neural network is predicting, but my best guess is that you have 4 outputs to select "best" action and are treating the neural network as a classifier. This training approach seems closest to Cross Entropy Method (CEM) due to how you are selecting "successful" navigation only.

  When training the DQN, how do i apply Q-values?

This question is the most revealing that you are not using DQN. This is too complex to describe in full in an answer, but the basics are:

Your neural network (NN) should be estimating Q values. Typically in DQN, you input the state and the NN outputs an array of estimates for Q of each action (although other architectures are possible). This should be a regression problem, so last layer of network needs to be linear.
Current best guess of optimal policy is to run the NN forward and find the maximising action.
In DQN you also have a "behaviour policy" - a simple and popular choice is to use $\epsilon$-greedy action selection, which just means to take the maximising action (as calculated above), except with probability $\epsilon$ (some small value, e.g. 0.1) to take a random action.
To figure out your training data to improve the NN, you need Q values to calculate a TD Target. In single-step Q learning that would be $r + \gamma Q(s',a*)$ where $r$ is the immediate reward $s'$ is the next state seen, and $a*$ is the maximising action in that state. You should force $Q(s', a*) = 0$ (i.e. not use the NN) if $s'$ is a terminal state.

This means you typically need to work with Q values in 2 or 3 places in your inner loop. Your inner loop should look something like this per time step, given a current state current_state:

# Figure out how to act
current_q_values = NN_predict(current_state)
current_action = argmax(current_q_values)
if random() < epsilon:
  current_action = random_action()

# Take an action
reward, next_state, done = call_environment(current_state, current_action)

# Remember what happened
store_in_replay_memory(current_state, current_action, reward, next_state, done)

# Train the NN from memory
repeat N times: # This can be vectorised for efficiency
  mem_state, mem_action, mem_reward, mem_next_state, mem_done = sample_replay_memory()
  mem_q_values = NN_predict(mem_next_state)
  mem_max_action = argmax(mem_q_values)
  if done:
    td_target = mem_reward
  else
    td_target = mem_reward + gamma * mem_q_values[mem_max_action]
  target_q_values = NN_predict(mem_state)
  target_q_values[mem_action] = td_target
  NN_train(mem_state, target_q_values)

# Maybe end an episode (this can include generating new map)
if done:
  current_state = reset_environment()
else:
  current_state = next_state

You can see above, NN_predict is called three different times to get Q values in slightly different contexts. I have ignored extras such as using a separate target network.

  I train the network using only a short replay memory of the last move, or the last N moves that led to success. Is this the right way to approach this?

It is important to include moves that lead to failure so that the NN learns the difference. Typically you will need a replay memory with from a few hundred to a few hundred thousand entries. You would get away with a few hundred perhaps for your simple problem. The idea is to use this training data a bit like a dataset from supervised learning.

  I use Keras, and am simply training the network every time it does something right - and ignoring failed attempts. - But is this anywhere near the right approach?

This is not the right approach for DQN, although perhaps could be considered a crude version of CEM.


Why does a Lipschitz continuous discriminator in GANs assure statistical boundedness?

To put it simply GANs suffer from a problem of uneven learning rate. Imagine the learning rate of a pitcher and hitter if the pitcher gets to a point where they can throw much better than the hitter can hit then the hitter may fall into a 'training pit' as to be unable to ever learn how to hit from the pitcher.

This follows a continues relationship in between the two learning rates where if the pitcher is becoming a much better pitcher at a faster rate they could become too good and make learning impossible for the hitter. So the rate must be 'slowed down' as to ensure the pitcher doesn't ruin the hitter.

If the cone of the Lipschitz continuity function of either function is outrunning/outpacing the other than the learning for the one who is in front must be slowed down so the other catches up.

Two runners trying to push each other athletically is another example. If one outpaces the other an injury may occur in the one lagging behind while trying to keep pace this happens, when the adversarial network becomes too good at generating training material that the behind network is not ready to learn with.

The GAN will do best when the learning rates are adjusted to slow down the fast learner artificially.

The statistics will not be bounded correctly if the learning rates are not kept in check similarly to how step size needs to be right to find local minimum and maximum. If the learning is not artificial augmented so both keep relatively same pace getting stuck at local minimum and maximum of the solution space will occur.


What is the difference between artificial intelligence and artificial neural networks?

Artificial intelligence is a broad field, ANN is a specific technique with in that field.


Equivalence of the $Q(s,a)$ and $V(s)$ under optimality conditions?

In general they are not the same and that should be clear as to why -- mathematically you are conditioning on an extra random variable being known in the state-action value function. You have the correct relationship between them, but I think your understanding of the two may be slightly off. The state-action value function is a function of both $s$ and $a$ whereas the value function is a function of just $s$. As you have noted, the value function is equal to the expected value of the state-action value function, with the expectation taken over the action distribution induced by the policy.
However, if you have a deterministic policy, then two are equivalent only if you evaluate the state-action value function at the action that the deterministic policy gives. That is, $v_\pi(s) = Q(s, \pi(s))$. This is because, as you say, for a given state the policy will assign probability one to a certain action and 0 to all other actions in the action space and so the equation I just wrote is what the expectation in your question reduces to.
Now, you might think 'well the policy is deterministic so we are always going to choose this action, so my theory was correct'. This is not true. The definition of the state-action value function is the value of the expected future (possibly discounted) returns given that we are in state $s$ and given that we have taken action $a$. This means that we have already chosen our action and we are not choosing it according to the policy -- the policy will only choose all future actions.
You also say that "If we are operating a greedy policy and hence acting optimally" which is not true. Just because we act greedily it does not imply we are acting optimally. Value functions are functions of a policy. If the policy is completely random then you would get certain value and state-action value functions which you could act greedily to but this would not be optimal.


Reinforcement Learning to Grouped Scheduling Optimisation Problem

It is often possible, by describing certain parts of a problem as state, others as actions, and some measurement you want to optimise as a reward, to frame a problem as a reinforcement learning task. However, this is not always helpful.

I suspect your problem is in the "RL is not that helpful" category. You could give it a try, and maybe expect to find working solutions but take an unreasonable amount of time to find optimal solutions.

What you appear to have here is a problem in combinatorial optimisation. There are many applicable algorithms, a good selection are described in the free book Clever Algorithms: Nature-Inspired Programming Recipes - which is best will be determined by the specific details of your constraints and the size of the problem.

If many constraints are soft and can be assigned a cost, you may be able to use genetic algorithms (with genome being group assignments in order) or simulated annealing for example.


How to deal with different actions for different states of the environment?

There are so many reinforcement learning algorithms. I recommend you this articles:

Introduction to Various Reinforcement Learning Algorithms. Part
1
Introduction to Various Reinforcement Learning Algorithms.    Part
2

There are several approaches. A NN can get state and action as an input  and expected discounted reward as an output. But you can just not pick an action if this action isn't available and there isn't necessity to give it negative or zero reward just don't learn your net to change it behavior for these actions because they are neither good nor bad.


Number of parameters in Keras/Tensorflow Dense layers

Check the documentation for Dense layer:

Note: If the input to the layer has a rank greater than 2, then Dense computes the dot product between the inputs and the kernel along the last axis of the inputs and axis 1 of the kernel (using tf.tensordot). For example, if input has dimensions (batch_size, d0, d1), then we create a kernel with shape (d1, units), and the kernel operates along axis 2 of the input, on every sub-tensor of shape (1, 1, d1) (there are batch_size * d0 such sub-tensors). The output in this case will have shape (batch_size, d0, units).

That is what happening in your first case - for input dimensions (4,1) you've got d0=4 and d1=1. So it creates a kernel of shape (1,32) that gets applied along the axis of dimension 4. That's why your output shape is (4,32) and you've got 32 weights + 32 biases = 64 parameters.
In second case you've got a "standard" 32 * 4 fully-connected weight matrix + 32 biases = 160.


Why is it not advisable to have a 100 percent exploration rate?

No - imagine if you were playing an Atari game and took completely random actions. Your games would not last very long and you would never get to experience all of the state space because the game would end too soon. This is why you need to combine exploration and exploitation to fully explore the state space.


You have a simple mistake in your TD Error function:

TD_error = reward + gamma * max(Q[next_state]) - Q[state][action]

You have made Q[next_state] a Python dict, so this will take the maximum key which is 3 for all your Q table entries. This is why you end up with values very close to 3 at the end, which is impossible for your problem, the maximum return will be 1.0 when stepping from adjacent grid points to the positive terminal state.

The correct code is:

TD_error = reward + gamma * max(Q[next_state].values()) - Q[state][action]

Alternatively, since your actions are all numeric, you could use a list instead of a dict in your Q table.


Can I categorise the user input which I get as free text?

Have you tried NLTK, what you are looking for is in Chapter 6 of the book. Basically what you need to do is:

Tokenize the user input.
Extract vector set from the tokenized words.
Train your model with some given texts, and same vector sets.

And you can use your model to categorize the document.

One other suggestion, instead of extracting vector sets you can use every word in the input
to be evaluated in to some category using a training set of large corpus, which you are sure
it contains all the words.
And then you multiply probability of each word being on a category to decide where the document belongs.


What is the meaning of "stationarity of statistics" and "locality of pixel dependencies"?

locality of pixel dependencies probably means that neighboring pixels tend to be correlated, while faraway pixels are usually not correlated. This assumption is usually made in several image processing techniques (e.g. filters). Of course, the size and the shape of the neighborhood could vary, depending on the region of the image (or whatever), but, in practice, it is usually chosen to be fixed and rectangular (or squared).

stationarity of statistics might mean that the values of the pixels do not change over time, so this could be related to diffusion techniques in image processing. stationarity of statistics might also mean that the values of the pixels do not change much in a spatial neighborhood, even though, stationarity, e.g. in reinforcement learning, usually means that something does not change over time (so, if that's the case, the terminology stationarity is at least misleading and confusing, in this context), so this might be related to the locality of pixel dependencies property. Possibly, stationarity of statistics could also indirectly mean that you can use the same filter to detect the same feature in different regions of the image.

With while their theoretically-best performance is likely to be only slightly worse the authors probably thought that, theoretically, CNNs are not as powerful as feedforward neural networks. However, both CNNs and FFNNs are universal function approximators (but, at the time, nobody probably had yet investigated seriously the theoretical powerfulness of CNNs).