How can I reduce the GPU memory usage with large images?

As the other links suggest, you basically have four options:

"restrict your CNN". This means making your model smaller and simpler, possibly by inserting a pooling layer at the front, or reducing the total number of layers. From a memory perspective, this isn't likely to produce really large gains though.
"stream your data in each epoch". By default, the entire training set will be stored on the GPU. This is a good idea, because the bus connecting the GPU and the RAM has extremely high latency. It takes a very long time to start sending data across the bus. However, most systems have much more space in the RAM than in the video memory on a GPU. Instead of storing all the training data in the GPU, you could store it in main memory, and then manually move over just the batch of data you want to use for a given update. After computing the update, you could free the memory assigned to the batch. I am not sure how to do this in Keras. In the past, I have done this by writing a custom CUDA kernel.
"Use less data". If you train on a random subset of the training data, you can keep your images at high quality, but your model will probably overfit. If you train on downsampled images, your model may not be able to discriminate between them well. However, both of these options are easy to do.
"Get more memory". If you buy a video card with more video RAM, you can train more complex models. This is why scientific-grade cards have much larger memories than gaming cards. The Tesla V100 has 32GB of memory, and 16GB cards are common, whereas even the most advanced cards for gaming have only 11GB.


Training Keras Towards Or Against Analog Value?

What is the technical name for this kind of training?

The name for the problem is Sequential Decision Making or Optimal Control.
There are a few different approaches you can take when solving this kind of problem. However, I think that the way that you are describing your project, Reinforcement Learning (RL) would match your approach the best.

I cannot use a traditional Xy method but need another method that goes like this:
network.train_against_value(X, y, determinator)

Although this method signature could probably be made to work by re-structuring one or other RL frameworks, it is more usual when using RL with neural networks to treat the experience-gathering and scoring systems - the core ideas behind RL - as a data generator for either a supervised learning problem or in some cases directly producing the output layer gradients.
One approach you could use is called Deep Q Networks (DQN) which effectively generates mini-batches for supervised learning of a neural network, based on gathering experiences.
In brief, DQN trains a neural network to predict what you are calling the "determinator", but that RL would call the "value" of each action. So you may move the action choice (y) to the input of the neural network, or alternatively predict multiple values (one output for each possble action) - the RL theory is the same for each approach here, it is an implementation detail. The agent (cat or mouse) would then pick the action with the highest predicted value by default (with the cat's values being the negative of the mouse's), trying other actions at random whilst training so as to learn all values. Whenever the agent gained new experience, it would add that to the training data so it could improve its predictions.
RL is a complex subject in its own right, and to begin understanding it properly you would do well to start by tackling even simpler problems that don't require neural networks or running two agents against each other. There is a good introductory book on the subject with the option to read a free PDF version: Reinforcement Learning: An Introduction (Sutton & Barto)


If the point of the ResNet skip connection is to let the main path learn the residual relative to identity, why are there convolutional skips?

Well, I found an answer that satisfies me.

The zero-padded identity is not ideal. Suppose we're mapping from 64 channels to 128 channels. Then the zero-padded identity will map to an output where half of the channels are the same as the inputs, and the other half are all zeros. So that means the main path is learning a residual for half of the output channels, and learning a mapping from the ground up for the other half of the channels.

Now, the alternative is to use option (B) which is the single 1x1 convolution. After reading their ResNetV2 paper I realised that they really double down on this concept of maintaining a "clean" alternative path all the way up and down the network. So then the question is, what's "cleaner"?: a block of two convolutions of the form 3x3x64 then 3x3x128, or a shortcut with a 1x1x128 convolution? The shortcut is a tempting choice here. And in fact, in the original paper they show empirically that the convolutional shortcut is better than the identity shortcut. Maybe it might be worth running a test to see if the convolutional shortcut is better than no shortcut at all, but until I decide to run such a test, I'll presume the authors would have mentioned it. The other thing worth noting is that there are only a few places in the whole network where the dimensions need to be increased, so maybe the impact is small.


How much of Deep Mind's work is actually reproducible?

On the suggestion of the O.P. rcpinto I converted a comment about seeing "around a half-dozen papers that follow up on Graves et al.'s work which have produced results of the  caliber" and will provide a few links. Keep in mind that this only answers the part of the question pertaining to NTMs, not Google DeepMind itself, plus I'm still learning the ropes in machine learning, so some of the material in these papers is over my head; I did manage to grasp much of the material in Graves et al.'s original paper{1] though and am close to having homegrown NTM code to test. I also at least skimmed the following papers over the last few months; they do not replicate the NTM study in a strict scientific manner, but many of their experimental results do tend to support the original at least tangentially:

• In this paper on a variant version of NTM addressing, Gulcehere, et al. do not try to precisely replicate Graves et al.'s tests, but like the DeepMind team, it does demonstrate markedly better results for the original NTM and several variants over an ordinary recurrent LSTM. They use 10,000 training samples of a Facebook Q&A dataset, rather than the N-grams Graves et al. operated on in their paper, so it's not replication in the strictest sense. They did however manage to get a version of the original NTM and several variants up and running, plus recorded the same magnitude of performance improvement.2

• Unlike the original NTM, this study tested a version of reinforcement learning which was not differentiable; that may be why they were unable to solve several of the programming-like tasts, like Repeat-Copy, unless the controller wasn't confined to moving forwards. Their results were nevertheless good enough to lend support to the idea of NTMs. A more recent revision of their paper is apparently available, which I have yet to read, so perhaps some of their variant's problems have been solved.3

• Instead of testing the original flavor of NTM against ordinary neural nets like LSTMs, this paper pitted it against several more advanced NTM memory structures. They got good results on the same type of programming-like tasks that Graves et al. tested, but I don't think they were using the same dataset (it's hard to tell from the way their study is written just what datasets they were operating on).4

• On p. 8 of this study, an NTM clearly outperforms several LSTM, feed-forward and nearest-neighbor based schemes on an Omniglot character recognition dataset. An alternative approach to external memory cooked up by the authors clearly beats it, but it still obviously performs well. The authors seem to belong to a rival team at Google, so that might be an issue when assessing replicability.5

• On p. 2 these authors reported getting better generalization on "very large sequences" in a test of copy tasks, using a much smaller NTM network they evolved with the genetic NEAT algorithm, which dynamically grows topologies.6

NTMs are fairly new so there hasn't been much time to stringently replicate the original research yet, I suppose. The handful of papers I skimmed over the summer, however, seem to lend support to their experimental results; I have yet to see any that report anything but excellent performance. Of course I have an availability bias, since I only read the pdfs I could easily find in a careless Internet search. From that small sample it seems that most of the follow-up research has been focused on extending the concept, not replication, which would explain the lack of replicability data. I hope that helps.

1 Graves, Alex; Wayne, Greg and Danihelka, Ivo, 2014, "Neural Turing Machines," published Dec. 10, 2014.

2 Gulcehre, Caglar; Chandar, Sarath; Choy, Kyunghyun and Bengio, Yoshua, 2016, "Dynamic Neural Turing machine with Soft and Hard Addressing Schemes," published June 30, 2016.

3 Zaremba, Wojciech and Sutskever, Ilya, 2015, "Reinforcement Learning Neural Turing Machines," published May 4, 2015.

4 Zhang; Wei; Yu, Yang and Zhou, Bowen, 2015, "Structured Memory for Neural Turing Machines," published Oct. 25, 2015.

5 Santoro, Adam; Bartunov, Sergey; Botvinick, Matthew; Wierstra, Daan and Lillicrap, Timothy, 2016, "One-Shot Learning with Memory-Augmented Neural Networks," published May 19, 2016.

6 Boll Greve, Rasmus; Jacobsen, Emil Juul and Sebastian Risi, date unknown, "Evolving Neural Turing Machines." No publisher listed

All except (perhaps) Boll Greve et al. were published at the Cornell Univeristy Library arXiv.org Repository: Ithaca, New York.


Why is it harder to achieve good results using neural network based algorithms for multi step time series forecasting?

ANNs & RNNs can be used to create some great models in many different domains, including time-series forecasting. However, across all of these domains, they suffer from the problem of hyper-parameter optimization. Because neural networks are so flexible, it is not clear, at the outset, which arrangement of neurons will be most effective to solve a given problem. It is also not clear how fast the network should learn from new signals, what sorts of activation functions to use in the different layers of the network, and which of several possible regularization methods might be best. Making these decisions well requires either years of practice and experience, or a lot of trial and error (or, maybe both!).

In contrast, a regression-based method like ARMA will typically have just a couple of simple hyperparameters, each of which has a clear, intuitive, meaning. This means that an untrained practitioner can probably get an ARMA result that is close to the result of a trained practitioner using ARMA.

Essentially: neural networks are brittle and sensitive to the choice of hyper-parameters, while regression generally is not.


Neural Network that Predicts Game State Based on Actions

You'll probably want to start out with "Action-Conditional Video Prediction using Deep Networks in Atari Games" (arXiv link: https://arxiv.org/abs/1507.08750). That's from 2015 though, I'm sure there have been lots of other interesting developments since then. This paper may still be a good starting point though, and provide you with the correct terminology to plug into google / google scholar to find more recent papers that build on top of this. Google scholar also provides functionality to automatically find papers that cite this one (interesting recent papers will probably cite this one).

As an additional point, you may want to reconsider your desired output. Given a current state-image and action, it may be easier to train a network to predict only the change in image (i.e., predicting NEW_IMAGE - OLD_IMAGE), rather than predicting the full image. You can then always still manually reconstruct the predicted new image simply by adding that output to the old image again. I'm quite sure I've seen this being done in a more recent paper too, but don't remember exactly the title / authors.


Video engagement analysis with deep learning

One of the key terms in the literature that you are looking for is video captioning.

You can have a look at some of the relevant papers with code on this subject. In short, it is an active area of research and a difficult problem, one reason is because videos are still difficult to learn about (because of larger amount of data + larger model, etc...) and this model has to be working with two modalities of data: text and image.

A paper that you might want to start with is Deep Visual-Semantic Alignments for Generating Image Descriptions which works on single images. In short, you can use something similar like in the paper: object detector (e.g. Faster RCNN) to extract visual features and feed them into the state of an RNN (LSTM) which would output a sequence of words in your summary (see picture below).


What does the parameter $y$ stand for in function $g(y,\mu,\sigma)$ related to REINFORCE algorithm?

If you take a look at the Wikipedia page related to the normal distribution, you will see the definition of the Gaussian density
$$
{\displaystyle f(x)={\frac {1}{\sigma {\sqrt {2\pi }}}}e^{-{\frac {1}{2}}\left({\frac {x-\mu }{\sigma }}\right)^{2}}} \label{1}\tag{1}
$$
and you will see that the $y$ in your formula corresponds to the $x$ in equation \ref{1}.
I've seen this notation in the context of computer vision and image processing, where the Gaussian kernel is used to blur images.
So, as pointed out by someone in a comment, $y$ should indeed be the point where you evaluate the density.
Maybe the confusing part is that all parameters are treated equally in terms of their purpose, while $\mu$ and $\sigma$ are clearly the parameters that define the specific density, so they are not the inputs to the specific density.
After having read the relevant section of the paper, I now understand why you're confused. The author refers to $y$ as the output (not yet sure why: maybe it's the output of another unit that feeds this Gaussian unit?), but I think that this explanation still applies. The output of the Gaussian density $g$ is not $y$, but the density that corresponds to $y$. In fact, in appendix $B$ of the paper, the author says that $Y$ is the support of $g$ and $y$ is an element of $Y$.


Is it common to have extreme policy's probabilities?

Your policy gradient algorithms appear to be working as intended. All standard MDPs have one or more deterministic optimal solutions, and those are the policies that solvers will converge to. Making any of these policies more random will often reduce their effectiveness, making them sub-optimal. So once consistently good actions are discovered, the learning process will reduce exploration naturally as a consequence of the gradients, much like a softmax classifier with a clean dataset.
There are some situations where a stochastic policy can be optimal, and you could check your implementations can find those:

A partially observable MDP (POMDP) where one or more key states requiring different optimal actions are indistinguishable to the agent. For example, the state could be available exits in a corridor trying to get the end in a small maze, where one location secretly (i.e. without the agent having any info in the state representation that the location is different) reverses all directions, so that progressing along it is not possible for a deterministic agent, but a random agent would eventually get through.

In opposing guessing games where a Nash equilibrium occurs for specific random policies. For example scissor, paper, stone game where the optimal policy in self-play should be to choose each option randomly with 1/3 chance.

The first example is probably easiest to set up a toy environment to show that your implementations can find stochastic solutions when needed. A concrete example of that kind of environment is in Sutton & Barto: Reinforcement Learning, An Introduction chapter 13, example 13.1 on page 323.
Setting up opposing agents in self-play is harder, but if you can get it to work and discover the Nash equilibrium point for the policies, it would be further proof that you have got something right.


What are the technological challenges that AI faces today?

Computational Creativity is not an unassailable challenge (depending on who you talk to;) Philosophers have claimed algorithms can't be creative, but Marcel Duchamp, one of the most significant artists in modernity, famously stated that:

  "All artists are not chess players, but all chess players are artists"

This would seem to have been validated by commentators referring to move 37 in game 2 of  Lee Sedol's match with AlphaGo as "beautiful"  (In the game of Go, aesthetics are considered in regard to strategy, not just outcomes.)  The takeaway is it was a choice humans would never have considered, because the structure of our brains is different, and thus our approach to creativity different, than automata.

Current algorithmic creativity is a function of monte carlo methods, which utilize randomness, Monte Carlo Tree Search as a major method. MCTS has great utility in intractable models such as non-trivial combinatorial games, which produce complexity akin to nature, so it's not surprising it's slowing extending to real world applications.

The main issue with computational creativity is it is still not as efficient as human creativity, requiring a great deal of processing power for non-trivial problems.  (This was why it took so long for a computer to beat the best human a Chess--human insight seems rooted in semantics/understanding.)

Procedural content generation is an area of research that is steadily progressing, and includes games, music and visual art.

Algorithmic Bias is the most pressing issue facing the field of AI and Machine Learning methods specifically, which are statistical.  (If the dataset is incorrect, imcomplete or biases, the output will be biased.)


Neural Network for Optical Mark Recognition?

From what I understand, don't bother with a CNN, you have essentially perfectly structured images.

You can hand code detectors to measure how much filled in a circle is.

Basically do template alignment and then search over the circles.

Ex a simple detector would measure the average blackness of the circle which you could then threshold.


How can I classify policy gradient methods in RL?

DP, Monte Carlo, and TD are methods of estimating returns. Policy gradient describes methods of learning a policy. So policy gradients serve a different purpose than the other things you mentioned. For clarity, you can use Monte Carlo or TD methods to estimate returns to construct the loss that you get your policy gradient from.


Would you categorize policy iteration as an actor-critic reinforcement learning approach?

Keeping this taxonomy intact for model-based Dynamic programming algorithms, I would argue that value iteration is a Actor only approach, and policy iteration is a Actor-Critic approach. However, not many people discuss the term Actor-Critic when referring to Policy Iteration. How come?

Both policy iteration and value iteration are value-based approaches. The policy in policy iteration is either arbitrary or derived from a value table. It is not modelled separately.

To count as an Actor, the policy function needs to modelled directly as a parametric function of the state, not indirectly via a value assessment. You cannot use policy gradient methods to adjust an Actor's policy function unless it is possible to derive the gradient of the policy function with respect to parameters that control the relationship bewteen state and action. An Actor policy might be noted as $\pi(a|s,\theta)$ and the parameters $\theta$ are what make it possible to learn improvements.

Policy iteration often generates an explicit policy, from the current value estimates. This is not a representation that can be directly manipulated, instead it is a consequence of measuring values, and there are no parameters that can be learned. Therefore the policy seen in policy iteration cannot be used as an actor in Actor-Critic or related methods.

Another way to state this is that the policy and value functions in DP are not separate enough to be considered as an actor/critic pair. Instead they are both views of the same measurement, with the value function being closer to raw measurements and policy being a mapping of the value function to policy space.

  Also, I am not familiar with any model-based/dynamic programming like actor only approaches? Do these exist? If not, what prevents this from happening?

The main difference between model-based dynamic programming and model-free methods like Q-learning, or SARSA, is that the dynamic programming methods directly use the full distribution model (which can be expressed as $p(r, s'|s,a)$) to calculate expected bootstrapped returns.

There is nothing in principle stopping you substituting expected returns calculated in this way into REINFORCE or Actor-Critic methods. However, it may be computationally hard to do so - these methods are often chosen when action space is large for instance.

Basic REINFORCE using model-based expectations would be especially hard as you need an expected value calculated over all possible trajectories from each starting state - if you are going to expand the tree of all possible results to that degree, then a simple tree search algorithm would perform better, and the algorithm then resolves to a one-off planning exhaustive tree search.

Actor-Critic using dynamic programming methods for the Critic should be viable, and I expect you could find examples of it being done in some situations. It may work well for some card or board games, if the combined action space and state space is not too large - it would behave a little like using Expected SARSA for the Critic component, except also run expectations over the state transition dynamics (whilst Expected SARSA only runs expectations over policy). You could vary the depth of this too, getting better estimates theoretically at the expense of extra computation (potentially a lot of extra computation if there is a large branching factor)


Random value generator using a single neuron or DNN

AI is supposed to do anything human or traditional computer can do, that is what we expect AI to be.

Technically you would need AGI (Artifical General Intelligence) to do anything a human can do. This is not a technology that exists, but a goal of some AI research to perform more and more general tasks.

  So 'generating random value' is also a task included in the scope that AI should be able to do

Humans are actually very bad at generating random values directly. In fact it is possible to identify a person by getting them to generate random numbers from their mind. Indirectly, a human could pick up a die, roll it, and read the number - this is also something that is within capabilities of modern AI, but not something you could implement with a small neural network.

Computer software cannot generate random numbers directly, only pseudo-random numbers that are deterministic but follow a statistical pattern very similar to theoretical randomness. When combined with frequently changing seed data from the environment, this becomes very close to an ideal random source, but it does require some hardware input. Modern computer chips include an internal source of randomness to provide this seed data, but it is important to note that it is a piece of specialised hardware, not something that can be coded or simulated using a simple neural network.

  I'm trying to generate random value using a single neuron but the outcome isn't much good. Any suggestions?

This is going to be much harder than you assumed. Random number generators are always more complex than a single function, and require at least a little bit of architecture. The simplest architecture would be to add a state value to use as the next input to the generator, and which is somehow affected by the previous output. A simple feed-forward neural network does not have any architecture like this, you need to add it.

My first suggestion would be to learn how some simple software pseudo-random number generators (PRNGs) work. A good place to start might be linear congruential generators which are considered very poor quality PRNGs nowadays, but are the sort found in early computer systems.

It should be possible to create a rough approximation to a PRNG from a single neuron. Picking the weight, bias and non-linear functions carefully should allow you to produce something similar to the linear congruential generator, where it generates the next pseudo-random number when you input the previous one. A slightly more sophisticated approach would be to make this a simple recurrent neural network (RNN) so that the neural network has an internal state - such a network could still have a single neuron.

An example neuron that works like this might have a weight of 3, bias of 1 (to generate a logit from input $x$ of $z= 3x+1$) and have activation function $y = 1000\text{sin}(z) \text{ mod } 1$. This is not a normal activation function that you would use to train a neural network - in fact this neural network probably could not be trained at all. It has been specially designed to achieve your goal of generating (pseudo-)random numbers using a single neuron. It is a valid neural network though.

I created the above example neuron, started it with a seed of 0, and got the sequence 0.4710, 0.8543, 0.1536, 0.9479, 0.1142, 0.0683, 0.7741, 0.2985, 0.7315, 0.0777, 0.5078 by feeding back previous values - you can see it is working, although it is probably a very bad random number generator even compared to linear congruential generators. Possibly better than a human though.


There is no thorough proof, theoretical or experimental that Double DQN is better then vanilla DQN. There are a lot of different tasks, paper and later experiments only explore some of them. What practitioner can take out of it is that on some tasks DDQN is better. That's the essence of Deep Mind's "Rainbow" approach - drop a lot of different methods into bucket and take best results.


From what aspect to measure the performance of an object detector?

This is a high level explanation: So most object detectors are generally deep neural networks that return a set of boxes:

box1 : coordinates : confidence
 box2 : coordinates : confidence
 etc...

and loss is computed different model to model. But i think your specifically curious how most do box comparison and loss. Generally they first check if the box is even relevant to the target. To do this often they calculate the Intersection over Union (IOU) as known as the Jaccard index wikipedia link. This checks how overlapped they are biased on the box sizes. Now if its above some threshold (example: boxes have IOU over .5 caluclate a loss, otherwise loss is just 0) they will compute some objective. Objectives can differ, but ideally its something differentiable. Examples include vectorizing and approximiating the IOU, dice loss, etc..

After a quick google search, heres a good tutorial explaining some components commonly used in the field (region proposals, feature extraction, etc) [guide]


Why does Monte Carlo policy evaluation relies on action-value function rather than state-value function?

In Model Based Reinforcement learning, state and state-action values for all states can be calculated based on the bellman equations. The equations are taken from Andrew Ng's Algorithms for Inverse Reinforcement Learning $$V^{\pi}(s) = R(s) + \gamma \sum_{s'}P(s'|s,a)V^{\pi}(s') \\ Q^{\pi}(s,a) = R(s) + \gamma \sum_{s'}P(s'|s,a)V^{\pi}(s')$$
In this setting, $Q^{\pi}$ can be obtained from $V^{\pi}$ because we have access to the transition model $P(s'|s,a)$. The $Q^{\pi}$ values allow us to carry out a step in $\textbf{policy improvement}$ as in policy iteration.
To answer the first bullet point, the first visit or every state visit policy evaluation in the model free setting for $\textbf{state values}$ is not helpful in determining how to carry out model free control because we cannot compute $Q^{\pi}(s,a)$ from $V^{\pi}$ in the model free case.
The update for SARSA in model free control is $$Q(s,a) \rightarrow Q(s,a) + \alpha (r(s) + \gamma Q(s',a') - Q(s,a))$$
Even though we do not know the transition model, we are essentially $\textbf{sampling}$ from $P(s'|s,a)$ by allowing the environment to provide us the possible next states $s'$ that we may end up in. The following update for SARSA is equivalent to computing $$Q^{\pi}(s,a) = R(s) + \gamma E_{s' \sim P(s'|s,a)}[Q^{\pi}(s',a')]$$
Essentially this should give the same $Q^{\pi}(s,a)$ values when we have the ground truth $P(s'|s,a)$ values for the model free case.


Issues with and alternatives to Deep Learning approaches?

I have very little experience with ML/DL to call myself either practitioner, but here is my answer on the 1st question:

At its core DL solves well the task of classification. Not every practical problem can be rephrased in terms of classification. Classification domain needs to be known upfront. Although the classification can be applied to any type of data, it's necessary to train the NN with samples of the specific domain where it'll be applied. If the domain is switched at some point, while keeping the same model (NN structure), it'll have to be retrained with new samples. Furthermore, even the best classifiers have "gaps" - Adversarial Examples can be easily constructed from a training sample, such that changes are imperceptible to human, but are misclassified by the trained model.


What's the threshold to call something 'machine learning'?

T. Mitchell defines machine learning in "Machine Learning" book as

a computer program is said to learn from experience  concerning some class of tasks  and performance measure , if its performance at tasks in , as measured by , improves with experience

Hence, based on the above definition, we can't say a machine learning method to every iterative method.
In your specific example, it is just a non-linear solver such as the Newton method to finding roots.
However, you should notice that a non-specific machine learning method can be used in the learning process. For example, you might need some numerical methods to compute the measure $P$ (in the above definition). But, we can't say that the specified method is a machine learning method.


Word including Hash tag need to be tokanised. It has a special meaning and given a context. The word by itself has a different meaning.


How can the policy iteration algorithm be model-free if it uses the transition probabilities?

Everything you say in your post is correct, apart from the wrong assumption that policy iteration is model-free. PI is a model-based algorithm because of the reasons you're mentioning.

See my answer to the question What's the difference between model-free and model-based reinforcement learning?.


Why is the convolution layer called Conv2D?

A 2D convolution is a convolution where the kernel has the same depth as the input, so, in theory, you do not need to specify the depth of the kernel, if you know the depth of the input.
I don't know which library you are referring to (although you tagged your post with TensorFlow and Keras), but, in TensorFlow, you only need to specify the width and height of the kernel in the Conv2D class, given that the depth is automatically calculated from the depth of the input.
Thus, the $2$ comes from the fact that you slide across two dimensions (i.e. width and height).
On the other hand, in a 3D convolution, the depth of the kernel does not necessarily have the same depth of the input, so, in that case, you also slide across the depth. In TensorFlow, you need to specify the width, height and depth of the Conv3D class.


What noteworthy contributions have Chinese AI researchers made in the field of artificial intelligence?

Well, there's Baidu with former chief scientist Andrew Ng that's doing research on AI and the Microsoft Asia team has won some ImageNet competitions.


Today, prominent machine learning techniques involve trying to minimize some cost function. In many simple cases this cost function is easy to specify, for instance, linear regression is simply trying to minimize the distance between input data and a line of best fit. No matter what the cost function, the agent is trying to minimize it (or maximize a reward function). That is its motivation.

However, as problems become harder it becomes more challenging for humans to design a cost/reward function such that an system/agent is actually trying to do what the humans want it to do. For instance, one might want a cup of coffee and rewards the agent for getting it to them very quickly. In this case, the agent might make the coffee and then throw it at the human which isn’t what the human actually wanted. Something was misspecified (eg. don’t throw or spill it).

Problems like these could result in a rouge AI and its sole motivation would be to minimize its cost function. For instance, this coffee-AI may think that it would never screw up getting coffee (thus get bad reward) if there were no humans to ask for one.


Is there an equivalent model to the Hidden Markov Model for continuous hidden variables?

Kalman filter is what you're looking for.
According to Wikipedia:

The Kalman filter may be regarded as analogous to the hidden Markov model, with the key difference that the hidden state variables take values in a continuous space (as opposed to a discrete state space as in the hidden Markov model).


What AI designs are suited for producing title replacements?

You can use a seq2seq LSTM model. Giving a article will generate a title for it.

You need to train it on data which comprises news articles and their corresponding titles.
You can refer here and here.


What does it mean to have epochs=30 in Keras' fit method given certain data?

Batch size and epochs are independent parameters - they serve very different purposes. Your main question as I understand it (and for general, non-library specific consumption) is what is an epoch and how is the data used for each epoch?

Simply put, an epoch is a single iteration though the training data. Each and every sample from your training dataset will be used once per epoch, whether it is for training or validation. Therefore, the more epochs, the more the model is trained. The key is to identify the number of epochs that fits the model to the data without overfitting.

Your explaination of how batch size affects the training process is correct but not relevant to the question since it has no relation to the epoch training iterations. That is not to say that these values should be considered independently since they have similar effects on the model training process.


Code freezes and never returns when linear_kernel (sklearn.metrics.pairwise) is used on 20M Movielens dataset

A kernel is a precomputed distance function. This means that instead of computing the distances between pairs of points every time you need them, you'll just compute them all a single time, at the start, and cache the values in memory. This is a great idea if the dimensionality of your data is very high, and you expect to have to make a lot of distance computations. For example, if each vector was an high definition image with millions of pixels, computing the distance between two vectors would cost millions of operations.

A problem with this approach is that if you have many exemplars in your dataset, computing and storing the distances between every pair will become hugely expensive. In your case, you have 20 million exemplars. This means you have ~$20,000,000^2$ = $4*10^14$ pairs of distances to compute. If your computer does ~$10^9$ computations per second (as is normal these days), this will take you about $10^5$ seconds, or a bit more than a whole day. Much worse than this, you will need approximately 400TB of RAM to store the kernel. If you don't have this, your system will eventually start writing to disk, which will cause the calculation to take about 3 years instead of 1 day.

The solution here is not to do quadratic work with millions of datapoints. Either use a subset of the data, or try approximate distance methods.


How to treat (label and process) edge case inputs in machine learning?

Unfortunately, the answer here is that "it depends". People have taken different approaches to this problem and I'll describe a few here. None of which however is the "right" answer.
Labeling
When generating benchmark datasets, we actually do have this problem. To be honest, most of the time the labeling is done to the best ability of the human. Sometimes ambiguous or difficult cases are separated out and cleaned, but, usually, labelers are given a set of concrete guidelines to say whether or not something is a cat. In the case where a human is unsure, usually, that data is thrown out or moved over to the "difficult" pile. Unfortunately, this difficult pile isn't publicly available for the most part in many public datasets. However, if you look at most public datasets, even with significant cleaning these cases exist.
Bayesian Deep Learning
One common theme that I've seen is that people add proper probabilistic uncertainties into their models. This is different than the output of a softmax at the end of an object detection network. The output of a softmax is just some number $\in [0, 1]$ that represents the regressed output of a classification within the model. For example, in SSD the softmax is just the classification of a specific anchor box. There is no real "certainty" information associated with it. In most standalone models (and without some pretty hard assumptions) it doesn't have any rigorous probabilistic meaning.
So how would we add a probability? How can we say that "There is a 80% chance that this image is a cat?. The paper, "What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision? does a pretty good job at looking at this problem.
Basically what they do is build Bayesian models to break have their models explicitly output two types of uncertainties.

Aleatoric uncertainty
captures noise inherent in the observations. On the other hand, epistemic uncertainty accounts for uncertainty in the model – uncertainty which can be explained
away given enough data.

You can go through the paper to get a better understanding of what's going on, but, basically, they fit models to regress both the uncertainty associated with the model, and the uncertainty associated with the data.
Zero-Shot Learning
What about the case where you have a very well defined object, but you've never seen it before? Let's say you've seen a bunch of horses but you've never seen a zebra. As a human, you would look at it for a while and basically just assume that it's a horse with black and white stripes. There is an entire field of machine learning dedicated to this topic. I'm not an expert in it personally, but there are plenty of resources online if you're interested.
A Practical Note
In industry, we usually try to scope the problem as best as we can such that we don't have to deal with this as much. Now, there are times when this is clearly inevitable. What I've seen if an object isn't clearly detectable, then the algorithm might fallback to just saying that there's "something" there. Consider the case for self-driving cars. It's good to detect if there are pedestrians in the road, but if you don't know if something is a pedestrian it's still useful to know that there's something in the road. For this, you can fallback to unsupervised methods to help distinguish objects. From a labeling perspective, you could imagine an ontology of objects for this purpose. At the root node of this ontology would be just, "something in the road" and branches off to "car", "pedestrian" or, "bike" for example. If a labeler is not sure if something is a pedestrian, but it's def something in the road that shouldn't be hit then it would be labeled as "something in the road". Again though, this is highly dependent upon the application.


What are the pros and cons of supervised, semi-supervised and unsupervised relation extraction in NLP?

Supervised

Pros:

highest accuracy

Cons:

need a large human-labeled training set
brittle (doesn't work well with examples that are in a different genre from the training set)

Semi-supervised

Relation bootstrapping

Pros:

only requires a small set of labeled data (seed relations)

Cons:

complex iterative process

Distant supervision

Pros:

training happens in one go (no iterative process)

Cons:

requires a big database of relations

Unsupervised

Pros:

don't need any labeled data

Cons:

need to process a huge quantity of unlabeled data (usually web crawling)


How much can/should the non-player character know about the game's world?

Art of NPC creation
I'm assuming this is a standard game, not a game theory application. These AIs tend to be far simpler in theory than actual artificially intelligent agents which are used to solve real world problems. The challenge of building games is that there are few right answers. A game NPC opponent could know practically nothing of the player and be guided by random world exploration activating certain characteristics (attack, strategy) whenever a human is in range or on the other end could know basically everything and have some limiting factors built into the code which reduces their challenge level to the player. This is the art aspect of gaming design. NPCs with a high level of complexity could learn from their interactions and build a "pattern" with some internal reward scoring system for success (similar to reinforced learning). This would represent a very challenging game NPC, especially if the strategy changes upon failure. Think about who will be playing your games, the audience that you would tend to draw, and hone the NPC challengers to them. Hardcore gamers deserve a challenge, and hobby gamers deserve a chance to win. Where that balance between rewarding game experience and challenging game experience lies depends on the 'feel' which you are cultivating as the game designer. You mentioned resources and exploration as elements of the game. For exploration types of games, you may rely on environmental interactions to offset player interactions. A distracted NPC (gathering resources) is less likely to heavily attack the human player unless this player enters the perimeters of the NPC base or scouting area. This may work to keep the gameplay balanced. Keep in mind that some limitations for your NPCs must be absolute, or the game will experience unexpected behavior (bugs). A completely unrestricted NPC is a liability. If the NPC goes too far from the player or gets stuck, it completely ruins the feel of your game.


Is it possible to make a 'forked path' neural network?

Keras Functional APIs can help you define complex models.

You can find the documentation here :
https://keras.io/getting-started/functional-api-guide/.

For example:

# prev_layer is the layer you want to be forked
fork1 = Dense(32, activation='relu')(prev_layer)
fork2 = Dense(32, activation='relu')(prev_layer)

# you do some operations on fork1 to get output1
# and on fork2 to get output2

model = Model(inputs=input_layer, outputs=[output1, output2])


How is neural architecture search performed?

You could say that NAS fits into the domain of Meta Learning or Meta Machine learning.

I've pulled the NAS papers from my notes, this is a collection of papers/lectures that I personally found very interesting. It's sorted in rough chronological descending order, and *** means influential / must read.

Quoc V. Le and Barret Zoph are to good authors on the topic.

The Evolved Transformer
Exploring Randomly Wired Neural Networks for Image Recognition
GRAPH HYPERNETWORKS FOR NEURAL ARCHITECTURE SEARCH
Backprop Evolution
Progressive Neural Architecture Search
*** DARTS: Differentiable Architecture Search
*** Efficient Neural Architecture Search via Parameter Sharing - ENAS
*** Progressive Neural Architecture Search
AlphaX: eXploring Neural Architectures with Deep Neural Networks and Monte Carlo Tree Search
Automatic Machine Learning - Prof. Frank Hutter
Google Brain - Neural Architecture Search - Quoc Le
*** Regularized Evolution for Image Classifier Architecture Search
Autostacker: A Compositional Evolutionary Learning System
Generating Neural Networks with Neural Networks
Finding Competitive Network Architectures Within a Day Using UCT
Neuroevolution: A different kind of deep learning
Evolving Deep Neural Networks
Pieter Abbeel: Deep Learning-to-Learn Robotic Control
*** SMASH: One-Shot Model Architecture Search through HyperNetworks


What is $I$ in the noise described in the paper "Parameter Space Noise for Exploration"?

Yes, since $\tilde{\theta}$ is a vector, to define its distribution one needs a covariance matrix. Here $I$ is the identity matrix, which means that the noise has a zero-mean normal distribution with standard deviation $\sigma$, and different components of this noise are uncorrelated.


Using the opponent's mixed strategy in estimating the state value in minimax Q learning

My understanding is now that the author's formula is deliberate. It seeks to learn a worst-case maximizing policy. The formula I instead suggest would, I believe, instead be Nash Q learning where the agent seeks to learn to play a Nash equilibrium.

After debugging, I have gotten good results with the second formula but cannot speak for the original Minimax Q learning one.


Why does training an SVM take so long? How can I speed it up?

The most likely explanation is that you're using too many training examples for your SVM implementation.

SVMs are based around a kernel function. Most implementations explicitly store this as an NxN matrix of distances between the training points to avoid computing entries over and over again.

In your case, with 75% of 700,000 examples, this matrix will require approximately 250GB of RAM to store, which is more than you're likely to have in consumer hardware.

If your SVM implementation can avoid caching the values, you might get a speedup that way, or you might not (you'll waste a lot of time recomputing them).

A much better way to deal with this is to just not use all of the data, since most of it will be redundant from the SVM's perspective (it only benefits from having more data near the decision boundaries). A good starting place would be to randomly discard 90% of the training data, and see what performance looks like.


Aren't scores in the Wasserstein GAN probabilities?

Figure 3 in the original WGAN paper is actually quite helpful to understand the difference between the score in WGAN and the probability in GAN (see screenshot below). The blue distribution are real samples, and the green one are fake samples. The Vanilla GAN trained in this example identifies the real samples as '100% real' (red curve) and the fake samples as '100% fake'. This leads to the problem of vanishing gradients and the well-known mode collapse of original GANs.
The Wasserstein GAN, on the other hand, gives each sample a score. The benefit of the score is that we can now identify samples that are more likely real than others, or more likely fake. For example, the further to the left a distribution is located, the more negative the WGAN score will be. We have therefore a continuum that doesn't end in 0 and 1, but can compare between samples that are 'good' and those which are 'better'. A normal GAN would identify both as 'good', making further improvement difficult.


Why is the validation performance better than the training performance?

Assuming you pass through the entire validation dataset, this can't be due to shuffling since you still compute the loss/accuracy over the entire dataset, so order does not really matter here. It is more likely that you have a significantly smaller or less representative validation dataset, e.g., distribution of the validation dataset can be skewed towards classes where your model performs better.
What do you mean exactly by improving the situation? Having a better validation accuracy is not necessarily bad. In any case, if you decrease the effect of regularization, e.g., lowering weight decay, training accuracy might go up but your model might generalize worse, i.e.,  you might get a lower validation accuracy.
No, the goal of training is never to maximize training accuracy. You can trivially do so by just memorizing the training dataset. In short, the goal of training is to get good generalization and as long as you get a satisfactory validation accuracy, it is likely that this has happened to some degree (assuming you have a good validation dataset of course).


Image comparison algorithm, trying to figure out how similar two "binary" forms are

I think you look for the meaning of the image (pic of 'a' = a) rather than the regular image similarity. For example, you would like the two images of the 'a' you posted to be classified to the same label instead of, for example, to be classified as similar to an 'o' image which would give a smaller Hamming distance or mean square error.
So as @Jaden Travnik suggested, MNIST digits classification is a good start. It classifies handwritten digits and has human-level solutions implemented by any existing CNN framework. (TensorFlow, Caffe, Theano, to name a few).


How can I calculate the "mean best fitness" measure in genetic algorithms?

The typical way you'll see a GA measured is that an algorithm with a population size of $N$ is ran $K$ times from new random seeds each time. That gives you $K$ total runs of the algorithm, each of which, at the end, had a final population of $N$ individuals. If you take the best of those $N$ from each run, you get $K$ "best" solutions found. The average fitness value of those $K$ solutions is your MBF.
AES here refers to the number of evaluations of the objective function the algorithm required. The reason for this is to provide a standardized amount of computation budget each algorithm used. Imagine if you compared algorithms on the basis of how many generations it took to find a good solution. In that case, I could just increase the population size of my algorithm by 100-fold or 1000-fold, and I'll probably look better. Same number of generations, but I gave my algorithm far more chances to find something good. Suppose I use wall-clock time. Now my algorithm might look better than yours just because I ran it on a much more powerful computer than you did.
The insight to make here is that GAs work by evaluating new search points. Let's just count how many times that happens. You can run a larger population for a shorter number of generations, or you can allocate the same amount of computation to running a longer time with fewer points each generation. What matters is just how many fitness function evaluations you exhausted in finding the answer.
For sudoku, you might have a fitness function that counted the number of rows, columns, or blocks that don't contain the correct digits of 1-9 and you minimize that function. You run your algorithm $K$ times from random seeds, and for each run, you record how many times you had to evaluate that fitness function before you found a $0$ (i.e., the puzzle was successfully solved). Average all $K$ of those counts, and that's your AES.
Generalizing a bit, you might calculate average evaluations to find some "good enough" solution, but the general concept is the same.


You can use LSTM in reinforcement learning, of course. You don't give actions to the agent, it doesn't work like that.

The agent give actions to your MDP and you must return proper reward in order to teach the agent. For example if you implement trading bot, the policy(policy=the agent, which is your LSTM network) will say that at step T it is going to have action 34, which means something to your MDP and you return reward for example -0.03 or +0.05 or whatever depending what that actions is doing at the moment T.

So I get the question like you want to do a supervised learning on a reinforcement learning environment.

You can mimic supervised learning as well, but the idea of reinforcement learning is not that.

Here is how to mimic:

Scenario: you are at step T, lets say you have 3 possible actions -1,0,+1;

In a supervised learning you must give the desired action to the learning process.
In reinforcement learning you must give reward based on if you are happy or not from the agent's action.

So you must have predefined that for -1 you are not happy and you give reward 0.0, for action 0 you are not happy and you give reward 0.0 and for action +1 you are happy and you give reward +100;

I hope this makes things clear.


How does size of the dataset depend on VC dimension of the hypothesis class?

Given a hypothesis set $H$, the set of all possible mappings from $X\to Y$ where $X$ is our input space and $Y$ are our binary mappings: $\{-1,1\}$, the growth function, $\Pi_H(m)$, is defined as the maximum number of dichotomies generated by $H$ on $m$ points. Here a dichotomy is the set of $m$ points in $X$ that represent a hypothesis. A hypothesis is just a way we classify our points. Therefore with two labels we know,

$$\Pi_H(m)\leq 2^m$$

This is just counts every possible hypothesis. The VC dimension is then the largest $m$ where $\Pi_H(m)=2^m$.

Consider a 2D perceptron, meaning our $X$ is $\mathbb{R}^2$ and our classifying hyperlane is one-dimensional: a line. The VC dimension will be 3. This is because we can shatter (correctly classify) all dichotomies for $m=3$. We can either have all points be the same colour, or one point be a different colour - which is $2^3=8$ dichotomies. You may ask what if the points we are trying to classify are collinear. This does not matter because we are concerned with resolving the dichotomies themselves, not the location of the points. We just need a set of points (wherever they may be located) that exhibits that dichotomy. In other words, we can pick the points such that they maximize the number of dichotomies we can shatter with one classifying hyperplane (a triangle): the VC dimension is a statement of the capacity of our model.

To make this clear, consider $m=4$. We can represent the truth table of the XOR gate as a dichotomy but this is not resolvable by the perceptron, no matter where we choose the location of the points (not linearly separable). Therefore, we can resolve a maximum of 8 dichotomies, so our VC dimension is 3. In general, the VC dimension of perceptrons is $d+1$ where $d$ is the dimension of $X$ and $d-1$ is the dimension of the classifying hyperplane.


How to tell a neural network that: "your i-th input is special"

Assume the image can contain objects of class $C_1 \dots C_c$. Assume a set of additional inputs that has a meaning of questions as "contains the image a C_i or C_j or ... ?".
The main problem for the system is classify the image in classes $C_i$. Second problem is answer the implicit question proposed by the remainder inputs.
Thus, better combine two NNs:

first one an object recognizer, with input the image data.
second one a NN that will answer the question implicit in remainder bits, with inputs the output of previous NN and the "question" bits.

In concrete for the example that the question describe:

a NN to classify dog/cat from 32x32 image. Two outputs for probabilities of dog an cat. Or simply one binary output 0:"is a dog", 1:"is a cat".
a second NN with the "1025" binary input ( 0: "look for dogs", 1: "look for cats") and output from previous one. In this case, if everything is ok, it will infer the logic "a==b": "is a dog" and "look for dogs" or "is a cat" and "look for cats".

Note that if you try to solve the problem directly with a full-connected NN with all inputs (1025 in the example), you loss the possibility of use CNN and max-pooling layers, etc. Moreover, split decreases training times, join subjects increases them "exponentially". Not a promising way.


Is there a clustering algorithm that can make n clusters and the n+1 "others" cluster?

So, I've prepared some data that resembles your sketch:
n , u = np.random.normal , np.random.uniform
x = np.concatenate([ n(1.0,0.2,100), n(3.0,0.3,100), u(0,10.0,100)])
y = np.concatenate([ n(7.0,0.4,100), n(5.0,0.3,100), u(0,10.0,100)])
# lets shuffle it a bit
idx = np.arange(x.shape[0])
np.random.shuffle(idx)
data = np.array([x,y])[:,idx]

And then I just tried using sklearn.mixture.GaussianMixture with n+1 = 3 components and default parameters:
gmm = GaussianMixture(n_components=3).fit(data.T)
cls = gmm.predict_proba(data.T).argmax(axis=1)

# Plotting
color = [['r','k','g'][i] for i in cls]
scatter(data[0],data[1],c=color, marker='.')
scatter(gmm.means_[:,0],gmm.means_[:,1],c='b',marker='o')

That seems awfully lot like the thing you've sketched.


Why does an LSTM cycle on initialisation?

This doesn't necessarily answer the question, but it does give some possible solutions to mitigate the problem.

Apparently, the emphasised looping behaviour above is a result of improper initialisation, in which I was initialising with only positive and 0 weights. The looping behavior is largely diminished by proper initialisation, however it is not totally removed. See below for examples:

djbgnuywjkfrrkrkrpueeeeeffrrdv
kmkkkkkkkkkkkrkkkkkkkkkkkkkkkk
clfwtpjbsqeeeeeeesssjjeeeeeeee
ldwgmbvcmmbvcmmmgmmmgmmmgmmmmg
ywzrfotntntnttntntunuwqzffotnt
lvvurrrrrrxafllvvurrrrrrxcfhla

(These are samples from 6 different uniquely initialised LSTM's on the exact same architecture).

I have also found that this common behaviour is typically combated by adding a certain degree of randomness in feeding in the LSTM's output. This is normally done by interpreting the output at time step $t$ of the LSTM as a probability distribution, used to pick the next value fed into the LSTM at $t+1$. This helps break non-confident looping behaviour which is where looping behaviour most commonly occurs through some quick experimentation, while still mostly retaining confident predictions.

I also tested this by randomising the input a little bit, using a probability distribution where 90% of the time the correct input is selected and 10% a random one is, and back-propagating as normal. This didn't seem to have much of an effect on the looping behaviour, although perhaps might work as a good form of regularisation. I am yet to test it.

You can also use the temperature method in LSTMs, which explained really well here: https://cs.stackexchange.com/q/79241/20691.


What is the definition of each of these neural network cell types?

The diagram is nice-ish-looking, and it does lack a convenient list of node type descriptions.  There is some description in their Zoo Flashcards page. Although the zoo is not a clean presentation in every respect, it is quite instructive in several ways. These are of particular interest.

The symbolic shapes and colors visually display the relationship between cell types and the types of networks that employ them, which is quite instructive as an overview.
The use of the term cell, as opposed to neuron, acknowledges the fundamental difference between current artificial network cells and the amazingly intricate and self-modifying nature of biological neurons.

What is still missing is a concise list of each cell type and its purpose for existing in the contexts of its use.  This may be useful.

Input Cell — Provides a point of input and often uses an identity activation function (the output = the input)
Back-fed Input Cell — Provides a point where feedback is deliberately introduced
Noisy Input Cell — Provides a point where chaotic (pseudo-random), thermal, or quantum noise can be introduced
Hidden Cell  — A cell that attenuates (applies a parameterized weight to) the input from all or a select range of activation function outputs from the cells of the immediately prior layer
Probabilistic Hidden Cell — Applies a radial basis function to the difference between the test case and the cell's mean1
Spiking Hidden Cell — Employs a spiking model of activation rather than a perceptron model, working toward a more accurate simulation of biological neurons at the cell level
Output Cell — Provides a point of output and applies the range (min and max) or ordinality (binary, category index)
Match Input Output Cell — Used in auto-encoders to converge on the identity function from network input to network output, usually through a narrowed data path through which features are extracted after convergence
Recurrent Cell — Used in RNN designs such as recurrent artificial networks as a method of applying historic signal state to improve convergence where the time domain is indicated as an important dimension of learning 2
Memory Cell — Used in LSTM (long short term memory) designs as a method of applying historic signal state to improve convergence where the time domain is indicated as an important dimension of learning 2
Different Memory Cell — Used in GRU (gated recurrent unit) designs as a method of applying historic signal state to improve convergence where the time domain is indicated as an important dimension of learning 2
Convolution Cell — Passes the result of a convolution kernel multiplied to a relative range of the cells of the immediately previous layer to the next layer
Pool Cell — Aggregates (usually arithmatically) the results of the cells of a convolution layer

A table of cell types with the features for each might be a next step in presenting the cell types in a comparative summary.  The table would particularly helpful if it had sort up and down gadgets in the header row for each column.

Footnotes

[1] Typically $\varphi(\mathbf{x}) = \sum_{i=1}^I a_i e^{\left[ -\beta \left \Vert \mathbf{x} - \mathbf{c}_i  \right \Vert ^2 \right]}$ where c is the center value of the cell

[2] These cell types integrate historical influence into the signal path at the cellular level, introducing statefulness beyond the learning parameters and network hyper-parameters, providing the potential of Turing completeness to the network after training.


Why are policy iteration and value iteration studied as separate algorithms?

Policy iteration is made up of two steps. The first is a full policy evaluation, where a value function is calculated for the current policy. The second is policy improvement, where the policy is made greedy with respect to the value function.
Value iteration looks to speed things up by stopping policy evaluation after one iteration, make the policy greedy with respect to that value function, and repeat until convergence.
Clearly, these are two different algorithms, hence why they are considered to be different. They are, however, very closely linked, which is why you might consider them to be 'the same thing'. I guess you could say they belong to the same family of algorithm.


How much memory does the Deepstack poker program require?

A DeepStack-style algorithm only requires that you have a way of approximating equilibrium counterfactual values for subtrees at the leaves of lookahead trees from each of its decision points. So if I'm acting at the beginning of the pre-flop and I only have the time and memory available to look ahead to the start of the flop, then I need to approximate the equilibrium counterfactual values at the root of each subtree that begins at the start of the flop.

DeepStack as in the Science paper's study used three neural networks, one for values at the end of the pre-flop, one at the beginning of the flop, and one at the beginning of the turn. These, along with the lookahead tree data structures, had to fit onto the 8GB of GPU memory on the 1080. Although not needed while playing, the networks were trained on millions of examples (ten million for the pre-flop and turn networks, and one million for the flop network).

We have an example DeepStack implementation on GitHub that plays Leduc hold'em if you want to look at some of the programming details in greater depth. Leduc hold'em is a small toy poker game that is commonly used in the poker research community.

The DeepStack algorithm arises out of a mathematically rigorous approach to approximating Nash equilibria in two-player, zero-sum, imperfect information games. Most of the strong poker AI to date attempt to approximate a Nash equilibria to one degree or another. So rather than analyzing poker specifically, we usually use an analysis of properties of imperfect information games more generally to get a general algorithm, then we apply these techniques to poker as a specific example of an imperfect information game.


As a starter: what is the form of training data for image processing

Your question depends heavily on the method you are using for machine learning. It sounds like you want to extract certain features like "curves and straight lines" from your images and use them as training data. This step of extraction is usually not considered part of the training process but part of pre-processing. During pre-precessing you read your image in, extract certain features or perform some transformation and store the new data as your actual training samples or use them for training right away without intermediate storage. Usually storing this information is a good idea if you want to use the image for training more than once and you want to skip the pre-processing step in future training cycles.

Comparing File Storage and Database Storage

How you store your processed data for training is relatively independent from the application of machine learning and the general principles for data storage apply. Storing the data in a flat file is usually very convenient as disk storage is readily available and the APIs for storing and reading files are part of your programming language and OS. A database adds additional complexity to your architecture but has of course it's benefits, especially if you want to make the training data available to other instances or different learning engines. If you are working with a really huge amount of data, a well structured database can help you organize your data better and provides helpful functions for handling this data efficiently.

Keep in mind that the actual training of your AI takes place initially and once completed, you can roll out your AI without all the training data. So you only need to handle training data for a limited time in your application lifecycle. The speed of training is usually not very important, because it happens once and not during the daily use of the AI. Therefore the speed of file access during training can be neglected.

Conclusion

In conclusion, for most applications, the simpler implementation using just flat files is good enough. If you want to store each sample in an individual file or pack them together on bigger batches and use some meta information to identify the individual samples in the file is more a matter of taste than real technical relevance.

Storing data after training

Judging from your question, you sound like you understand machine learning in general. Just to clarify one potential misunderstanding for beginners - you don't have to store any of the "learned" information as data after training, as each training step just adapts weights and biases in the neural network. The actual training data can be thrown away after successful training, if you don't need it for future training cycles.

Further Information / References

To finish my answer I want to recommend a current course from Stanford University about CNNs and visual recognition (Dated Spring 2017). It is a great source of information for implementing neural networks for image recognition.


What is the difference between a semantic network and an ontology?

A semantic network is a way to implement an ontology. An ontology is just a generalised way of representing knowledge in a particular domain, and there are multiple ways of doing so. The key that distinguishes an ontology from, say, Wikipedia, is that it is formally defined, so that the knowledge represented can be used in programs to reason with.

Semantic networks can be used to do that. Please note that there are many types of semantic networks, so you can use them to represent a wide range of relevant information. Since the point of an ontology is to show the relationships between entities relevant to a domain, they are usually represented as networks — if the information is stored as RDF triples, these can usually be visualised as an equivalent network.

So ontology is the broader, more general term, whereas a semantic network is a more specific way of representing information.


How do I improve accuracy and know when to stop training?

Is there anything else I could do to improve accuracy for both training and testing?

Yes, of course, there are a lot of methods if you want to try to improve your accuracy, some that I can mention:

Try to use a more complex model: ResNet, DenseNet, etc.
Try to use other optimizers: Adam, Adadelta, etc.
Tune your hyperparameters (e.g. change your learning rate, momentum, rescale factor, convolution size, number of feature maps, epochs, neurons, FC layers)
Try to analyze your data, with ~75% and 4 categories, is that possible there is one category that difficult to classified?

In essence, you have to do a lot of experiments with your model until you think "it is enough" (if you have a deadline). If you don't have a hard deadline, you can keep improving updating your ML model.

  How can I know if this is the best accuracy I can get?

No, you can't until you compare it with other models/hyperparameters. If you do more experiments (some ways like the one I mentioned above) or compare with other people's experiments that using the same data, you'll find which one is the best. For an academic paper, for example, you need to compare at least 3 to 4 models that similar or experiment with hundreds of different hyperparameters combination.


Using different timesteps for features and target value

I don't know what kind of price data you're dealing with. I suppose the order of the data matters a lot, so my suggestion would be:

Use LSTM as it handles time series better
You can predict 3 consecutive numbers from an RNN as the next three days' predictions
Try regression first, it is likely it will not work (or just flatten the curves, depend on your data noise), then classification is an easier approach
Don't forget normalization


Why is the optimal policy for an infinite horizon MDP deterministic?

Suppose you learned your action-value function perfectly. Recall that the action-value function measures the expected return after taking a given action in a given state. Now, the goal when solving an MDP is to find a policy that maximizes expected returns. Suppose you're in state $s$. According to your action-value function, let's say actions $a$ maximizes the expected return. So, according to the goal of solving an MDP, the only action you would ever take from state $s$ is $a$. In other words $\pi(a'\mid s) = \mathbf{1}[a'=a]$, which is a deterministic policy.
Now, you might argue that your action-value function will never be perfect. However, this just means you need more exploration, which can manifest itself as stochasticity in the policy. However, in the limit of infinite data, the optimal policy will be deterministic since the action-value function will have converged to the optimum.


How would an AI learn idiomatic phrases in a natural language?

Short answer: Yes.

TL;DR

In the presence of good datasets this can be accomplished with a pipeline.

Long Answer

In reality an idiom is a series of words which is supposed to have a semantic meaning that is not denoted by the literal reading (source). This means that any system that is used must be capable of considering multiple words at a time. Additionally, some idioms are context dependent. Example:

The fisherman broke the ice with his tool.

Are we to believe that this is a very suave fisherman?

  So, it is possible to teach an AI to use idiomatic phrases to keep up with the culture of humans?

Observe that humans do not come linguistically "pre-loaded" with idioms. So we can safely assume that idiom usage is a learning task and that the only way for them to keep up is for them to keep learning. So if we solve the idiom learning task we just need to keep our agent online or periodically retrain it on nascent corpora.

One difficulty is that, in the absence of a label, a metaphor could be easily mistaken for an idiom and vice versa. So semantic outlier (sorry it's not free) approaches  may suffer from precision issues. Example:

She's a thorny wildflower (metaphor - could easily be an idiom)
She's a diamond in the rough (idiom - could easily be a metaphor)

Though, idioms will most likely be repeated if a dataset is large whereas a "custom metaphor" is less likely to repeat.

Additionally, some idioms (eg bite the bullet or break a leg) do not have readily available "interpretable semantics" that allow us to extract their intended meaning. For example, if one did not know the idiom "cut me some slack" one could think:

"Slack implies loosening or to make less tight/taut. I was being very uptight. They probably want me to loosen up and not be so critical."

Of course the human understanding of it might happen in a flash and not follow such a delineated path. The idea is that some NLP pipeline might be constructible that satisfactorily handles idioms in some specific use cases (example of a pipeline). For example, one module might attempt to process outliers like "diamond in the rough" which have said interpretable semantics. Though, something like "bite the bullet" may have to be labelled with the correct semantics.

I've only scratched the surface of this. Natural language understanding is already a hard problem - and idioms are thus a tough task in a tough task. I hope that this motivates the reading of some more thorough articles. I have gathered some articles that can be used as a springboard into the literature.

Here's a source that uses a dictionary type approach to train the model to recognize idioms. Excerpt:

  For identification, we assume data of the form ${(⟨p_i,d_i⟩,y_i) : i = 1...n}$ where $p_i$ is the phrase associated with definition $d_i$ and $y_i ∈ \{literal, idiomatic\}$.

This source provides pseudo-code for idiom extraction.

This source describes a dataset to help solve the idiom difficulties.


How to deal with random weights initialization in hyperparameters tuning?

I don't think you can.

Say a NN with 3 layers gives an accuracy of 95.3% and another NN with 4 layers gives an accuracy of 95.4%. Then there is no guarantee that the 4 layer NN is better than the 3 layered NN. Since with different initial values the 3 layer NN might perform better.

You could run multiple times and probabilistically say that this is better, but this is computational intense.


How to build a DQN agent which can be trained through interactive learning?

I suggest you to start reading this one on BERT and this one on GPT-2.

  I am aware this question seems too broad, but some hints will be very helpful for a newbie like me.

I'm not sure you want to create your chatbot using RL architecture at all. But if you want to implement such an idea the right way to go with that is by using iterative approach and starting from a really simple base:

Define a list of actions for your agent. Might be 10-words
vocabulary with simple one-word answers.
Define you simple environment. It might have 20 different states (e.g. greetings/conversation starters hey, hello)
Map good responses (hello -> hi) to positive rewards and -1 for the rest of them.
If you want to move forward with DQN try a simple Q-learning algorithm on your simple environment and train agent to properly response to those.
Once you have done the above you can make your vocabulary bigger
Now replace your table of input words which gives you a state for your network with NN/word2vec/any other NLP approach which will convert your input sentence to a state.
Manually provide a reward as a response for what you agent did according to learned policy.
Do that infinite amount of time and your agent probably will figure out how to respond properly.

NOTE: DQN uses discrete action space. So in such a case you will have only limited or REALLY HUGE actions space. It's all possible combinations between all possible words in you vocabulary (assuming action maps to some sentence)


How can I find what does an specific neuron do in neural network?

In TensorFlow Playground, the horizontal line show where each class is separated for each neuron. What happens when you take any intermediate neuron to make the decision? You can see the answer by the line provided by that neuron. And this decision is a result of the weighted sum from the decisions of the previous neurons (up to activation).

Take the middle-top neuron in the link you share, which is an almost horizontal line - slightly tilted to the right. This neuron classifies everything above it as a blue, and everything below it as an orange. Hover over the neuron to see a larger picture on the output.

You can also see how this is actually calculated by hovering over the line coming from the neurons in the previous layer to the neuron you are looking at. For the case of the same neuron (center-top), the weight coming from the first input ($x_1$) is 0.091, while from the second one ($x_2$), it is 0.49. The neuron ends up being almost horizontal because the contribution from the horizontal input ($x_2$) is so much larger compared to the vertical one ($x_1$).

Of course you need to take into account the nonlinearity coming from the activation function but the idea presented above is the essence of it. The example uses tanh activation, which behaves very linear in its intermediate region so one can ignore this issue to some extend for this particular case.

Edit: It appears that the values for the weights change at every browser session, so the neuron I describe might look a little different to you. To get the same configuration, simply click on the colored lines between neurons to edit them and use the values above for the connections.


How does backpropagation with unbounded activation functions such as ReLU work?

Backprop through ReLU is easier than backprop through sigmoid activations. For positive activations, you just pass through the input gradients as they were. For negative activations you just set the gradients to 0.

Regarding softmax, the easiest approach is to consider it a part of the negative log-likelihood loss. In other words, I am suggesting to directly derive gradients of that loss with respect to the softmax input. The result is very elegant and extremely easy to implement. Try to derive that yourself!


Spontaneous emergence of replicators in Artificial Life

Systems Approach

Let's set out to replicate a real time system $S: \mathcal{X} \Rightarrow \mathcal{Y} \; | \; I$, where $\mathcal{X}$ is a empirical continuous history of input and $\mathcal{Y}$ empirical continuous history of output, conditioned upon a real initial system state $I$.  Based on some definition, we require $S$ to be alive.

We cannot simulate a replication of a theoretical model of life, with a selfish gene or any other such attribute, simply because no mathematically terse model on which the simulation could be based exists.  As of this writing, only hints to and minutia of such a model are known.

Furthermore, models are mathematical representations that, throughout human history, are found to be approximations of complexities once anomalies are addressed and new models develop to incorporate them into the theory.1

Simulation Roughly Defined

If we examine a general algorithm $\mathcal{A}$ to replicate $S$, replication can be roughly sketched as follows.

Estimate system $S$, essentially forming hypothesis $H$.
Simulate initial state $I$.
Initiate a series of discrete stimuli $\mathcal{X}_t$ approximating the real and continuous $\mathcal{X}$.
Acquire resulting system behavior $\mathcal{Y}_t$ as discrete observations of $\mathcal{Y}$.
Verify the difference between simulated and actual systems to be within allowable error $\epsilon$.

Defining Spontaneous Emergence

By spontaneous emergence is meant that such an astronomically large array of initial states and sequences of stimuli occurred that there is a high probability of one of the permutations being alive, based on some specific and reasonable definition of what is living.

Defining What Life Is

Reviewing several definitions of living organisms, the most reasonable definitions include these:

The organism can be distinguished from its environment.
The organism can acquire and cache potential energy and materials required to operate.
Its operation includes continued acquisition, producing a bidirectional and sustainable relationship with its environment.
The organism can roughly reproduce itself.
The reproduction is similar to but not exactly like the parent(s).
The method of energy and materials acquisition may include the consumption of other organisms or its energy and materials.

Competing for resources, natural selection, and all the other features of evolutionary theory are corollary to the above five requirements.  In addition to these, the current trend toward recognizing symbiogenesis as a common theme in the emergence of species should not be dismissed.

Replication of one organism may be influenced by the composition of another organism through forms of assimilation or symbiosis such that traits are passed across categories of organisms.

Artificial Life as a Simulation

These seven criteria poses a challenge for humans attempting to artificially generate life.  It is easy to create a computer model such that life is simulated in some way.  Consider how.

The environment contains virtual energy and virtual matter.
The model of the organism, distinguished from its environment, can acquire its operational requirements from the environment through a set of operations on it.
Mater and energy are conserved because the temperatures are far below nuclear thresholds.
The model of the organism allows acquisition only if enough of the energy and materials acquisition has occurred to maintain the cache.
Mater and energy acquired by one organism cannot be acquired by another organism except by consumption or absorption of an organism that acquired it or produced it from that which was acquired.
The model of the organism can self-replicate in such a way that stochastic differences in the replication is introduced in small quantities.
Operational information, including replication information, may be acquired through consumption or symbiotic relationship under some conditions.

Magical Genes for Spontaneous Life

Notice that the selfish gene is not mentioned above.  Selfishness, the prerequisite of which is intention, is not a requirement for life.  An amoeba does not think selfishly when it moves or eats.  It operates witlessly.  We should not anthropomorphize every organism we study, or develop theory based on anthropomorphic conceptions.

Similarly, symbiotic relationships form that are neither loving nor altruistic.  They exist because there is a mutual benefit that appeared as an unintended byproduct of normal operations and both symbiotic parents happened to pass that symbiotic connection to their respective offspring.
The mutual benefit, the symbiosis, and the replication are witless and unintended.

There need not be a control mechanism distinct from all other replicated mechanisms to control either symbiotic collaboration or competition.  They too are natural consequences of living things sharing an environment.  Whether an organism dies because it

Lost its symbiont,
Starves because other organisms consumed its necessities,
The organism itself depleted its own resources, or
Those needed resources were otherwise rendered unavailable,

it is still unable to replicate, so its traits die with it.

Note also that there is no known molecule that can replicate itself.  Complex systems of molecules in a variety of chemical states and equilibria are required for reproduction to take place.

Returning to Simulating an Already Existing Organism

Running a time sharing system or distributing these simulated organisms in a parallel processing arrangement may some day simulate a biosphere, but it is not one in that only transistor electro-chemistry is involved.  There is no actual direct relationship between the energy and mater of the system used to assemble the simulation and the energy and mater of the simulated environment in which the simulated systems $S$ reside.

Certainly genetic algorithms, such as Avida and Tierra, have been developed.  Compare those simulations to the modelling scenario described above, and their deficiencies become clear.  Human researchers have not yet found $\mathcal{A}$ to replicate $S$ in a way that aligns with biological reality.

Open-endedness Requires Verification to Have Merit

The most significant limitation on implementations in silico, is that they can never be truly open-ended.

There is no way as of this writing to replicate that which was simulated outside the simulation system.  Until nanotechnology reaches a point where 3D construction and assembly can migrate alive simulations into the unsimulated universe, these simulations are closed-ended in that way and their viability in vito is untested.  The value of open-ended simulations without any way to validate them is essentially zero except for amusement.

Even in the space of digital simulation, as far as that technology has progressed, nothing even close to von Neumann's universal constructor has been accomplished.  Although generic functional copy constructors are available in Scheme, LISP, C++, Java, and later languages, such is a minuscule step toward living objects in computers.

Digital Soup

The simulation of life's origins is considerably more difficult than finding an algorithm $\mathcal{A}$ to replicate $S$, where $S$ is a single life form and a sufficient portion of its environment to be representative of the biosphere on earth with an organism in it.

The issue with primordial digital soup is one of the combinatory explosion.  There are 510 million square Km on the earth's surface, and there are only three categories of life origin time frames possible.

The current estimates are close to correct, that the earth formed 4.54 billion years ago and extremely primitive life emerged 3.5 billion years ago
The organic material found in Canada that is allegedly 3.95 billion years old shortens the gap between planetary formation and life formation on it and older terrestrial life may be found
Vladimir Vernadsky's comment that life may have preexisted earth is more than just a possibility

If we go with the 1.04 billion year gap, then $(4.54 - 3.5) \cdot 10^9 \cdot 510 \cdot 10^6$ Km-years of soup must be simulated, since we cannot assume that life started in the ocean or a puddle or even on the surface.  It could have started underground or in the atmosphere.  The biosphere is currently thought to be 1,800 m above to 8,372 m below thick.

With nanobes being 20 nm in diameter and the possibility that the emergence may have only taken one second we have to simulate in three dimensions over time the following space-time domain in finite elements with at least 50% overlap in all three dimensions.

$$\dfrac {2^3 \cdot (4.54 - 3.5) \cdot 10^9 \cdot 510 \cdot 10^6 \cdot (1,800 - 8,372) \cdot 365.25 \cdot 24 \cdot 60 \cdot 60} {(20 \cdot 10^{-9})^3} \\ = 170,260,472,379 \cdot 10^{9+6+27} = 1.7 \cdot 10^{56}$$

With a quantum computer two stories high the size of Switzerland, the computing time would vastly exceed the duration of the average species on earth.  Humans are likely to be extinct before the computation completes.

As the dating of the oldest found fossils converges on the dating of earth, it may seem that life emerged quickly on earth, but that is not a logical conclusion.  If life formed as soon as the earth cooled sufficiently and no evidence of continuous emergence is found in the remaining billions of years, then Vernadsky's inference that life arrived on earth through one or more of the bodies that struck it becomes more probable.

If that is the case, then one must ask the question, if all assumptions are dropped, whether life had a beginning at all.

Simulating Life Versus Simulating Its Formation

We may simulate what life is, that is, find an algorithm $\mathcal{A}$ to replicate $S$, where $S$ is a single live organism.  It is not realistic to, by brute force, simulate how life began without learning more about what conditions can lead to its formation theoretically to drastically reduce the soup simulation space.  It is that learning that is an ongoing area of research in the genetic algorithm field.

Early musings about the possibility of an algorithm $\mathcal{B}$, which can provide the conditions that allow an arbitrary organism $S$ conforming to the above definition of life to form with out a parent or parents were interesting. Given algorithm $\mathcal{A}$ that simulates a life form and algorithm $\mathcal{B}$ that simulate the formation of life, it may be the later algorithm that proves significantly more difficult.

Conforming physics outside a computer to the simulation may be impossible.  Whether simulated life, when embodied in a robotic system is actually going to be considered life will be left to our descendants, should the species endure sufficiently.

Footnotes

[1] Classic cases include the heliocentric Copernican system giving way to the Law of Gravity, that law being shown an approximation of general relativity as shown by the proper prediction of the orbit of Mercury and light's curvature near the sun, the Four Elements dismissed in light of Lavoisier's discovery of oxygen, and absolute provability of truth within a closed symbolic system disproved by Gödel in his second incompleteness theorem and then recouped partially (in terms of computability) by Turing's completeness theorem.


What jobs cannot be automatized by AI in the future?

The Oxford study from 2013 in The future of employment paper assess this and estimated the probability of computerisation for 702 detailed occupations using a Gaussian process classifier (using job data from the UK partially merged with data from US), and based on these estimates they identified three areas of computerisation bottleneck areas and nine skills that people are still needed for each profession, this includes:

Perception and Manipulation.

Finger dexterity.

  The ability to make precisely coordinated movements of
  the fingers of one or both hands to grasp, manipulate, or
  assemble very small objects.

Manual dexterity.

  The ability to quickly move your hand, your hand together
  with your arm, or your two hands to grasp, manipulate, or
  assemble objects.

The need for a cramped work space.

  How often does this job require working in cramped work
  spaces that requires getting into awkward positions?

Creative Intelligence.

Originality.

  The ability to come up with unusual or clever ideas about
  a given topic or situation, or to develop creative ways to
  solve a problem.

Fine arts.

  Knowledge of theory and techniques required to compose,
  produce, and perform works of music, dance, visual arts,
  drama, and sculpture.

Social Intelligence.

Social perceptiveness.

  Being aware of others’ reactions and understanding why
  they react as they do.

Negotiation.

  Bringing others together and trying to reconcile
  differences.

Persuasion.

  Persuading others to change their minds or behavior.

Assisting and caring for others.

  Providing personal assistance, medical attention, emotional
  support, or other personal care to others such as
  coworkers, customers, or patients.

Source: The future of employment: how susceptible are jobs to computerisation: Table 1.

What this study is basically saying, around 50% of all jobs will be replaced by robots in the next 20 years.

Based on the above study, the BBC assembled a handy guide that calculates which jobs are likely to be automated within the next two decades:

Will a robot take your job?

See also: replacedbyrobot.info website.

  With this tool, you can check the prediction of over 700 jobs.

Related:

When robots can do all manual labor and service jobs, what will the majority human population do?
TED: The jobs we'll lose to machines — and the ones we won't (by Anthony Goldbloom)
Labore Ad Infinitum: AI & Automation vs Timeless Tasks

Which suggests: Military/Peacekeeper, Athletes, Therapist, Musical Performer, Actors and Dancer, Visual Artists, Religious/Spiritual Leaders, The World’s Oldest Profession, Virtual Goods, Politicians, Judges, Parenting.


In grammar, a predicate-argument relationship is one which is implied from text but not expressed in the syntactic structure. (Asher S)

Predicate logic or first order logic is a collection of formal systems used in mathematics, philosophy, linguistics and computer science.

The NLP community is interested in recognizing, representing and classifying predicate arguments for shallow semantic parsing which can be viewed as the process of assigning a WHO did WHAT to WHOM, WHEN, WHY, HOW e.t.c. structure to plain text.

This process entails identifying groups of words in a sentence that represent these semantic arguments and assigning specific labels to them. This could play a key role in NLP tasks like information extraction, question answering and summarization.

Automatic and accurate techniques that can annotate naturally occurring text with semantic/predicate argument structure can facilitate the discovery of patterns of information in large text collections.

For reference I recommend this paper that uses Support Vector Machine algorithms for predicate argument classification. (Sameer P Kadri H 2005) https://link.springer.com/article/10.1007/s10994-005-0912-2#enumeration


Using a neural network to identify a stable region within a set of data?

If you're really just trying to find long contiguous flat regions in a sequence, you do not need machine learning. Your PI is mistaken. You would be better off simply writing a short data processing program. Your program could find the finite differences between adjacent datapoints, and then count whether a long string of them are below some threshold to identify long flat regions. This will be faster, simpler, and perhaps more accurate than using ML on data visualizations for this task.

If you are trying to find something more complex than these long flat regions, you could instead train an LSTM on the raw sequential data that you are using to generate the images. Again, that will probably be more accurate than trying to train a CNN, or any non-sequential model, on the image data itself.


Have GANs been used to solve regression problems?

In reality GANs are not made for image classification, but for data generation, and they have gained popularity on image generation. They are also used for tabular data generation, see for example TGAN, or for time series generation, e.g. Quant GAN. You have even some application for the field of graphs and networking, e.g. NetGAN and GraphGAN.


Is traditional machine learning obsolete given that neural networks typically outperform them?

"Assuming that we have sufficient data..." — that's quite a big assumption. Also, traditional methods are well understood, while neural networks (and especially deep learning) is still something of a black box: you train it, and then you get a mapping from input to output. But you don't really know how that mapping is achieved.

It's not only about performance, it's also about efficiency (speed, use of power, etc) and transparency (being able to explain why something happens).

So there are several reasons why we don't put all our eggs into the NN basket:

it is a lot easier to see what's happening and diagnose errors with 'traditional' methods which are well-understood. This is an important point in real-life applications
in many cases we do not have the required amount of training data available that is necessary for deep learning approaches to work
training a DL system is much more time- (and energy) consuming than other algorithms

I'd much rather have a nuclear power plant operated by a traditional algorithm that makes a few mistakes, but nothing drastic (and being aware that it makes these kinds of mistakes allows you to guard against them), than have a total black box doing it where I have no idea why decisions are reached and what happens in edge cases not covered by the training data.

It's fine for toy projects where the stakes are low, but in real-world applications there are often different constraints that DL systems cannot satisfy.


Feature selection of an acoustic voice dataset for classification

It isn't obvious whether the 19 features are spectral and, if so, whether they contain detected transients for consonant phonics.  It's not obvious whether features were extracted using RBMs (restricted Boltzmann machines), from auto-correlation in the time domain, or some other way.  The intention of the classification, which would probably drive the nature of differentiation, isn't stated either.  If more details are added to the question, the answers may be more detailed.  Here are a few general options to compare.

Principal component analysis (PCA)
Factor analysis
Principal axis factoring
Cluster analysis
Regression or gradient descent if category labels exist and then checking early layer parameters for zeroes
Feature selection
Independent component analysis
Feature Selection: A Data Perspective, Jundong Li et. al. 2016


Have any chatbots fooled the judges and won the Loebner Prize Gold medal yet?

The 2016 finals haven't started yet, they will start on Saturday, 17 September 2016.

In the 2015 finals or before that, nobody won the Gold Medal or the Silver Medal.
The most up-to-date data can be found here, where we can find both the results from 2015 and the timeline of the 2016 contest.


What is the internal state of a Simple Neural Attentive Meta-Learner(SNAIL)?

Here's what I understand, welcome to point out any mistakes.

When starting a new episode(but still in the same task), SNAIL does not clear its batches. Instead, it makes decisions based on the current observation and observation-action pairs from the previous episode. In this way, it keeps knowledge of the previous episode whereby achieving few-shot learning in the test time.


Rescaling time-series data with very spiky pattern for training data in LSTM network

First, if your data has a minimum of 0 and maximum of 5000, 1000 will get rescaled to .2 and 5000 will get rescaled to 1. So it's not a .001 difference as you suggest.
If you just used a regular loss function (e.g. mean squared error), I'm not sure you'd be able to achieve good predictions. In the literature this type of data might be called "sharp". There are some special loss functions such as DILATE (link) or soft-DTW (link) which are specially designed for time series like the one you show.