Why am I getting a difference between training accuracy and accuracy calculated with Keras' predict_classes on a subset of the training data?

One problem could be with the selection of the validation set. For your model to work well on data it has not seen as training data is to have a high validation accuracy, but that is not sufficient on its own. The validation set must be large enough and varied enough that its probability distribution is an accurate representation of the probability distribution of all the images. You could have 1000 validation images, but, if they are similar to each other, they would be an inadequate representation of the probability distribution. Therefore, when you run your trained model to make predictions on the test set its performance would be poor.
So the question is: how many validation images did you use and how were they selected (randomly or handpicked)?
Try increasing the number of validation images and use one of the available methods to randomly select images from the training set, remove them from the training set, and use them as validation images.
Keras's flow_from_directory can achieve that or sklearn's train_test_split. I usually have the validation set be selected randomly and have at least 10% as many images as the test set does.
Overtraining is a possibility, but I think unlikely given your validation accuracy is high.
Another thing is how were the test set images selected? Maybe their distribution is skewed. Again, the best thing to do is to select these randomly.
What was your training accuracy? Without a high training accuracy, the validation accuracy may be a meaningless value. Training accuracy should be in the upper 90's for the validation accuracy to be really meaningful.
Finally, is there any possibility your test images were mislabeled? Try switching your test set as the validation set and see what validation accuracy you get.
Here is an example of what I mean. A guy built a CNN to operate on a set of images separated into two classes. One class as "dogs", the other class was "wolves". He trained the network with great results almost 99.99% training accuracy and 99.6% validation accuracy. When he ran it on the test set, his accuracy was about 50%. Why? Well, it turns out all the images of wolves were taken with snow in the background. None of the images of dogs had snow in the background. So, the neural network figures out if snow must be a wolf, if no snow must be a dog. However, in his test set, he had a mixture of wolves in or not in snow, dogs in or not in snow. Great training and validation results but totally useless performance.


Difference between softmax and Logistic Regression?

As written, SoftMax is a generalization of Logistic Regression.
Hence:

Performance
If the model has more than 2 classes then you can't compare. Given K = 2 they are the same.
Computation Requirements
Please explain as the computational requirements require the data, enough memory to hold it and enough time to let run.
Ease of Calculation of Derivatives
The cost function is summation hence once you do it for one element you do it fol all.
Ease of Visualization
Well, it is easy to visualize the Confusion Matrix even for K = 10 classes. So no issue here.
Cost Function
The cost function is convex. Yet not Strictly Convex hence there infinite number of minimas.


Which approach should I use to classify points above and below a sine function $y(x) = A + B \sin(Cx)$?

You can try using Fourier basis functions to transform your observable variables and then apply a general linear regression model. To clarify, if you have pairs of observables $(y_i, x_i)$ where $y_i$ is $i$-th output, and $x_i$ is $i$-th input then you can transform your input variable into vector
\begin{equation}
\phi = [1, \sin(x), \cos(x), \sin(2x), \cos(2x), \ldots, \sin(nx), \cos(nx)]
\end{equation}
where $n$ is some arbitrary number.
Then you will have a standard linear regression model
\begin{equation}
Y = A\Phi
\end{equation}
where $Y = [y_1, \ldots, y_n]$, $\Phi = [ \phi_1, \ldots, \phi_n]$ and $A$ is matrix of unknown parameters that you need to learn.


How could logistic loss be used as loss function for an ANN?

I see why you might be confused. First, the logistic-loss or log-loss is technically called cross-entropy loss. This function is very simple:
$CE = -[y \log(p) + (1 - y) \log(1 - p)]$
This tells basically if the predicted class $y$ was right $y=1$ then the loss is $CE=-\log(p)$, if the predicted class was not the right one then the loss is $CE=-\log(1-p)$.
If we look at the function as a pure math concept we see that:
$CE = f(x) = - \log(x)$
And as you point out, that function is minima-unbounded as its domain is $D(f(x)) = [0, +\infty]$. You can check that in here:

However the trick is that the inputs must be bounded, meaning, the inputs to the loss function must be in range $[0, 1]$. This bounding is achieved by applying a sigmoid activation function as the final "layer" of the network. Then if the inputs to the loss function are bounded the function has a clear minima.
Check how the function looks in reality from one of the most important papers in loss function in the AI world: Focal Loss (I really encourage you to read it as the first section explains in detail the cross-entropy loss). The blue curve is the one you are looking for.

Finally, you might want to review your log-loss/CE function since it should have an asymptote for $f(x=0) = \infty$


Concrete examples of OpenCog's functionality

It depends what you mean by "what OpenCog can do?".

OpenCog, at a high-level, is a loosely coupled collection of various theoretical and variants of conventional methods aimed at constructing the beginnings of an AGI.

With that said, it's purely applied uses are fairly limited. It can do some typical NLP and ML tasks if used correctly, albeit, it will almost always be ineffective when compared to a problem specific solution. However, this is understandable, as OpenCog is not geared at solving narrow problems.

OpenCog, as well as Goertzel's other projects(SingularityNet, AGI society), are broad, top-down attempts at formulating an AGI system.
Outside of AGI circles they are also not oft referenced or researched(although, I am utilizing some aspects of the system in work I am currently crafting).


What chatbots can answer this type of simple question that only requires to keep track of the context?

I think the issue here is that the chatbots you're using aren't very good at "short-term memory". What I mean by is that the bots construct responses that are slowly and incrementally tuned according to the overall usage of the chat bot, from every user. The bots are responding to each message based on how a new user would expect them to. As Alan1 notes, "Men are Man". It's making this response based solely off your single most recent message.

Instead, you are looking for a bot who focuses moreso on persistent memory of the individual conversation. The problem here becomes you're now almost asking for a Natural Language Parser, a big problem many people are working on and something that's years away from existing as robustly as you suggest.

The chat bot not only has to recognize the words 'Socrates', 'Name', and 'Dog'; but that in this sentence, it's the dog's name that is Socrates. That's a lot of information to gain beyond just the words. Which is why from a server / implementation standpoint, the above method is also a lot easier to program (every message just query your server, no need to maintain state - that is, memory of the conversation).

The chat bots can't possibly get enough information from one person to train how to speak and respond, so they 'crowd-source' that information for training. But that means that Clever Bot (or any similar caliber chat bot) won't respond in terms of parsing the meaning of what you're asking.

Taking this even further, one can consider the notion of such a program being Turing-Complete. Supposing we had a chat bot like you're suggesting, we could perhaps show equivalence to a Turing-Machine, or even perhaps show we can do something like decide the halting problem. Off the top of my head I imagine the procedure being basically showing you would be able to decide halting given initial conditions. E.g. Given "Socrates is a man" and "All men die" can we decide if the chat bot will ever be able to deduce if Socrates dies?

I'll work on a formal proof from the latter and post it if it works out.


What are the value of the pixels of the convolved image?

The convolved image can be considered a feature map, where each new neuron represents some indication (or lack-there-of) of a feature in some receptive field of the original image, so no it does not need to be a valid image in the output.

If you specifically care for it to be an image as an output, you can do a couple of things:

1) normalize the produced feature map to some set range that youre working in (0-255 or 0-1)

2) make the filter a valid probability distribution, and you know the output will stay in the same range as the input (ex: Gaussian filters)


Well to give you a short answer, I would say that YES, it would be MORE resistant than a more standard captcha approach...
This being said, I would still go as far as to predict something like a 75-80% successful prediction rates, for a custom model which is designed specifically for defeating a mechanism such as what you describe. The reason why I am fairly confident in such an appraisal, is primarily because of the following:

New techniques which researchers have begun to explore, which are intended to be "Structure Preserving Convolutions" which utilize a higher dimensional filter to store the extra correlation data.

I think that the obfuscation efforts that you mention will definitely help to some degree, although they can be easily defeated by training the model on a dataset which you pull out some portion of the samples during pre-processing and inject the same sort of noise and glitch treatments, etc.

An idea that would be worth exploring would be to process your dataset with an adversarial model which you could then use to generate Adversarial Noise that could then be fed into a pre-process step for your images and replace (or extend) the obfuscation efforts!

TL;DR: If you cant beat 'em, then join 'em! Just train a model to defeat your captcha implementation, and then use the model to generate adversarial examples and then apply obfuscations to your data set accordingly!
For more information on what I am talking about in my suggestion for further obfuscation efforts explore some of the papers you can find on Google Scholar - Ensemble Adversarial Training Examples


Aesthetics of images has a strong subjective element and possibility of multiple dimensions depending on purpose of the media. That means:

It is hard to define what we mean by scoring aesthetics.
Given any well-constrained definition, it is then time-consuming to collect relevant data.

However, there is some interest in the machine-learning community, as media quality would be a very useful metric to sort and filter data on (provided the metric is close enough to the end user who wants to select it). As a result, there are data sets, research papers and pre-built models for this.

Media quality training data can be crowdsourced in a variety of ways, including looking at popularity of items on social media, to paying experts to assess large numbers of images. An example of one open dataset compiled by researchers for this purpose is called AVA.

This data might be reduced to image/quality pairs which you can then train a CNN model to predict the quality metric (score out of 10 for example). This might just be a regression, but other more complex loss functions are also considered.

A quick search for existing models brings up Google's NIMA project, which has more than one implementation available as open-source code. NIMA appears to use multiclass classification approach to predict which ratings humans would most likely give the image, and the resulting score is then a weighted average of the predicted scores - the claimed benefit of that seems to be that it better matches how the quality ratings are sourced, and it will better capture split opinions (e.g. half of people think the image is terrible, but half think it is great is a different type of image to one where everyone thinks it is just average).

Here is an implementation of NIMA by Github account "idealo" looks complete with documentation, and ready to use with pre-built scripts.

Just to show this is not a one-off, here's a blog by Andrej Karpathy about using CNNs to rate selfies which includes some introduction to core CNN concepts.


Should the training data be the same in each epoch?

Let's quickly get out our copies of Deep Learning by Goodfellow et al. (2016). More specifically, I'm referring to page 276.
On this page, the authors argue for a relatively small minibatch size, since there are less than linear returns for estimating the gradient when increasing the minibatch size. Returns here refer to the reduction of the standard error of the mean (gradient per weight) computed over a minibatch.
So, yes. In theory, having unlimited resources, you will get the best performance when averaging the loss over all samples in your dataset. In practice, however, the larger the size of minibatches, the slower the training procedure, and consequently the less the total number of weight updates that can be afforded. Reversely, in practice, the cheaper the weight updates, the quicker the training procedure can converge to a (subjectively) satisfactory result.
Eventually, also Goodfellow et al. state that rapidly computing gradients leads to much faster convergence (in terms of total computations) for most optimization algorithms than when training them more slowly on exact gradients.
So, to summarize: If the main concern is to get to a specific level of accuracy at all, go for rather low minibatch sizes, whereas you could go up to a few hundreds (as the Goodfellow et al. state as a reasonable upper bound on page 148) if you are interested in more accurate gradients for your weight updates.


What is the most biologically plausible representation for the actor and critic?

For the actor, I'd say the 'not mutually exclusive' option is more biologically plausible in the context of muscle systems, where the actions can be seen as simultaneous muscle activations. Maybe at a higher level, an agent thinks of the action as 'go north' or 'go south', but the final outputs which have to control muscles at a lower level have to represent simultaneous muscle activations.

For the critic, I'd say the 'single output node' is more biologically plausible. Agents perceive the world in the form of high dimensional inputs, such as images. The approach where a value function is learned in a tabular fashion where you know the value for every single state doesn't really scale very well and is limited to small discrete state spaces. For biological agents, it makes sense to have a function that senses the current state of the environment and outputs a single number that represents the value, which gives the agent an idea of how things are going so far given the actions it took in the past.


Welcome to AI.SE @vdbuss, and great first question!

This point is touched on in Section 15.2.3 (page 576 in my copy), in the second paragraph, and there's a good exercise at the end of the chapter (15.4) that is designed to get you to think through exactly why these are different procedures. If you want to really absorb it, I suggest trying to work out that exercise! If you want the quick answer, read on.

The basic action of filtering is to generate a probability distribution $P(X_{t+1} | e_{1:t+1})$ using only two pieces of information, specifically the current state distribution $P(X_{t} | e_{1:t})$, and the new piece of evidence $e_{t+1}$. So, when computing the most likely sequence, the algorithm cannot take into account the sequences that are actually possible, while Vitirbi can.

Here's a simple example: suppose I tell you that I'm going to drop you in a maze at one of two locations. I drop you near the top right corner with probability 0.75, and near the bottom left corner with probability 0.25. Suppose further that a Grue is known (with certainty) to live somewhere near the bottom left corner. Using filtering, your maximum aposterori estimate for your location after being dropped in the maze ($t=1$) is that you are in the top right corner. You then move 1 step to the right and can see a Grue. Clearly your estimate for your position in the second timestep $(t=2)$ must be the bottom left, because Grues only live there. But, you definitely can't end up moving to the bottom left by moving right from the top right, so your sequence has probability zero overall, despite using the maximum aposterori estimate for position at every step. To avoid this, Vitirbi uses a linear amount of extra space to select the maximum aposterori sequence, which in this case is clearly that you are near the bottom left in both timesteps.


Are absence of labels for classes of interest in a vision dataset a big problem?

So depends on sizes— if the second dataset has a lot of traffic lights without labels and you in a basic setup train on it, it will cause large performance decreases on that class. If it’s small, then it may just be “noise” to the model and it’ll filter it out, and still perform well. My recommendation is to adjust the network slightly and add a flag input saying which dataset the input was from: if from first, train normally. if from the second then ignore all losses pertain to boxes/classes that the network thinks a traffic light was in: that way your labels don’t try to tell the mode l “there’s no traffic light there”


Reinforcement Learning can be explained by a few equations. However I assume that this is not what you are looking at since the explanation should be for someone having a non-STEM background. Not to say non-STEM folks are not able to understand math equations, but intuition comes easier with words and examples in my opinion.

Reinforcement Learning is about learning an optimal behavior by
repeatedly executing actions, observing the feedback from the
environment and adapting future actions based on that feedback.

Let's break down the last sentence by the concrete example of learning how to play chess:
Imagine you sit in front of a chess board, not knowing how to play. The optimal behavior you'd like to learn is what moves to execute in order to win the game. So you start learning the game by playing a few moves (actions) with some figures and observing what is happening on the board (environment) and identifying which moves bring you closer victory or give you a better position on the board (feedback). Therefore in future games you will prefer moves which gave you a positive outcome in previous games.
Admittedly this is a very slow process of learning if you don't have a teacher which helps you in the beginning and you would have to play a lot of games until your first victory. But this essentially how computers (and sometimes humans in some sense) learn to do certain things by Reinforcement Learning. Behaviors which lead to positive experiences are collected, memorized and thus reinforced.


What is the optimal value function of the scaled version of the reward function?

The Bellman optimality equation is given by
$$q_*(s,a) = \sum_{s' \in \mathcal{S}, r \in \mathcal{R}}p(s',r \mid s,a)(r + \gamma \max_{a'\in\mathcal{A}(s')}q_*(s',a')) \tag{1}\label{1}.$$
If the reward is multiplied by a constant $c > 0 \in \mathbb{R}$, then the new optimal action-value function is given by $cq_*(s, a)$.
To prove this, we just need to show that equation \ref{1} holds when the reward is $cr$ and the action-value is $c q_*(s, a)$.
\begin{align}
c q_*(s,a)
&= \sum_{s' \in \mathcal{S}, r \in \mathcal{R}}p(s',r \mid s,a)(c r + \gamma \max_{a'\in\mathcal{A}(s')} c q_*(s',a')) \tag{2}\label{2}
\end{align}
Given that $c > 0$, then $\max_{a'\in\mathcal{A}(s')} c q_*(s',a') = c\max_{a'\in\mathcal{A}(s')}q_*(s',a')$, so $c$ can be taken out of the $\operatorname{max}$ operator. Therefore, the equation \ref{2} becomes
\begin{align}
c q_*(s,a)
&= \sum_{s' \in \mathcal{S}, r \in \mathcal{R}}p(s',r \mid s,a)(c r + \gamma c \max_{a'\in\mathcal{A}(s')} q_*(s',a'))  \\
&= \sum_{s' \in \mathcal{S}, r \in \mathcal{R}}c p(s',r \mid s,a)(r + \gamma \max_{a'\in\mathcal{A}(s')} q_*(s',a')) \\
&= c \sum_{s' \in \mathcal{S}, r \in \mathcal{R}} p(s',r \mid s,a)(r + \gamma \max_{a'\in\mathcal{A}(s')} q_*(s',a')) \\
q_*(s,a) &= \sum_{s' \in \mathcal{S}, r \in \mathcal{R}} p(s',r \mid s,a)(r + \gamma \max_{a'\in\mathcal{A}(s')} q_*(s',a')) \tag{3}\label{3}
\end{align}
which is equal to the the Bellman optimality in \ref{1}, which implies that, when the reward is given by $cr$, $c q_*(s,a)$ is the solution to the Bellman optimality equation. Consequently, in this case, the set of optimal policies does not change.
If $c=0$, then \ref{2} becomes $0=0$, which is true.
If $c < 0$, then $\max_{a'\in\mathcal{A}(s')} c q_*(s',a') = c\min_{a'\in\mathcal{A}(s')}q_*(s',a')$, so equation \ref{3} becomes
\begin{align}
q_*(s,a) &= \sum_{s' \in \mathcal{S}, r \in \mathcal{R}} p(s',r \mid s,a)(r + \gamma \min_{a'\in\mathcal{A}(s')} q_*(s',a'))
\end{align}
which is not equal to the Bellman optimality equation in \ref{1}.


When to expand and when to simulate in Monte Carlo Tree Search?

The most common strategy is to simply expand exactly one node per iteration; you can view this as expanding the first node of the Play-Out phase ("simulation" in your image), and not expanding any other nodes of the Play-Out phase. This is also what's done in your image.

That is the most common and probably most simple strategy, but it's certainly not the only one. It is pretty much the minimum you have to expand, but you can expand more if you like. The most common approach of only expanding one node per iteration is basically there because it minimizes the risk of running out of memory; when you only add one new node to the tree per iteration, the tree grows relatively slowly, so you'd have to keep the algorithm running for a really long time before you run out of memory.

If you're not afraid of running out of memory, you can choose to expand as many nodes as you like. For example, my General Video Game AI agent expands every single node encountered during the play-out at once, because in that particular domain we only get to run relatively few iterations anyway so I'm not afraid of running out of memory, even if I do expand lots of nodes.

The backpropagation step can only store results (information) from your iterations in nodes that actually exist, nodes that have been expanded. So, the benefit of expanding more (if possible) is that you'll retain more information from early iterations, your backpropagation step immediately gets to store results in all nodes along the path instead of only the top few nodes. Generally, this is a relatively minor benefit; nodes deep in the tree / deep inside the simulations are quite unlikely to ever get visited more than once anyway, and then this doesn't matter. But expanding more can, in theory, result in slightly more accurate value estimates in nodes closer to the root.


What are the flaws in Jeff Hawkins's AI framework?

The short answer is that Hawkins' vision has yet to be implemented in a widely accessible way, particularly the indispensable parts related to prediction.

The long answer is that I read Hawkins' book a few years ago and was excited by the possibilities of Hierarchical Temporal Memory (HTM). I still am, despite the fact that I have a few reservations about some of his philosophical musings on the meanings of consciousness, free will and other such topics. I won't elaborate on those misgivings here because they're not germane to the main, overwhelming reason why HTM nets haven't succeeded as much as expected to date: to my knowledge, Numenta has only implemented a truncated version of his vision. They left out most of the prediction architecture, which plays such a critical role in Hawkins' theories. As Gerod M. Bonhoff put it in an excellent thesis1 on HTMs,

  "In March of 2007, Numenta released what they claimed was a “research
  implementation” of HTM theory called Numenta Platform for Intelligent
  Computing (NuPIC). The algorithm used by NuPIC at this time is called
  “Zeta1.” NuPIC was released as an open source software platform and
  binary files of the Zeta1 algorithm. Because of licensing, this paper
  is not allowed to discuss the proprietary implementation aspects of
  Numenta’s Zeta1 algorithm. There are, however, generalized
  concepts of implementation that can be discussed freely. The two most
  important of these are how the Zeta 1 algorithm (encapsulated in each
  memory node of the network hierarchy) implements HTM theory. To
  implement any theory in software, an algorithmic design for each
  aspect of the theory must be addressed. The most important design
  decision Numenta adopted was to eliminate feedback within the
  hierarchy and instead choose to simulate this theoretical concept
  using only data pooling algorithms for weighting. This decision is
  immediately suspect and violates key concepts of HTM. Feedback,
  Hawkins’ insists, is vital to cortical function and central to his
  theories. Still, Numenta claims that most HTM applicable problems can
  be solved using their implementation and proprietary pooling
  algorithms."

I am still learning the ropes in this field and cannot say whether or not Numenta has since scrapped this approach in favor of a full implementation of Hawkins' ideas, especially the all-important prediction architecture. Even if they have, this design decision has probably delayed adoption by many years. That's not a criticism per se; perhaps the computational costs of tracking prediction values and updating them on the fly were too much to bear at the time, on top of the ordinary costs of processing neural nets, leaving them with no other path except to try half-measures like their proprietary pooling mechanisms. Nevertheless, all of the best research papers I've read on the topic since then have chosen to reimplement the algorithms rather than relying on Numenta's platform, typically because of the missing prediction features. Cases in point include Bonhoff's thesis and Maltoni's technical report for the University of Bologna Biometric System Laboratory2. In all of those cases, however, there is no readily accessible software for putting their variant HTMs to immediate use (as far as I know). The gist of all this is that like G.K. Chesterton's famous maxim about Christianity, "HTMs have not been tried and found wanting; they have been found difficult, and left untried." Since Numenta left out the prediction steps, I assume that they would be the main stumbling blocks awaiting anyone who wants to code Hawkins' full vision of what an HTM should be.

1Bonhoff, Gerod M., 2008, Using Hierarchical Temporal Memory for Detecting Anomalous Network Activity. Presented in March, 2008 at the Air Force Institute of Technology, Wright-Patterson Air Force Base, Ohio.

2Maltoni, Davide, 2011, Pattern Recognition by Hierarchical Temporal Memory. DEIS Technical Report published April 13, 2011. University of Bologna Biometric System Laboratory: Bologna, Italy.


Neural network Model to infer inputs given an output

I'm assuming that you mean your inputs are 4-dimensional vectors and your outputs are 1 dimensional vectors. If you think that the output vectors give you enough information to completely determine the inputs (at least approximately), then just treat your 1D "outputs" as inputs and your 4D "inputs" as outputs.

Another case is that very different inputs are mapped to similar outputs in your dataset, and you really just want to compute a prototype input for a given output. For example, you might want to produce an image of the letter "a" from only its one-hot label. There are many ways to do this. Here's one that I think is cool:

First, train a neural net the way you normally would with 4D inputs mapping to 1D outputs. Now let's say you're given a 1D output y. Generate some 4D noise x' and pass it through the net to obtain y'. Now pretend that the values in this input are parameters in your network, and hold your actual parameters constant. Use y and y' to compute a loss, and perform gradient descent on your synthetic input. If you repeat this enough times, you should get something that your network thinks deserves the output y.


How can I leverage artificial intelligence and virtual reality to create intelligent automatic story generation?

The VR and computer speech aspects are entirely corollary.  Adding them in would be relatively trivial in comparison to creating an algorithm that can dynamically generate stories of interest to humans.  Essentially, aesthetic components not related to story structure (images, sounds, speech) are "window dressings".

A story generation algorithm would have to focus on the structural elements of storytelling, possibly in conjunction with human-user reactions, and generate narratives narratives dynamically.

Such a system would be validated by human reaction to the stories, and the validation would not require graphics, sounds or speech.  Think of a movie or video game with a story as the expressions of stories that originally had textual form.

Game theory could certainly be applied to this process--creation of narrative in particular utilizes game theoretic concepts, consciously or unconsciously, in terms of strategies to engage reader/viewers and provide "emotional payoffs".

It should certainly be possible to develop an algorithm with these capabilities, even where the algorithm doesn't not really "understand" the content it is generation.

A similar endeavor might be found in Cameron Browne's work on algorithms that can create simple, combinatorial games of interest to humans: Evolutionary Game Design.

It's important to note that the game created by Ludi are based on pre-existing mechanics, and algorithms don't yet have the capability to make profound leaps in the way that human game designers can. (In other words, it will likely be a very long time before an algorithm can create a game like Magic: The Gathering where no such gaming system previously existed.)

However, this doesn't present much of an issue in the context of procedural narrative generation in that the most successful literary work in terms of audience/sales tends to be based on pre-existing formulas.  So an AI that can merely reconfigure elements in a formula should be able to generate narratives of interest to the average human.


What are support values in a support vector machine?

In the least-squares SVM (LS-SVM) the non-zero Lagrange multipliers ($\alpha$) are the support values.  The corresponding data points are the support vectors.  Johan Suykens explains this in Least Squares Support Vector Machines.


What is the relation between an environment, a state and a model?

Environment
This is the manifestation of the problem being solved. It might be a real physical situation (a road network and cars), or virtual on a computer (a board game on a computer). It includes all the machinery necessary to resolve what happens. E.g. in the real world the objects involved, how the agent exerts its control when taking actions, and the applicable real-world laws of physics. Or, in a simulated world, things like the rules of a board game, implemented in code.
State
This is the representation of a "position" at a certain time step within the environment. It may be something the agent can observe through sensors, or be provided directly by the computer system running a simulation.
For RL theory to hold, it is important that the state representation has the Markov Property, which is that the state accurately foretells the probabilities of rewards and following state for each action that could be taken. You do not need to know those probabilities in order to run RL algorithms (in fact it is a common case that you don't know). However, it is important that the dependency between the state+action and what happens next holds reliably.
The state is commonly represented by a vector of values. These describe positions of pieces in a game, or positions and velocities of objects that have been sensed. A state may be built from observations, but does not have to match 1-to-1 with a single observation. Care must be taken to have enough information to have the Markov Property. So, for instance, a single image from a camera does not capture velocity - if velocity is important to your problem, you may need multiple consecutive images to build a useful state.
Model
In reinforcement learning, the term "model" specifically means a predictive model of the environment that resolves probabilities of next reward and next state following an action from a state. The model might be provided by the code for the environment, or it can be learned (separately to learning to behave in that environment).
Some RL algorithms can make use of a model to help with learning. Planning algorithms require one. So called "model free" algorithms do not because they do not make use of an explicit model, they work purely from experience.
There are broadly two types of model:

A distribution model which provides probabilities of all events. The most general function for this might be $p(r,s'|s,a)$ which is the probability of receiving reward $r$ and transitioning to state $s'$ given starting in state $s$ and taking action $a$.

A sampling model which generates reward $r$ and next state $s'$ when given a current state $s$ and action $a$. The samples might be from a simulation, or just taken from history of what the learning algorithm has experienced so far.

In more general stats/ML, the term "model" is more inclusive, and can mean any predictive system that you might build, not just predictions of next reward and state. However, the literature for RL typically avoids calling those "model", and uses terms like "function approximator" to avoid overloading the meaning of "model".


How can Transformers handle arbitrary length input?

Actually, there is usually an upper bound for inputs of transformers, due to the inability of handling long-sequence. Usually, the value is set as 512 or 1024 at current stage.
However, if you are asking handling the various input size, adding padding token such as [PAD] in BERT model is a common solution. The position of [PAD] token could be masked in self-attention, therefore, causes no influence. Let's say we use a transformer model with 512 limit of sequence length, then we pass a input sequence of 103 tokens. We padded it to 512 tokens. In the attention layer, positions from 104 to 512 are all masked, that is, they are not attending or being attended.


What is expectation-maximization in machine learning?

EM algorithm is a numerical method. It is not specific to any machine learning model. Common applications include hidden markov model and mixed Gaussians. The algorithm is not a classifier.
Logistic regression is a statistical model. You need to pick a numerical method for logistic regression.
Naive Bayesian is a statistical model. You need to pick a numerical method (if closed-form posterior distribution not available).

You will need to understand maximum likelihood before you tackle the EM algorithm. Briefly, the maximum likelihood is a method for estimating the most likely parameters in your model. For instance, if you have a sequence of randomly and identically distributed Gaussian random variables, the maximum likelihood estimator for your Gaussian mean is just the sample mean.

When you fit a logistic regression, you use a numerical method (e.g. iteratively reweighted least squares) to maximise your log-likelihood function.

Everything is good, but it's not possible to maximum the likelihood directly if you have some latent variables. A common example is modelling your DNA sequences with hidden markov model, where the latent state is unknown.

You can't do it because you don't know the latent variables. If you do, they are not latent by definition.

EM algorithm is a numerical method to estimate maximum likelihood when you have latent variables. The mathematics is complicated but the idea is simple. You start off with some initial values for your parameters. You update your parameters and latent variables, and the algorithm converges when the change in the log-likelihood function falls below some threshold.


Are there any advantages of using rules-based approaches versus models for detecting spam?

Yes, there would be a point. Assuming your rule set is accurate, then you can use data classified with it to train a model. This model can be expected to be more robust and properly categorise emails that your rule-set will not handle.

Why? Machine learning algorithms generally work on features, and identify relationships between those features that lead to a classification decision. A human rule author basically does the same, but they might not notice subtle relationships; a good ML algorithm, however, might pick those up.

So you could have a hybrid model, where you first use your rule-based classifier, and anything that does not get classified is then run through the ML classifier.


Will an AI ever understand its own functionality?

This is a great question, elements of which I have also been pondering on, though we are very far from being able to actually wrestle with it algorithmically.  This question raises all kinds of metaphysical questions (Kant himself showed that pure reason is not sufficient for all questions, but I'm going to avoid that rabbit hole and focus on the mechanics of your question.)

Consciousness: This is distinct from self-awareness, and fundamentally, may be said to require only awareness of something.

  Consciousness, most scientists argue, is not a universal property of all matter in the universe. Rather, consciousness is restricted to a subset of animals with relatively complex brains. The more scientists study animal behavior and brain anatomy, however, the more universal consciousness seems to be. A brain as complex as the human brain is definitely not necessary for consciousness.
  Source: Scientific American "Does Self-Awareness Require a Complex Brain?"

Thus, an automata that receives input may be said to be consciousness, with the caveat that this idea is probably still considered radical.  The key is distinguishing mere "consciousness" from much more complex concepts such as self-awareness.

Self-Awareness: the holy grail.  This is the idea that a set of elements, such as a human organism, is aware of itself.

But this is sticky, because automata that use Machine Learning are "aware" of themselves in that the may modify their "thought" process and even their "physical" structure.

But ML systems are certainly not self-aware in the human sense.  A question might be, is this simply a function of these systems not being full Algorithmic General Intelligences, or is there more to it?  If there is more to it, is it strictly a metaphysical question, or can an answer be derived through purely rational means?  Even if the latter were the case, there is still the problem of subjectivity, as in: "Is the automata truly self-aware or is it just mimicking self-awareness?" which brings us back to the metaphysical question of "Is there a difference?".

However,

If there were a full Algorithmic General Intelligence that had consciousness equatable with human consciousness, that was aware, and even able to work with the basic components of it's corpus*, it would certainly be able to grasp that it's consciousness is a function of the "bits and bytes", just as a human is aware we are soft machines, and that our consciousness is a function of our bodies and minds.

I intentionally use corpus because it relates both to text (which may be code or even a string of bits in its most reduced form, per the concept of a Turing Machine) and also has an anatomical meaning, as in the body of an organism.  Corpus comes from the Latin and the extension of its meaning to include matter-as-information is modern.


Why are the value functions sometimes written with capital letters and other times with lower-case letters?

In the Sutton and Barto book $q(s,a)$ is used to denote the true expected value of taking action $a$ in state $s$, whereas capital $Q(s,a)$ is used to denote an estimate of $q(s,a)$. However, there is likely to be a lot of inconsistency in the literature as each author has their own preference on how to denote things. I would encourage you to consider whether the value you are reading is to denote an estimate or the true value.


What are the main differences between a language model and a machine translation model?

The simple language model will give you the probability of a sequence of tokens(sentence) for that language. So lets say if you have trained a model for English language your model can give you the probability for any random english sentence.
Consider some sentence
$X$ $=$ "the quick brown fox jumps over the lazy dog" $=$  $x_1 \ x_2 \ x_3 \ ... \ x_n$
model will give you $P(X)$
Moreover if the model has been trained properly, in the following scenario, where
$X$ $=$ "the quick brown fox jumps over the lazy dog" $=$  $x_1 \ x_2 \ x_3 \ ... \ x_n$
$Y$ $=$ "dog brown quick fox over the jumps lazy the" $=$  $y_1 \ y_2 \ y_3 \ ... \ y_m$
model will always give $P(X) > P(Y)$, as it has learnt the structure of language.
On the other hand Machine translation model gives you the conditional probability of the next token given your source sentence and partial-target sentence.
So if
$X = $ "I am a student" and $Y = $ "je suis" $=y_1, \ y_2$
model will give you $P(y_3 | X,y1_,y_2)$ .
where $X$ is a source sentence and $Y= y_1, y_2$ is a partial target sentence. The probability of word/token "étudiant" would be maximum among all words of vocabulary.


Noise in the data, to a reasonable amount, may help the network to generalize better. Sometimes, it has the opposite effect. It partly depends on the kind of noise ("true" vs. artificial).

The AI FAQ on ANN gives a good overview. Excerpt:

  Noise in the actual data is never a good thing, since it limits the accuracy of generalization that can be achieved no matter how extensive the training set is. On the other hand, injecting artificial noise (jitter) into the inputs during training is one of several ways to improve generalization for smooth functions when you have a small training set.

In some field, such as computer vision, it's common to increase the size of the training set by copying some samples and adding some noises or other transformation.


I am not sure what you are really looking for but I leave here this paper here, where some intuition into that direction is provided. This paper compares the performance of a deep learning model scaling in 3 dimensions: resolution, width and depth. As depicted in their definition:

If you go to section 3.2 you will see how scaling the different dimensions independently (resolution, width and depth) affects performance and how by performing a compound scaling the performance of the model is maximized (so there is a close relation). It is a very thorough ablation study. For me this was the paper where I understood how these Width, Depth and Resolution parameters come together.
Rethinking Model Scaling for Convolutional Neural Network: https://arxiv.org/abs/1905.11946


What are some new deep learning models for learning latent representation of data?

Here's a link to my answer on CV Stack Exchange, where I have mentioned about latent spaces and some deep learning models that learn these representations: https://stats.stackexchange.com/questions/442352/what-is-a-latent-space/442360#442360

In short, deep learning models for Domain Adaptation, Computer Vision, Natural Language Processing, Recommendation Systems, Music/Speech/Audio processing, Adversarial models, etc., all learn some form of latent representation of data.

In fact, any place we're learning a function to map input and output spaces of a dataset, the model essentially learns a latent representation of data irrespective of whether the model is based on deep neural networks or a stochastic method or any other.


Techniques for sentence transformation and consistency

Strictly speaking, this is impossible. Changing the form of a sentence also changes its meaning. Even active-passive can be important, as you would use it to emphasise what is important: was it relevant what the cat was doing, or was it more relevant what happened to the mouse? True, the purely propositional meaning is not affected by this, but that is only one component of a sentence's meaning.

There has been a lot of work in traditional linguistics about sentence form. You could look at one of the seminal works, Syntactic Structures by Noam Chomsky, where he introduces the concept which later lead to transformational grammar. This influenced a lot of subsequent linguistic approaches, but as far as I am aware transformations are not really that much in the linguistic focus anymore.

For your second question, stylistic consistency, you could look at the work of Douglas Biber. His book Variation across speech and writing introduces a number of (easily extractable) linguistic features that you could use to quantify consistency.


What are the ways to calculate the error rate of a deep Convolutional Neural Network, when the network produces different results using the same data?

Unless the entropy is collected from quantum, thermal, or other true entropy sources, it is considered pseudo-random at best.  This is why you can have a deterministic process that is not perfectly repeatable, exhibiting some variance.

List of Sources of Pseudo-random Behavior

The list of sources in the question is a good start.

Initialization
Ddropout used in regulation
Mini-batch shuffle

If stochastic gradient descent is used in the back-propagation, that could be an additional source, and there may be others.  It is difficult to say without scrutinizing the code.  CDN emerged along different paths from CNN and MLP so there is no unambiguously distinct classical version of it.

The Right Way

Since the variance is a known phenominon and the norm, there may be no particular question for which the academic community needs to be consulted for an answer.

The right ways to deal with the non-repeatability of validation statistics related to object recognition also has a norm: Follow standard reporting conventions that apply to all experimental results reporting.

The only solution that is mathematically rigorous and not changing the experiment to produce a desired result (which is not science) is the first of the two found in your investigation.

The suggestion to perform $N$ runs where $N \ge 30$ and report mean and variance (not standard deviation) is reasonable and responsible.
The suggestion to do this with $O$ ADDITIONAL models where $O \ge 3$ is another such practice.

Freezing is Necessarily Arbitrary

Some would consider freezing pseudo-random sources arbitrarily something other than, "making reproductive results," and consider it fabricating reproducable results.

Unfortunately, the temptation to please the reader or corporate stakeholder occasionally preempts truth in reporting.  The result is counterproductive.  In the worst case, it leads to broad acceptance of the wrong way.  Those in the academic community should know that, whether or not they unanimously do.  Be a leader and do what is backed by solid reasoning, whether that is the first suggestion found or a third choice that does not involve modifying the experiment arbitrarily to force results to appear more exact than the algorithm, methodology, or framework actually is.


Are there any agents that are based on quantum computing?

I think you are looking for quantum machine learning (QML), which is a relatively new field that sits at the intersection of quantum computing and machine learning.
If you are not familiar with quantum computing (QC) and you are interested in QML, I suggest that you follow this course by prof. Umesh Vazirani and read the book Quantum Computing for Computer Scientists (2008) by Yanofsky and Mannucci. If you have a solid knowledge of linear algebra, you should not encounter big problems while learning QC. Be prepared to deal with the weirdness and beauty of qubits, quantum entanglement, and so on.
If you want to directly dive into QML (although I am not familiar with the details of QML, I suggest that you first get familiar with the basics of QC, which I am familiar with) there are already several courses, papers, overviews, and libraries (such as TensorFlow Quantum) on quantum machine learning.
If you are interested in quantum reinforcement learning, maybe have a look at the paper Quantum Reinforcement Learning (2008) by Daoyi Dong et al.


Which Rosenblatt's paper describes Rosenblatt's perceptron training algorithm?

The paper (or report) that formally introduced the perceptron is The Perceptron — A Perceiving and Recognizing Automaton (1957) by Frank Rosenblatt. If you read the first page of this paper, you can immediately understand that's the case. In particular, at some point (page 2, which corresponds to page 5 of the pdf), he writes

  Recent theoretical studies by this writer indicate that it should be feasible to construct an electronic or electromechanical system which will learn to recognize similarities or identities between patterns of optical, electrical, or tonal information, in a manner which may be closely analogous to the perceptual processes of a biological brain. The proposed system depends on probabilistic rather than deterministic principles for its operation, and gains its reliability from the properties of statistical measurements obtained from large populations of elements. A system which operates according to these principles will be called a perceptron.

See also Appendix I (page 19, which corresponds to page 22 of the pdf).

The paper The perceptron: A probabilistic model for information storage and organization in the brain (1958) by F. Rosenblatt is apparently an updated and nicer version of the original report.

A more accessible (although not the most intuitive) description of the perceptron model and its learning algorithms can be found in the famous book Perceptrons: An Introduction to Computational Geometry (expanded edition, third printing, 1988) by Minsky and Papert (from page 161 onwards).


What is meant by the rank of the scoring function here?

$\text{rank}(f((r,e)|u))$ in $A_t(u)$ means to compute the value of scoring function $f$ for all pairs $(r,e)\in A_t$ which are conditioned by $u$, then sort them in a descending order. The rank of the $f((r,e)|u)$ in this order is equal to $\text{rank}(f((r,e)|u))$. Hence $\text{rank}(f((r,e)|u)) \leqslant \alpha$ means to select the $\alpha$ top most scored pairs.


How important is it that the generator of a generative adversarial network doesn't take in information about input classes?

If you're building a straight "vanilla" generative adversarial network, it's best to understand the network as a statistical engine:  You are training the generator on samples of a statistical distribution.  (And you're training the discriminator to distinguish between "ground truth" images, and images from that generator.)

Once you replace the input noise with another image... well.  Strictly speaking, it is probably still a generative adversarial network, if you're still doing everything else the same.  It is still a generator and a discriminator, acting in an adversarial fashion.

But you've radically altered the input distribution, so there is a good chance that you're no longer accomplishing what you want to accomplish unless you're being very careful and clever.

That said, there are GAN variants which do take images rather than noise as inputs.  See the wonderful paper on CycleGANs by Zhu, et al, along with a substantial body of followup literature. And note that CycleGANs use not one, but two discriminators, so even here the discriminator is necessary.


What is an activity vector in capsule neural networks?

In a traditional neural network, the network's vertices are neurons and the output of a single neuron is a single value (a "scalar"). This number is called its activation. A layer of neurons in the network outputs a vector of activations. We should not confuse this with the activity vectors in a Capsule Network.
Capsule Networks are different since the network vertices are Capsules rather than neurons. They are a higher-dimensional: the output of a Capsule is not a scalar but a vector representing a group of parameters related to the input. Hence the name activation vector.
Motivation
In a neural network, there is no inherent structure between the scalar outputs of the neurons, this is something the following layers have to learn. In Capsule Networks the output of a capsule represents all the parameters related to that together in a vector including a prediction for the activation of deeper layer Capsules. This adds a useful local structure.
For example, consider face recognition. If you have a capsule that knows how to recognize eyes it could output an activity vector representing e.g. "since I have recognized an eye position $(x,y)$ with probability $p=0.97$ I predict the parameters for the whole face will be $(f_1,\dots, f_n)$".
As explained in the Dynamic Routing Between Capsules paper you refer to this information is then used in the way that the capsules in earlier layers (the parts: eye, mouth, nose) predict the activations of deeper layers (face). For example, a face recognizer will only be strongly activated when there is an agreement between the eye, nose and mouth recognizers (the parts) and the face recognizer (the whole) about where the face is located (the $(f_1,\dots, f_n)$ parameters).
Historical Inspiration
Older computer vision algorithms like SIFT work in a similar way where recognition is based on agreement between the configuration of multi-dimensional features (key points) and the reference configuration.


Does the image is logistic regression or SVM, and why?

The straight dashed-line shows the typical decision line in logistic regression or any linear classifier. The dashed-circle shows the decision line from SVM.
Obviously, since the data is not linearly separable in the original 2D feature space, if someone makes a higher dimension space by taking into account non-linear interaction of the original 2 features then they can discriminate between x and o data using a linear discriminator applied in higher dimensions. This shows the beauty of kernel methods that can make a linear yet high-dimensional (infinite dimensions indeed) problem from a non-linear low-dimensional problem (finite dimensions actually).


What are sim2sim, sim2real and real2real?

The abbreviations sim2sim, sim2real and real2real refer to techniques that can be used to transfer knowledge from one environment (e.g. in simulation) to another one (e.g. in the real world).

sim2sim stands for simulation-to-simulation,
sim2real stands for simulation-to-real, and
real2real stands for real-to-real.

In sim2sim, knowledge acquired during one simulation is transferred to an agent (or robot) in another simulation. Similarly, in sim2real, knowledge acquired during the simulation is used in a real-world problem (or environment). Finally, in real2real, knowledge acquired in a real-world problem can be transferred to another agent in another real-world problem.

The main challenges are related to the differences that exist between one environment and the other (either in simulation or in the real-world). For example, in sim2real, the simulation is almost never a perfect model of the real-world environment, so an agent trained in a simulation will probably not behave optimally in the real-world environment, which is often a lot more complex than the simulated environment. However, it is often the case that a robot needs to be trained in simulation, given that a robot trained in a real-world environment is subject to crashes.


Should neural nets be deeper the more complex the learning problem is?

Deeper models can have advantages (in certain cases)

Most people will answer "yes" to your question, see e.g. Why are neural networks becoming deeper, but not wider? and Why do deep neural networks work well?.

In fact, there are cases where deep neural networks have certain advantages compared to shallow ones. For example, see the following papers

The Power of Depth for Feedforward Neural Networks (2016) by Ronen Eldan and Ohad Shamir
Benefits of depth in neural networks (2016) by Matus Telgarsky.
Depth-Width Tradeoffs in Approximating Natural Functions with Neural Networks (2017) by Safran and Shamir
Optimal approximation of piecewise smooth functions using deep ReLU neural networks (2018) by Petersen and Voigtlaender

What about the width?

The following papers may be relevant

Wide Residual Networks (2017) by Sergey Zagoruyko and Nikos Komodakis
The Expressive Power of Neural Networks: A View from the Width (2017) by Zhou Lu et al.

Bigger models have bigger capacity but also have disadvantages

Vladimir Vapnik (co-inventor of VC theory and SVMs, and one of the most influential contributors to learning theory), who is not a fan of neural networks, will probably tell you that you should look for the smallest model (set of functions) that is consistent with your data (i.e. an admissible set of functions).

For example, watch this podcast Vladimir Vapnik: Statistical Learning | Artificial Intelligence (AI) Podcast (2018), where he says this. His new learning theory framework based on statistical invariants and predicates can be found in the paper Rethinking statistical learning theory: learning using statistical invariants (2019). You should also read "Learning Has Just Started" – an interview with Prof. Vladimir Vapnik (2014).

Bigger models have a bigger capacity (i.e. a bigger VC dimension), which means that you will more likely overfit the training data, i.e., the model may not really be able to generalize to unseen data. So, in order not to overfit, models with more parameters (and thus capacity) will also require more data. You should also ask yourself why people use regularisation techniques.

In practice, models that achieve state-of-the-art performance can be very deep, but they are also computationally inefficient to train and they require huge amounts of training data (either manually labeled or automatically generated).

Moreover, there are many other technical complications with deeper neural networks, for example, problems such as the vanishing (and exploding) gradient problem.

Complex tasks may not require bigger models

Some people will tell you that you require deep models because, empirically, some deep models have achieved state-of-the-art results, but that's probably because we haven't found cleverer and more efficient ways of solving these problems.

Therefore, I would not say that "complex tasks" (whatever the definition is) necessarily require deeper or, in general, bigger models. While designing our models, it may be a good idea to always keep in mind principles like Occam's razor!

A side note

As a side note, I think that more people should focus more on the mathematical aspects of machine learning, i.e. computational and statistical learning theory. There are too many practitioners, who don't really understand the underlying learning theory, and too few theorists, and the progress could soon stagnate because of a lack of understanding of the underlying mathematical concepts.

To give you a more concrete idea of the current mentality of the deep learning community, in this lesson, a person like Ilya Sutskever, who is considered an "important and leading" researcher in deep learning, talks about NP-complete problems as if he doesn't really know what he's talking about. NP-complete problems aren't just "hard problems". NP-completeness has a very specific definition in computational complexity theory!


What are the state space and the state transition function in AI?

Initial state
How things are at first.
In your particular example, it would be where your k knights are placed on the board initially. Your problem doesn't precisely state this, so you could either place them at the bottom or at random.
Goal state
The board with the k knights placed on the target squares.
State transition function
A function that takes actions (bound presumably by rules) and returns a new state.
In the k knight problem, the legal actions are moving parallel and in L shape movements, after which the knight will be in a new position and the board in a new state.
State space
The set of all states reachable from the initial state by any sequence of actions.
So, in the case of the k knight problem, your state space would start at the top with your initial state followed down by each individual movement of the k knights and the resulting new state. A graph where lines are actions and nodes are new states or a table are common representations of state space.

Reference
Artificial Intelligence: A Modern Approach, by S. Russell and P. Norvig.


Imposing physical constraints (previous knowledge) in a neural network for regression

Some ideas out the top of my head:

In the case of $dy/dx_2>0$ you could compute the gradient using the chain rule and limit the weights so that the constrain holds
In the case of $y + x_5 + x_7 < K$ you could use a clipping function on the output layer?


How do I solve the problem of positioning 11 pieces into a 8x8 puzzle?

Using your app, I was able to find a (spoiler alert!) solution manually. At least now you know your puzzle is solvable and you did not waste your money :)

It seems your app has a bug, though. I was unable to put the last piece, as shown in the picture. I was wondering if your solver, as it stands, will ever find a solution.

Now the idea. It may be useful for your solver.

The board has 8x8=64 squares. Each piece will occupy 5 squares and you want to fit 11 pieces, so the final position will have 9 empty squares. Divide the board in two 8x4 rectangles, left and right. Now, it seems only fair that one rectangle should have 5 empty squares leaving the other with 4; with that in mind I've proceeded to fill the right part first and only then the left. After some trial and error, I got lucky.

I don't know if you can reach all solutions with this method. Notice that, in the solution given above, the bottom rectangle will end up with 6 empty squares.

I don't know how to write an efficient solver either. For starters:

Build up a list of bad configurations: small sets of pieces/positions such that whenever you reach them, you know there is no hope.
At each step, put a piece in such way it minimizes the number of forced empty squares.
A variant of the idea above: divide the board in four 4x4 parts; it seems reasonable each part should have at least one empty square but no so many; so, look for solutions forcing these parts to have (1,2,3,3) or (2,2,2,3) empty squares.
Lookup whether this puzzle is known. Names that come to mind: Martin Gardner, Ian Stewart, Sam Lloyd.

Can't say you will ever be able to see a list of all possible solutions.

Nice puzzle, nice app that one you wrote, I've had a good time. Thank you.


Regression with more than one output, neural network

Welcome to AI-SE! Please use shorter description next time because it makes it hard to understand what you ask :)

I believe you want a neural network that can predict future values of multiple variables given multiple inputs. This belongs to the general time series forecasting problem.

One of the best neural network architectures that can handle this problem is the LSTM Recurrent Neural Network (LSTM-RNN). The reason I say "best" is because of their architecture that allows them to develop memory of what they have seen in the past and use it for future predictions. In other words they can cross-correlate in a linear/nonlinear fashion several past steps of multiple variables to future values of multiple other variables, like a black box.

No other neural network type does that. They are the only type of network that can learn the dynamics of a system.

An extremely useful tutorial for your purposes is this, please take a look.


Are linear approximators better suited to some tasks compared to complex neural net functions?

This is just a case of supervised learning. You are trying to predict $s_{t+1}$ given $s_t$ and $a_t$, so the answer to your question depends on how complex your state dynamics are.
For example, if the state space is really complex, e.g. if your state space is an image and you want to predict the next image given the current image and an action, then linear methods are unlikely to work well.


How can I use feature extraction in CNN with image segmentation?

Feature extraction is a way that people use pretrained model to extract information from input data. For example, image segmentation task may use the VGG network or other image classifying network for feature extraction. The output of the last convolution layer is taken. Then, the features are feed into the untrained network to get outputs. The bottom network for image segmentation usually consists of upsampling and convolutional layers. Then output of size of original image is resulted in teh main network. Hope I can help you


How to transfer declarative knowledge into neural networks

One way to look at intelligence is it's the way to compress the universe. That means we have a short mental representation of meaningful concepts.
For example, if I would say "there is a red swan in your building, it's dangerous and can kill you", you already have concepts of "red", "swan", "danger" and this easy allows you to add the bird to your classifier "should I beware it or not". (and you probably have tried to imagine the red swan by now)
There are similar properties in deep networks when deeper layers could respond to more abstract representation concepts. For example, if you have a convolutional network classifying the faces, the first layers would detect simple shapes like lines and circles, then more complex shapes and then parts of the face like eyes, nose and ears (see example here)
Now, let's imagine you have a classification task "is the thing dangerous or not". You have a trained convolutional model and it has "red" and "swan" abstract features on some level. You could freeze all the preliminary weights and make a single backpropagation with a high learning rate only for the subnetwork which is of interest to you.
That could work in theory, but the key challenge here is that most of the neural network representations are not interpretable at all and you would need to solve this problem before (there is a whole research direction with workshops and conferences on the topic with varying success)


For a simple multi layer perceptron, you can refer to here:
https://www.kaggle.com/fchollet/simple-deep-mlp-with-keras

This is a great resource for kerad multi input label classification. Also, here is a few reminders for implementing such classification model.

One hot encoding

In the sample data you provided, it seems like you are using raw numbers as input. This will create unnecessary complications for the model. A better approach will be to encode them into a vector with 0 in all except one index with 1. The index will be the numerical value you are encoding. For example, 1 will be 01000000... Till end of your range of input, and 2 will be 001000 and so on. If you have decimal values, then this approach isn't for you. However you should still scale down your input to something smaller like 0-1.

output encoding

The output seems to be text, which cannot be done and also complicates the problem. Instead you can also use one hot encoding for the output. For example, let A be 0, B be 1, C be 2... AA be 27.... Until your list ends. And then use one hot encoding to encode them to vector of 0 and 1.

Hope I can help you.


Solving the supervised learning problem of learning $p(y \vert \mathbf{x})$ by using traditional unsupervised technologies to learn $p(\mathbf{x}, y)$

This is the definition of conditional probability + Total probability decomposition formula:

$p(y|x) = \frac{p(y,x}{p(x)} = \frac{p(x,y)}{\sum_{y'}p(x,y')}$.

The idea is to use some unsupervised learning algorithm to learn the distribution $p(x,y)$ for every possible value of $y$, and by using the previous formula you can find $p(y|x)$.


For your first question,take a look at using a skip grab model to find what the abbreviated text is. The skip gram model turns a word into a vector which allows it to be processed by other machine learning algorithms. Or , alternatively you can do some really cool addition and subtraction problems with the resulting vectors. With the skip gram model in your case you could generate a vector for your input and then compare it with other skip gram vectors and once you have found a near perfect match then that is the unabbreviated word.

You could also look at using sparse distributed representations of words to do this. This approach is similar to the skip gram model except that instead of a vector with maybe 500 values a sparse representation may contain thousands of binary digits of which only a couple are positive or 1.

If you would like to look at this approach take a look at cortical.io which has free API that you can use. As for your second question I reccomend that you use a deep neural network in combination with the skip gram model to produce your output.


What is a probability distribution in machine learning?

Random variables
You do not necessarily need to understand the concept of a random variable (r.v.) to understand the concept of a probability distribution, but the concept of a random variable is strictly connected to the concept of a probability distribution (given that each random variable has an associated probability distribution), so, before proceeding, you should get familiar with the concept of an r.v., which is a (measurable) function from the sample space (the set of possible outcomes of an experiment) to a measurable space (you can ignore the definition of a measurable space and assume that the codomain of the random variable is a finite set of numbers).
Probability measure, cdf, pdf and pmf
The expression "probability distribution" can be ambiguous because it can be used to refer to different (even though related) mathematical concepts, such as probability measure, cumulative distribution function (c.d.f.), probability density function (p.d.f.), probability mass function (p.m.f.). If a person uses the expression "probability distribution", he (or she) intentionally (or not) refers to one or more of these mathematical concepts, depending on the context. However, a probability distribution is almost always a synonym for probability measure or c.d.f..
For example, if I say "Consider the Gaussian probability distribution", in that case, I could be referring to either the c.d.f. or the p.d.f. (or both) of the Gaussian distribution. Why couldn't I be referring to the p.m.f. of the Gaussian distribution? Because the Gaussian distribution is a continuous distribution, so it is a distribution associated with a continuous random variable, that is, a random variable that can take on continuous values (e.g. real numbers), so a Gaussian distribution does not have an associated p.m.f. or, in other words, no p.m.f. is defined for the Gaussian distribution. Why don't I simply say "Consider the p.d.f. of the Gaussian distribution." or "Consider a Gaussian p.d.f."? Because it is unnecessarily restrictive, given that, if I say "Consider the Gaussian distribution" I am implicitly also considering a p.d.f. and c.d.f. of the Gaussian distribution.
Similarly, in the case of a discrete distribution, such as the Bernoulli distribution, only the c.d.f. and p.m.f. are defined, so the Bernoulli distribution does not have an associated p.d.f.
However, it is important to recall that both continuous and discrete distributions have an associated c.d.f., so the expression "probability distribution" almost always (implicitly) refers to a c.d.f., which is defined based on a probability measure (as stated above).
Notation
In the same vein, the notation $p(x)$ can be as ambiguous as the expression "probability distribution", given that it can refer to different (but again related) concepts. However, $p(x)$ usually refers to a probability measure (so it refers to a probability distribution, given that a probability distribution is almost always a synonym for probability measure). In this case, assuming for simplicity that the r.v. is discrete, $p(x)$ is a shorthand for $p(X=x)$, which is also written as $\mathbb{P}(X=x)$ or $\operatorname{Pr}(X=x)$, where $X$ is a r.v., $x$ a realization of $X$ (that is, a value that the r.v. $X$ can take) and $X=x$ represents an event. Given that an r.v. is a function, the notation $X=x$ may look a bit weird.
In the case of a discrete r.v., $p(x)$ can also refer to a p.m.f. and it can be defined as $p_X(x) = \mathbb{P}(X=x)$ (I added the subscript $X$ to $p$ to emphasize that this is the p.m.f. of the discrete r.v. $X$). In the case of a continuous r.v., the p.d.f. is often denoted as $f$. In the case of both discrete and continuous r.v.s, the c.d.f is usually denoted with $F$ and it is defined as $F_X(x) = \mathbb{P}(X \leq x)$, where $\mathbb{P}$ is again a probability measure (or probability distribution). The p.d.f. of a continuous r.v. is then defined as the derivative of $F$. At this point, it should be clear why a probability distribution can refer to different but related concepts, but, in any case, it always refers to a probability measure.
Empirical distributions
There are also empirical distributions, which are distributions of the data that you have collected. For example, if you toss a coin 10 times, you will collect the results ("heads" or "tails"). You can count the number of times the coin landed on heads and tails, then you plot these numbers as a histogram, which essentially represents your empirical distribution, where the adjective "empirical" usually refers to the fact that there is an experiment involved.
Multivariate r.v.s and distributions
To complicate things even more, there are also multivariate random variables and probability distributions. However, all the concepts above more or less are also applicable in this case.
Parametrized distributions
A parametrized probability distribution, often denoted by $p_{\theta}$, is
a family of probability distributions (defined by the parameters $\theta$), rather than a single probability distribution. For example, $\mathcal{N}(0, 1)$ refers to a single Gaussian distribution with zero mean and unit variance. However, $\mathcal{N}(\mu, \sigma)$, where $\theta=(\mu,  \sigma)$ is a variable, is a family (or collection) of distributions.
Conclusion
To conclude, it is completely understandable that you are confused, given that the terminology and notation are used inconsistently, and there are several involved concepts, which I have not extensively covered in this answer (for example, I have not mentioned the concept of a probability space). If you get familiar with the concepts of probability measures, random variables, p.m.f., p.d.f., c.d.f., etc., and how they are related, then you will start to get a better feeling of the whole picture.


Why do I get the best policy before Q values converge using DQN?

Even if the mean of the maximum Q-value increases from episode 300 onwards, it doesn't mean that the relative order of the Q-values of the actions that you can take in the states change, which means that the policy may not change, even though the value function changes, assuming you're acting greedily with respect to the value function.
More concretely, suppose that you can take one of two actions $\{ a_1, a_2\} = \mathcal{A}$ in each state $s \in \mathcal{S}$. Let's say that you pick $s_1, s_2$ to calculate the average of the maximum Q-value. Without loss of generality, suppose that the action associated with the highest Q-value in these states is $a_2$. If your policy is the greedy policy with respect to the state-action value function, then your policy will choose $a_2$ in $s_1$ and $s_2$. If all Q-values $\hat{q}(s_1, a_1)$, $\hat{q}(s_1, a_2)$, $\hat{q}(s_2, a_1)$ and $\hat{q}(s_2, a_2)$ increase (or even decrease) but their relative order remains the same (i.e. $\hat{q}(s_1, a_2) > \hat{q}(s_1, a_1)$ and $\hat{q}(s_2, a_2) > \hat{q}(s_2, a_1)$, from episode 300 onwards), the greedy policy also remains the same.
So, I think that what you observed is theoretically possible, although I cannot guarantee that you don't have another issue.


Can AI be understood as a generalized statistics tool?

My understanding is that AI can be understood as a very generalized and abstract statistics software package handling input data in a general way to find the "best fit" to some form of problem. Is that correct? I know it isn't. But is it vaguely correct?

No. It's not correct, in my opinion, not even vaguely and in many ways.

AI is not (necessarily) abstract (although there are examples of theoretical frameworks of AI, such as AIXI, which may be of interest to you, given that you are about to become a theoretical physicist)
AI does not necessarily generalize statistics or statistical concepts
AI is not just machine learning or, more precisely, supervised learning. You're very likely referring to supervised learning when you say "best fit", but there are other forms of learning (such as reinforcement learning) and there are other aspects of AI apart from learning, such as perception, control, etc. Some people sometimes claim that machine learning is glorified statistics, but, although the two are similar and related, I don't think that's exactly correct.

To know what AI is, you should read the book Artificial Intelligence: A Modern Approach. If you want to know what machine learning is, then there are many books that you can read, such as Machine Learning (1997) by Tom M. Mitchell. If you are interested in the relationship between machine learning and statistics, you may be interested in this post. You should also read this answer, which describes what AI is or may refer to. This answer gives a nice brief description of the difference between AI and ML.


Can the law of iterated expectation be used on the inner expectation of the DQN cost function described in the DQN paper

Using the related question and simplifying notation
$$\begin{align} E_{\mu,\pi}[E_{\mathcal{E}}[s'|s,a]^2] &= \sum_\limits{s,a}\left(\sum_\limits{s'}s'p(s'|s,a)\right)^2p(s,a) \\
&=\sum_\limits{s,a}\left(\sum_\limits{s_1'}s_1'p(s'_1|s,a)\cdot\sum_\limits{s_2'}s_2'p(s'_2|s,a)\right)p(s,a) \\
&= \sum_\limits{s,a,s_1',s_2'}s_1's_2'p(s_1'|s,a)p(s_2'|s,a)p(s,a) \\
&= E_{\mathcal{E}\sim (s_1',s_2'),\mu,\pi}[s_1'\cdot s_2']
\end{align}$$
This is the closest I could come up with, basically instead of the same sample squared, we pick two samples independently given the pair $(s,a)$.


Should we use a pre-trained model or a blank model for custom entity training of NER in spacy?

The reason you would load a pre-existing model is that it offers something of value to your task (e.g. named entity recognition for food) and the cost of training it from scratch is not worth it.  For example, to train GPT-3 from scratch would cost several million dollars.  Typically someone will use a model like BERT and fine tune it.  This is called transfer learning.  With spaCy you will typically use en_core_web_sm which was trained on the OntoNotes corpus and includes named entities.  Making a custom food NER using en_core_web_sm should be more accurate than making one from scratch. You should be able to build a good model with and without transfer learning fairly quickly if you have a GPU.


Are there any good alternatives to an LSTM language model for text generation?

The current state of the art in natural language generation are all auto-regressive transformer models. Transformers no longer use recurrent neural networks such as LSTM, because the recurrences makes long dependencies messy to calculate. Instead, Transformers only keep the attention layers, and apply attention on all the existing text so far, which can be done in parallel so therefore very fast, while being able to attend to long dependencies (e.g. understanding that "it" refers to "John" from 3 sentences ago). They are also faster to train than LSTMs (on powerful GPUs at least). The downside is more memory requirement, and you need large models and large datasets (LSTMs work better for small models and small datasets). Here is some background info on how they work.
Auto-regressive transformer models only use the decoder for text generation, and removes the encoder. Given an input, they predict the next word.
The most well-known one is GPT (GPT-3 has 175B parameters; GPT-2 has 1.5B parameters, and GPT-1 has 175M parameters) GPT is developed by OpenAI and is a commercial, paid-software if you want to use their official model, but I'm sure with a little digging you can find community-trained models that will perform slightly worse but is at least free to use. GPT is basically a vanilla transformer, but trained on a huge, huge dataset with a huge, huge model to achieve state-of-the-art performance.
Other auto-regressive transformer models include:

CTRL by Salesforce, which uses the novel idea of control codes to guide the style of generation (e.g. to generate Wikipedia article style text or book review style text).
XLNet by Google AI Brain team, which handles longer sequences more accuractely than the others because it re-introduces reccurrence back in the transformer model, allowing it to remember past sequences. Otherwise, vanilla transformers cannot handle dependencies that crosses sequences (note: a sequence is limited by the max length you can feed into the model, bottle-necked by your memory requirement, and can contain many sentences or paragraphs).
Reformer by Google Research, which is a more efficient transformer that significantly reduces the memory requirement while also being faster to compute on long sequences.

If your goal is just to generate English or another commonly researched language, you can use an existing pre-trained language model and avoid doing any training yourself. This saves a lot of time, and there should at least be free community-trained models readily available. Otherwise, for obscure tasks, you'd have to train one yourself, and these state of the art models will take immense resources.


Translating a single word Neural Networks

The training examples are not mentioned, but, assuming the assignment involves supervised learning, normalization is, in this case, mapping the inputs and outputs to binary values.

Translation, in this simplified context, is a map of strings to strings, which isn't really an AI device by contemporary standards. Since an artificial network is given as a requirement for the solution, the goal is to train the network to act like a hash map.

$$ \Big\{..., \;
\big(
[\text{"beer", "birra", "cervexa"}]:
\text{"pivo"}
\big)
, \; ...\Big\} $$

$$ \Downarrow $$

$$ \Big\{..., \; \big(\text{"beer" : "pivo"}), (\text{"birra" : "pivo"}), (\text{"cervexa" : "pivo"}\big), \; ...\Big\} $$

Assuming there is a list of pairs $(\mathcal{X}, \mathcal{Y})$, where $\mathcal{X}$ is the word in any of the source languages and $\mathcal{Y}$ is the word in the target language, and the vocabulary of both is known to have word counts of $X$ and $Y$ respectively, one can determine the input and output layer widths (cell counts) $w_1$ and $w_{\ell}$, where $\ell$ is the number of network layers.

$$w_1 = \text{ceiling} ( \log_2{X} )$$
 $$w_{\ell} = \text{ceiling} ( \log_2{Y} )$$

The assignment of source words to binary values is arbitrary. You can sort them by language and then by whatever common character encoding used (such as UTF-8) just for debugging purposes. The same is true of target words.

A simple MLP (multi-layer perceptron) with mini-batch training using basic propagation map be appropriate.


What would be the most effective self-learning algorithm for a 7 player social deduction game?

It is very likely that you want an algorithm like Counterfactual Minimax Regret. This algorithm has several variants, but they differ mostly in their efficiency.

CFR is the algorithm that was used to solve 2-Player Poker, although the solution comes from one of its more advanced versions. The algorithm is highly applicable to other games of incomplete information and games with roles, like Secret Hitler. Essentially, it learns how to act so that the actions it takes leak as little information to the other players as possible about its role (or hand of cards), while also learning about the roles of other players from their actions, and maximizing its chances of winning the game.


Is AI capable to replace pilots entirely on the commercial flights?

Self-flight is already running on all commercial airlines, but only at the cruising altitude. When you flight from London to New York, most of the time the pilots are monitoring the autopilot systems. Running AI on cruising altitude is arguably a simper problem than self-driving cars. The system has been in-place for years.

The real danger of airlines is taking off/landing. As you can imagine, the costs for a bad judgement could be quite serious. Anything could happen while the plane is still under the clouds. While our AI technology is improving, it will never replace human intervention, especially when the costs of collision is very high. If you don't believe me, google for how Facebook is hiring humans for reading "fake news". Facebook has publicity acknowledged AI will never be sufficient for them, then why would AI good enough for replacing well trained pilots?

AI on a commercial airplane is already taking place, but there is no possibility ever it will take over the world. We will always need pilots who knows how to work with the system, making important decisions.

Trained pilots + good AI assistance will make the world a better place.


What is the correct way to read and analyse images in machine learning?

If you are using a fully connected network (aka an MLP) and images with one channel (grey scale) and 100 x 100 = 10,000 pixels, then yes, MLP would have 10k inputs and 10k x N 1 trainable weights in the first layer (as noted by Neil Slater). If you have a color image with 3 channels, e.g. RGB, then you can expect 3 times as many weights because there are 3 times as many values used to represent the image.

A convolutional neural network is a common architecture for analyzing images. For a 100x100 (10k) pixel image, the input layer might have 3x3x1x32 = 288 weights (for 1 channel) or 3x3x3x32 = 864 weights in the first layer, much less than 10k x N 1 from a fully connected network. This would transform your image into a 98x98x32 size image. The main point is that you would have 3x3 weights per input channel per output channel at each layer, instead of 10k weights per input channel per output channel. CNNs also give you some invariance properties that are usually nice in machine learning with images.

For images in general, having a lot of weights is normal. ImageNet (linked above) has 60 million weights to train. Typically special hardware, like a GPU is used to handle this many weights. If you are using just a CPU, your model may not train well in any reasonable amount of time, i.e. years.


How to use sigmoid as transfer function when input is not (0,1) range in ANN?

There are several functions that can be denoted as sigmoid functions, such as the logistic function and the hyperbolic tangent, given that they have an $S$-shaped curve. You can find more info about them in the related Wikipedia article.
However, when people use the term sigmoid function, they typically refer to the logistic function, which is a function of the form $$\sigma: \mathbb{R} \rightarrow [0, 1],$$ So, it accepts any real number and produces a number between $0$ and $1$, so you can pass any real number (as input) to the sigmoid.
The standard logistic function $\sigma$ is mathematically defined as follows
$$\sigma(x)=\frac{1}{1+e^{-x}}\in [0, 1]$$
Here is the graph of the logistic function. The $y$-axis contains the output values, i.e. $\sigma(x)$ for any $x \in \mathbb{R}$, while the $x$-axis contains the input values, i.e. $x \in \mathbb{R}$.

That function can be between 0 and 1 but my input values for the network are between -5 and 20. Someone told me that I need to scale the function so that it is in the range of -5 and 20 instead of 0 and 1. Is this true? Why?

No, in principle, you do not have to scale your inputs, for the reasons I have just explained. However, this does not mean that, in practice, it may not be a good idea to standardize or normalize your inputs before feeding them into the neural network. See the vanishing gradient problem (VGP) in this article (by J. Brownlee), or this one (by M. Nielsen), or in this video (by A. Ng). Moreover, you should probably actually use another activation function, such as the ReLU, which does not suffer from the VGP (which, nevertheless, should not occur, if your neural network does not contain more than a couple of hidden layers).


Non-Neural Network algorithms for large state space in zero sum games

I was reading online that tic-tac-toe has a state space of $3^9 = 19,683$. From my basic understanding, this sounds too large to use tabular Q-learning, as the Q table would be huge. Is this correct?

That is a relatively small number of states that can easily be represented in a table on a modern computer. For a Q table, you would multiply by the number of moves possible in each state, but this is still a small amount of memory. Even with the most naive implementation that tracked impossible states and state/action pairs, using string state representations (so 10 bytes for each state/action key), and using double precision floating point for each action value (another 8 bytes), the full table would be around 3 megabytes in size.
So it is definitely possible to use tabular Q learning here. I have done just that whilst learning about RL - my Q-learning Tic Tac Toe agent is written in Python and available on Github. There are many ways to optimise the required space, e.g. only representing reachable states. Also in games with perfect control (moves directly result in a new state), it is common to use afterstate values instead of action values.

If that is the case, can you suggest other (non-NN) algorithms I could use to create a tic-tac-toe agent to play against a human player?

It is not the case, as explained. However, there are two basic approaches at the top level that are worth learning about for working with game trees:

Forward planning or search algorithms, as mentioned by skillsmuggler in their answer. These use game rules plus a heuristic (measure of likely success) to explore the future states of the game and pick the search result with the best heuristic. Minimax with a heuristic only based on win/lose at the end is a very basic approach, but would cope just fine with Tic Tac Toe.

Other planning algorithms include Negamax (a minor variation of Minimax for zero-sum games) and MCTS (famously used in Alpha Go).

Policy function improvement, which generate policies - maps of current state to actions - and have ways to assess them and select better policies.

Q learning, and the many algorithms of reinforcement learning are in this category, as well as genetic algorithms.
Tic Tac Toe is simple enough that you could hard-code a policy function that mapped any state to the next action. The xkcd webcomic encoded this policy into a diagram.

These two approaches are complementary, in that both can be used together to solve more sophisticated problems. For instance, value-based reinforcement learning algorithms - including Q learning - can be used to provide learned heuristics for a search algorithm. The categories I suggest above are not strict either, in that some algorithms are not clearly one thing or another.


How to predict time series with accuracy?

Things like this a really hot topic in research right now, and it's very difficult to get high accuracy on a chaotic system like the stock market. That being said, I would probably recommend preprocessing your data rather than having your primary neural network decide what to accept and what not to.

For example, in your specific case, you could model a bubble bursting as perhaps a negative exponential drop-off or something of the sort. This could include machine learning too. You could gather historical drops in stock market data, and use some sort of regression (Bayesian would probably work well) to estimate the best function to use as an indicator to whether a steep drop has occurred. If so, then use your neural network specifically to classify the fate of the stock. I would think you would have more success following a specialised route such as this rather than trying to train a network on general trends in the market.

In terms of the structure of your neural network, you may want to consider a convolutional neural network (CNN) instead of a recurrent neural network (RNN). RNNs assume the current point in your time-series depends on all previous points, from the beginning of your data. I wouldn't think this would hold true in general for the stock market. The filters a CNN learns are suited to learning to extract certain features and the CNN will apply the filters to specific portions of the data, in the way it considers optimal. They are both nonlinear models, but the CNN will be less computationally costly to train. You could also try a gradient-boosting regression approach instead of a neural network. That being said, something like an LSTM RNN (long short-term memory) won't necessarily be bad - just my two cents.


What techniques to explore for dynamic clustering of documents (emails)?

It sounds like you are trying to do some kind of semi-supervised learning. In semi-supervised learning, some data points are labelled (you know which class they belong to), and others are not. There are classification algorithms designed specifically for this kind of problem, like a transductive-SVM. I personally have not found these techniques to be more effective than simply discarding the unlabelled data and treating my problem as purely supervised, but YMMV.

TFIDF remains fairly popular, as do ngram-based approaches. A more modern vectorization to consider might be word2vec, which translated something like a bag-of-words style vector into a more meaningful feature space for words.


What are the main differences between sparse autoencoders and convolution autoencoders?

Sparse auto-encoders (SAEs) are auto-encoders that impose constraints on the parameters so that they are sparse (i.e. zero or close to zero). This can be achieved in different ways. For example, you can train an auto-encoder with a loss function that includes a penalty term (to constraint the parameters to be close to zero or zero) or you e.g. set the smallest activations to zero.

Convolution auto-encoders (CAEs) are auto-encoders that use the convolution operation. So, they can be viewed as the auto-encoder version of convolutional neural networks. For this reason, they are particularly suited to compress and reconstruct images. The authors of the original paper, Stacked Convolutional Auto-Encoders for Hierarchical Feature Extraction, train them with gradient descent and back-propagation by minimizing the mean squared error, so there's e.g. no penalty term, but you can probably combine SAEs with CAEs.

Of course, you could say that CAEs are sparse with respect to the traditional auto-encoder (in the same way that you can say that CNNs are sparse with respect to fully connected neural networks), so, in this sense, CAEs are also sparse auto-encoders.


Can I limit the possible choices for a computer vision framework to recognize?

I would recommend using a convolutional neural network. CNNs are the bleeding edge for image recognition currently. They should be able to solve your problem with a high accuracy, just maybe without the sort of dialogue that you provided in your question. In order to train a convolutional neural network, you would have to use a different computer than your desktop and "deploy" it to your raspberry pi. This means that your desktop computer would find the optimal wight configurations for your neural network and the images you want to detect, and then you would be able to send the neural network to your raspberry pi in order to be used. If you want to, you could train your entire convolutional neural network from scratch, using random weights to begin, or you could use a process known as transfer learning where you take an existing neural network, usually deep, and train it a little bit further according to your existing data set. In order to train your neural network from scratch, you will need a very large volume of sample images to train it. In order to do transfer learning, you will only need about 1000 images for each of your classes in order to achieve very accurate results for your specific task. For specific libraries that you can use, I would recommend using Matlab if you have access to it as it is extremely easy to train and run deep neural networks, and it is easy to deploy the model to your raspberry pi. You can implement transfer learning in Matlab with 10 lines of code. Next, if you do not have access to Matlab, I would recommend if you have some python experience, using tensorflow. Tensorflow supports a lot of deep learning algorithms and should be able to for your problem although it will not be as easy as using MATLAB. Finally, Microsoft has a toolkit called CNTK which is fairly similar to tensorflow, but instead you can use it in C# or other .net languages.