How will an AI comprehend the ethics of "right" and "wrong"?

Right and wrong only exist relative to some goal or purpose.

To make a machine do more right than wrong, relative to human goals, one should minimize the surface area of the machine's purpose. Doing that minimizes the intrinsic behavior of the AI, which enables us to reason about the right and wrong behaviors of the AI, relative to human purposes.

Horses are quite general over the domains of their purposes, but are still predictable enough for humans to control and benefit from. As such, we will be able to produce machines (conscious or unconscious) that are highly general over particular domains, while still being predictable enough to be useful to humans.

The most efficient machines for most tasks, though, will not need consciousness, nor even the needs that cause survivalistic, adversarial and self-preserving behaviors in eukaryotic cells. Because most of our solutions won't need those purposes to optimize over our problems, we can allow them to be much more predictable.

We will be able to create predictable, efficient AIs over large domains that are able to produce predictable, efficient AIs over more specific domains. We'll be able to reason about the behavioral guarantees and failure modes of those narrow domains.

In the event that we one day desire to build something as unpredictable as a human, like we do when having babies, then we'll probably do that with the similar intentions and care that we use to bring an actual baby into the world. There is simply no purpose in creating a thing more unpredictable than you unless you're gambling on this thing succeeding you in capability - which sounds exactly like having babies.

After that, the best we can do is give it our theories about why we think we should act one way or another.

Now, theoretically, some extremely powerful AI could potentially construct a human-like simulacrum that, in many common situations, seems to act like a human, but that in fact has had all of it's behaviors formally specified a priori, via some developmental simulation, such that we know for a fact that all such behaviors produce no intentional malice or harm. However, if we can formally specify all such behaviors, we wouldn't be using this thing to solve any novel problems, like curing cancer, as the formal specification for curing cancer would already have been pre-computed. If you can formally specify the behaviors of a thing that can discover something new, you can just compute the solution via the specification, without instantiating the behaviors at all!

Once AI has reached a certain level of capability, it won't need to generate consciousnesses to derive optimal solutions. And at that point, the only purpose for an artificial human to exist will be, like us, for its own sake.


Why are the landmark retrieval and facial recognition literature so divergent?

Landmark retrieval has photographs of landmarks that you need to find out. Consider the degrees of freedom for this, the landmarks can many different colours (more than humans' faces) and also the colour range is all over the place (a landmark may be blue or white or red). The shapes of the various landmarks will also vary.
Now, consider face recognition problem. All humans faces look alike morphologically. If you look at the colour, it is not as varied as landmark recognition.
Because of the inherent data in both the problems, the research focuses on the diverging lines of thought. Rich descriptors are good for landmarks because the data itself is very rich and mired with variation. On the other hand, discriminative features are more desirable for face recognition because faces are more similar and less rich in variation, so differentiating between is hard.
It is the requirement of the problem that steers research in diverging directions.


Is the singularity something to be taken seriously?

I can say that among AI researchers I interact with, it far more common to view it as wild speculation than as settled fact.

This is borne out by surveys of AI researchers, with 80% thinking strong forms of AI will emerge in "more than 50 years" or "never", and just a few percent thinking that such forms of AI are "near".

Software Developers are not the same as AI researchers, and I have found the Singularity myth to be much more widespread among developers. It has a nice ring to it: Computers keep getting faster, at some point they'll be faster than brains, at that point we just simulate brains. Soon after, we simulate something better than brains.

I suspect that the reasons AI researchers are less optimistic are rooted in the fact that we still don't have a good understanding of human intelligence, or even enough of an understanding of the brain to simulate it. For example, in the last two weeks we have discovered previously unknown types of brain cells. This gives the (correct) impression that even if we had a fast enough computer, we are not at all close to being able to accurately simulate a human brain. We don't really know what a human brain is.

Even if we did know that, simulations are necessarily lossy. We may not have good simulation techniques. Even if we did have good techniques, we may simulate the brain and discover our simulation does not behave as expected for reasons that we don't understand. This is very probable when simulating new systems. In some sense, proponents of the Singularity resemble people predicting that weather control was near in the 1940s. After all, we could simulate simple weather patterns already then, and generate forecasts that sort of worked. How much more complex could it really be to generate perfect forecasts?


What is the difference between DQN and AlphaGo Zero?

DQN and AlphaZero do not share much in terms of implementation.
However, they are based on the same Reinforcement Learning (RL) theoretical framework. If you understand terms like MDP, reward, return, value, policy, then these are interchangeable between DQN and AlphaZero. When it comes to implementation, and what each part of the system is doing, then this is less interchangeable. For instance two networks you have read about in AlphaZero are the policy network and value network. Whilst double DQN alternates between two value networks.
Probably the best resource that summarises both DQN and AlphaZero, and explains how they extend the basic RL framework in different ways is Sutton & Barto's Reinforcement Learning: An Introduction (second edition) - Chapter 16 sections 5 and 6 cover the designs of DQN Atari, AlphaGo and AlphaZero in some depth.
In brief:
DQN Atari

Is model-free
Uses an action value estimator for $Q(s,a)$ values, based on a Convolutional Neural Network (CNN)
Uses experience replay and temporarily frozen target network to stabilise learning process
Uses a variety of tricks to simplify and standardise the state description and reward structure so that the exact same design and hyperparameters work across multiple games, demonstrating that it is a general learner.

AlphaZero

Is model based (although some of the learning is technically model-free, based on samples of play)
Uses a policy network (estimating $\pi(a|s)$) and a state value network (estimating $V(s)$), based on CNNs. In practice for efficiency the NN for these share many layers and parameters, so how many "networks" there are depends how you want to count them.

The earlier AlphaGo version had 4 separate networks, 3 variations of policy network - used during play at different stages of planning - and one value network.

Is designed around self-play
Uses Monte Carlo Tree Search (MCTS) as part of estimating returns - MCTS is a planning algorithm critical to AlphaZero's success, and there is no equivalent component in DQN


Does eligibility traces and epsilon-greedy do the same task in different ways?

Epsilon-greedy is one method of making an agent explore the state space to ensure that the agent doesn't settle on a sub-optimal policy. By taking random actions, even with a small probability, the agent can get to places in the state space it normally wouldn't see and on the chance that the outcome is better than what it normally would have seen, it can modify its value function (or Q function) for that state (or action) to ensure that that action is taken in the future. An alternative to Epsilon-greedy methods is modifying the policy directly using Policy Gradient methods.
Eligibility traces attempt to solve a different problem. Their function is to keep a short term memory of what states have been recently visited. They unify and generalize TD and Monte Carlo methods producing a family of methods spanning a spectrum that has Monte Carlo methods at one end ($\lambda = 1$) and one-step TD methods at the other ($\lambda = 0$). Using traces, one can keep the features of previous states around, but at faded values depending on the choice of $\lambda$. Choosing a low $\gamma$ (near $0$) makes the traces myopic in that the traces quickly go to $0$. If a larger $\lambda$ is used (near $1$) the traces stay around for much longer and can help an agent determine the difference between what would otherwise be 2 identical states.
For more information, check out Reinforcement Learning: An Introduction, chapter 2 (page 30 introduces $\epsilon$-greedy methods) and chapter 12 (eligibility traces).


Why does Q-learning converge to the optimal policy, even if the agent acts sub-optimally?

Q-learning is an off-policy learning algorithm. We are following the behaviour policy, $b$, which is $\epsilon-$greedy. This behaviour policy need not be an optimal policy rather it is a more explorable policy. But we are learning the target policy, $\pi$, which is argmax of state action value $(Q(s,a))$. This target policy is by definition optimal policy.
From the $\epsilon$-greedy policy improvement theorem we can show that for any $\epsilon$-greedy policy (I think you are referring to this as a non-optimal policy) we are still making progress towards the optimal policy and when $\pi^{'}$ = $\pi$ that is our optimal policy (Rich Sutton's book Chapter-5). Here $\pi^{'}$ is the new policy and $\pi$ is the previous policy.

Think of this diagram, where we are selecting action based on $\epsilon$-greedy policy but still making progress towards the optimal policy $\pi_*$.


How can I teach a computer to play N64 games using Neural Nets?

The common options are:

Implement a simple simulation of the game, perhaps without graphics. Train the agent on the simulation. This is usually not very satisfying, because the agent won't play the real game.
Use an emulator that allows you to inspect the game's memory, and input values directly at the controls. BizHawk is a good choice for SNES and I think also for N64 emulation. Use a pipe to connect this program with your agent. Pass across the pipe any key values from the game's memory that represent the state of the game, and send back actions that represent the inputs the agent makes in that state. Usually this is the best option.
Scrape the screen in real time. Tools like SkyScraper will allow you to capture the raw pixels displayed in a window. You should then be able to pass these pixels to your agents via a pipe. You may be able to use tools like Selenium to pass values back from the agent to the game. This route is usually a lot more intensive than option 2, but is the most realistic. It may also require very good hardware to keep up with the game in real time.


As argued by Selvaraju et al., there are three stages of AI evolution, in all of which interpretability is helpful.

In the early stages of AI development, when AI is weaker than human performance, transparency can help us build better models. It can give a better understanding of how a model works and helps us answer several key questions. For example why a model works in some cases and doesn't in others, why some examples confuse the model more than others, why these types of models work and the others don't, etc.
When AI is on par with human performance and ML models are starting to be deployed in several industries, it can help build trust for these models. I'll elaborate a bit on this later, because I think that it is the most important reason.
When AI significantly outperforms humans (e.g. AI playing chess or Go), it can help with machine teaching (i.e. learning from the machine on how to improve human performance on that specific task).

Why is trust so important?

First, let me give you a couple of examples of industries where trust is paramount:

In healthcare, imagine a Deep Neural Net performing diagnosis for a specific disease. A classic black box NN would just output a binary "yes" or "no". Even if it could outperform humans in sheer predictability, it would be utterly useless in practice. What if the doctor disagreed with the model's assessment, shouldn't he know why the model made that prediction; maybe it saw something the doctor missed. Furthermore, if it made a misdiagnosis (e.g. a sick person was classified as healthy and didn't get the proper treatment), who would take responsibility: the model's user? the hospital? the company that designed the model? The legal framework surrounding this is a bit blurry.
Another example are self-driving cars. The same questions arise: if a car crashes whose fault is it: the driver's? the car manufacturer's? the company that designed the AI? Legal accountability, is key for the development of this industry.

In fact, this lack of trust, has according to many hindered the adoption of AI in many fields (sources: 1, 2, 3). While there is a running hypothesis that with more transparent, interpretable or explainable systems users will be better equipped to understand and therefore trust the intelligent agents (sources: 1, 2, 3).

In several real world applications you can't just say "it works 94% of the time". You might also need to provide a justification...

Government regulations

Several governments are slowly proceeding to regulate AI and transparency seems to be at the center of all of this.

The first to move in this direction is the EU, which has set several guidelines where they state that AI should be transparent (sources: 1, 2, 3). For instance the GDPR states that if a person's data has been subject to "automated decision-making" or "profiling" systems, then he has a right to access

  "meaningful information about the logic involved"

(Article 15, EU GDPR)

Now this is a bit blurry, but there is clearly the intent of requiring some form of explainability from these systems. The general idea the EU is trying to pass is that "if
 you have an automated decision-making system affecting people's lives then they have a right to know why a certain decision has been made." For example a bank has an AI accepting and declining loan applications, then the applicants have a right to know why their application was rejected.

To sum up...

Explainable AIs are necessary because:

It gives us a better understanding, which helps us improve them.
In some cases we can learn from AI how to make better decisions in some tasks.
It helps users trust AI, which which leads to a wider adoption of AI.
Deployed AIs in the (not to distant) future might be required to be more "transparent".


Why is the update in-place faster than the out-of-place one in dynamic programming?

When you make updates in-place, then some of the entries in state_values[next_i, next_j] that you are referencing here
value += ACTION_PROB * (reward + discount * state_values[next_i, next_j])

will already be updated earlier in the same loop. Which means you get to use the latest and likely more accurate values earlier.
The strength of this effect varies depending on the order that states and actions are visited. If you can manage to visit them in reverse order that they would appear in natural trajectories, then the speed improvement will be very noticeable.


Can a neural network learn to avoid wrong decisions using backpropagation?

What you are describing is conceptually close to adversarial training. you should read more on adversarial examples and generative adversarial networks for more information.

The idea is that there is a discriminator network, whose job is to correctly discriminate between positive and negative examples. We also have a generative network, that learns to produce "adversarial examples" that "confuses" the discriminator network. By training these two networks side by side, both networks get better at their task. But it's usually the generator network that people are more interested in.

Intuitively, the naive implementation of the method you've described (gradient ascent on incorrect examples from a network in a clean/randomly-initialized state) shouldn't work. This is because negative examples don't form a "natural class" (all triangles have 3 edges, all things that are not triangles however....)


Classifying non-labeled data with high dimensionality

I would recommend to have a look at Finding Groups in Data, which is a very readable introduction to clustering methods. It gives a good overview over a number of different algorithms, both agglomerative and hierarchical. As far as I remember, source code for the various algorithms is available on the web somewhere.

I am sure you will find a fitting algorithm for your problem in there.


Studying the speech-generation model and have question about the confusing nature of model input and outputs

There are two "inputs" into Wavenet:

the previously generated samples of the waveform, which are usually encoded into multiple channels, like into 256 channels using 8-bit mu-law encoding
local conditioning, which can be things like linguistic features such as phoneme classes (used in the original wavenet paper), or frequencies like mel spectrogram values (used in the Tacotron 2 paper)

The local conditioning signal usually has a much lower resolution than the waveform itself. For example, the Tacotron 2 paper mentioned using "50 ms frame size, 12.5 ms frame hop" to derive its mel spectogram values. At 24 kHz, each waveform sample has a duration of just 0.0416 ms. So in order to using the spectogram information to condition the waveform generation, it would have to be upsampled to spread out along the time dimension. (If you were locally conditioning using letters, you might turn "dog" into "dddddoooooooooggg" in order to use the letter "d" to generate 5 samples in the output waveform, etc.).

What is the input to the WaveNet, isn't this a mel-spectrum input and not just 1 floating point value for raw audio?

In the implementation you linked to, it looks like variables that start with "c" refer to "conditioning" signals. So "hparams.cin_channels" indicates how many channels the input conditioning signal has. If that signal is mel spectogram with 80 channels, then it would be set to 80.
It looks like the "inference/input_convolution" layer is processing the actual waveform, not the spectogram conditioning signal.

Is there reason for upsampling stride values to be [11, 25], like are the specific numbers 11 and 25 special or relevant in affecting other shapes/dimensions?

I suspect (though I'm not too familiar with tensorflow conventions) that the second dimension is time, which is why you're seeing the model upsample the conditioning signal in that direction. Those values may be required to match the resolution of the waveform being generated.

Why is the input-channels in residual_block_causal_conv 128 and residual_block_cin_conv 80? What exactly is their inputs?

The "c" in "cin_conv" appears to mean that it is for the conditioning signal, which is why is has a different dimension than the waveform's embedding size.
I hope this helps. You could also try opening up an issue on that github repo to try getting help directly from the author.


What is the difference between abstract, autonomous and virtual intelligent agents?

I don't think there are many contexts where there is any really meaningful distinction between these terms.  Even in the WP article you refer to, it is shown that "abstract intelligent agent" and "autonomous intelligent agent" are generally just synonyms for "intelligent agent" but used to highlight certain aspects of intelligent agents in some contexts.   Net-net, I'd say there just isn't any difference there that's going to matter in practice.

"Virtual intelligent agent" OTOH, used in the context you used it, suggests the distinction between an IA that's implemented in software only, versus one that has a physical manifestation.   I don't know how useful that distinction is and I haven't seen anybody else make it.

All in all, I expect that in almost every possible context, if you just say "Intelligent Agent" with no qualifiers, that's going to be sufficient.   But if there were going to be an exception, I'd say it would be around the term "autonomous" since an agent which is truly autonomous, versus one that needs to operate in a specific, constrained environment, is a distinction that - at least in principle - could be useful.


What, exactly, does the REINFORCE update equation mean?

The loss function you are looking for is cross entropy loss. The 'label' that you use is the action you took at the time point you are updating for.


Should I use U-net to label keys in a keyboard image?

If you just need to draw a rectangle around each key, this is an object detection or template matching problem, so you can use any of the available models for object detection (e.g. YOLO) or any technique for multi-template template matching (e.g. you can use sequential RANSAC or t-linkage). In the first case, you will need a labeled dataset, while, in the second case, you will need the original image and the templates (in your case, a template would be an image of a key).
So, no, this is not a segmentation problem (which would be the task of classifying each pixel in the objects of interest, and not just locating the objects).


How do AI researchers imagine higher dimensions?

The most I can visualize or perceive are 4 dimensions. Yes, 4, because I can also watch videos (which have 3 spatial dimensions and 1 temporal one). Remember Einstein's spacetime?
When dealing with $n$-dimensional spaces, for $n > 4$, I simply do not care about visualizing them in my head, but, as someone suggests, we can think of them as "degrees of freedom". Maybe something like a tesseract may be interesting to you, but that's not really useful to me, to be honest.
When dealing with the math that involves $n$-dimensional spaces or objects, you often do not have to visualize anything, but just have to apply the rules. For example, if you are multiplying multi-dimensional arrays, you just need to make sure that the external dimensions match, and stuff like that.
There are cases, when dealing e.g. with TensorFlow's tensors, where you can imagine that there are matrices for each of the elements at that coordinate of the tensor, but that's not very common.
In case you really want to visualize $n$-dimensional objects, you could first project them to $2$ or $3$ dimensions with some dimensionality reduction technique (e.g. t-SNE).


for this kind of ml training, you will need a ton of data first, at least in the thousands. If you have a bot program that fetches those data for you, AI is the way to go. I'm not sure how else you would do it though.

To train the nn you will need the inputs(the post) and the targets(the rating you want it to output). The targets could be anything you want, like the ratio of likes to views, etc.

There are tons of ML libraries out there and I recommend keras as it is easy to learn for beginners, hope it helps :)


Reinforce Learning: Do I have to ignore hyper parameter(?) after training done in Q-learning?

Epsilon (your rar parameter) is used to have the possibility of a random action which handles the problem of exploration and exploitation.

If you have a deterministic world and are certain that your learner has learned a policy with performance to your liking, removing it will ensure that only the greedy policy being learned will be taken and that you can expect the same performance afterwards.

The agent will still be learning at this point because alpha (learning rate) is not set to 0. Because of this, if the world is stochastic instead and possibly changes slowly over time (e.g. motor speed due to battery power), then having epsilon non zero still allows the agent to explore and overcome these changes when they happen before it's performance gets low enough to change is behaviour (e.g. q values dropping).

If instead you simply want to test the performance of the policy learned up until a timestep, setting alpha to 0 in a compares the epsilon-greedy policy, setting both alpha and epsilon to 0 compares the greedy policy.


A model for each sub-problem vs one model for the whole problem

The tendency in literature in the last years (at least for computer vision problems) seems to point towards the single model option (I'll try to remember to come back and add some links to papers mentioning this when I find them), although this IMO is really data- and problem-dependent.

In your case, I would set up a network for the mapping $x$ to $g(x)$, with a training-only auxiliary loss calculated on the mapping $x$ to $f1(x)$ and compare this with a model trained only on "$x$ to $g(x)$".


Neural network for pattern recognition in audio

I don't know for sure if I should use an RNN, CNN OR DNN. Which one is best for this situation?

This question or variations of of crop up a lot in DataScience stack exchange too. To paraphrase:

  I am trying to do something with X data, and I have a lot of choices for the model. Which is best?

Unfortunately, the answer is generally:

It depends on all the fine details of your project
Unless someone has done almost your exact project recently (so they were using latest techniques of the model type), then no-one knows a priori which model will get the best result.
Optimising machine learning is very much an empirical subject. If you want to know whether A is better than B, you have to try both and measure their performance.

In your case, I think that CNNs and RNNs are both applicable (and you might want to look at a WaveNet-like architecture, which is a variant of CNN, but that could be a bit too advanced to start with). You might have a slight preference for RNN as a starting point if the sequence length to process varies significantly, such that padding input to a CNN would be inefficient. You may also prefer an RNN if the output of your model needs to be a sequence, and doubly so if the output sequence varies in length and is not directly related to the input sequence length (think of natural language translation).


Artificial Intelligence and Automatic Scheduling

Scheduling can get quite complicated. It is fundamentally programming to find a solution given constraints. One approach would be to use MiniZinc, and here is a scheduling example. You can use other plain programming approaches such as Python and so on, but Minizinc is highly optimized and might well reward time invested in learning by finding a better solution more quickly.


Can a brain be intelligent without a body?

No. Don't forget that the main function of the brain is to provide homeostasis between the body and the environment. Without the body, the utility of the brain is no longer relevant.

Alternatively, why consider intelligence only in the brain? How far does our body extend? Embodied cognitive science asks us to consider our entire body and its
 surrounding extensions utility on the intelligent faculty; The fact that our thumbs stick out in a direction different to our other four fingers allows us to grab things in a effortless way is itself, intelligent.

From this understanding then, the segregation between the intelligent faculty and the "non-intelligent" seems to be murky at best. We'd might as well not consider it unless there is better motivation.

  Does intelligence require a context?

Pragmatically speaking, yes. Intelligent behaviour is typically understood as being goal-directed and intentional. Intentionality implies some sort of agency. It being an agent, implies some sort of agent/environment relationship, which implies an environment to act as context.

On the other hand, Karl Friston notes in his review that a general principle that the brain (and thus one would imagine, intelligent behaviour) entails is reduction of thermodynamic free energy while maintaining homeostasis with its environment.

This hints at a promise to describe intelligent behaviour purely in terms of thermodynamic processes, but interpretively what this means in a general language is still unclear.

Also, keep in mind that there is no guarantee that we'd recognize a agent as intelligent if its construction is radically different from ours.


How do layers in an artificial neural network transform inputs to outputs?

The basic calculation for a single neuron is of the form

$$\sigma\left(\sum_{i} x_i  w_i \right),$$

where $x_i$ is the input to the neuron $w_i$ are the neuron-specific weights for every single input and $\sigma$ is the pre-specified activation function. In your terms, and disregarding the activation function, the calculation would turn out to be

$$c\,a_c + d\,a_d + b$$

Note, that the bias term $b$ is just a weight that gets multiplied by the input $1$, thus it appears to have no input.

If you want to develop a further understanding for this, you should try to get familiar with matrix and vector notations and the basic linear algebra that underlies the feed-forward neural networks. If you do, an entire layer of neurons on a whole batch of data will suddenly simply look like this:

$$\sigma(WX)$$

and a FFNN with say 3 layers will look like this:

$$\sigma_{3}(W_3\sigma_2(W_2\sigma_1(W_1X)))$$


How to calculate the false positives and negatives?

Yes, you can use sklearn's confusion_matrix. To explicitly extract the false positives and negatives, you can do

from sklearn.metrics import confusion_matrix

y_true = [0, 1, 0, 1]
y_pred = [1, 1, 1, 0]
tn, fp, fn, tp = confusion_matrix(y_true, y_pred).ravel()


The main point in GPT-3 and already in 2 was the observation that performance was steadily increasing with increasing model size (As seen in Figure 1.2 in your linked paper). So it seems that while all progress made in NLP was definitely useful, it also is important to just scale up the model size.
This may not seem like surprising point, but it actually kind of is. Normally, performance would saturate or at least the gain would slope off, but this is not the case! So the main innovation may not be that big and is kind off brute-force but the point still stands: Bigger models are better.
Another point to mention is the way they did the training. Such a large model needs some tricks to be actually trained (and fast at that). You also want to make use of multiple GPUs for parallel training. This means they also had to develop new structures for training.
Why exactly it is predicted as a huge innovation may only be contained to some twitter demonstration, there are no real sources on this as far as I know. Especially because the model is not openly available.


The term classical AI refers to the concept of intelligence that was broadly accepted after the Dartmouth Conference and basically refers to a kind of intelligence that is strongly symbolic and oriented to logic and language processing. One basic point is the duality body vs. mind. It's in this period that the mind starts to be compared with computer software.
Two classical historical examples of this conception of intelligence

Deep Blue, whose aim in life was to be the master of chess, ruling over the (not-so) intelligent mankind

Eliza a computer-based therapist that turned out to trigger a critic to the classical AI

Two technical examples of classical AI

Expert systems, which are computer programs that strongly rely on the type of constraints and conclusions that you refer to, in order to accomplish feats of apparent intelligence

Fuzzy logic, which is an extension of multivalued logic, but with continuous values instead of discrete ones

Note that in all cases the hardware (once compared with the body) does not play any role: Intelligence is abstract and independent from the material world.


How does one characterize a neural network with threshold-based activation functions?

The model you describe is a kind of a leaky integrate-and-fire (LIF) neuron (see p. 7). It is leaky because the membrane potential decreases steadily in the absence of input. In contrast, in the simple integrate-and-fire (IF) model the membrane potential is retained indefinitely until the neuron spikes, at which point it is reset to 0. However, LIF neurons are usually modelled with exponential decay of the membrane potential, where you have a time constant $\tau$ and you compute the potential $P_{t}$ at time $t$ based on the potential $P_{t_{last}}$ at time when the last input arrived as

$P_{t} = P_{t_{last}} exp(- \frac{t - t_{last}}{\tau})$

This is the same formula as radioactive decay (see here for more details). The idea is that this model is inherently 'aware' of time, whereas the IF model (and your design above) do not factor in the timing of the spikes, so they act like a classical neural network activation. In any case, whether or not a neuron would fire does depend on the firing threshold, so I think that treating the threshold as a learnable parameter is justified - you just have to decide what rules to use for updating it.

Based on what you describe as your understanding of spiking neural networks, it seems that you have been reading about the Hodgkin-Huxley (HH) model (also in that paper I linked to). (Please correct me if I'm wrong.) You are correct in thinking that spikes in the brain are not infinitely narrow like a delta function but more like a very sharp sinusoidal signal, and the HH model faithfully reproduces that. However, the reason why the HH model is not actually used for simulations is that it is computationally very taxing. In practice, in most cases we do not actually care about the state of the neuron between inputs, as long as your model accurately describes the neuron state and what happens to it when an input arrives.

There are other models that approximate the HH model very closely but are much faster to simulate (like the Izhikevich model). However, the LIF model is very fast and sufficient in most cases.

Hope this helps!


Why is the space-complexity of greedy best-first search is $\mathcal{O}(b^m)$?

I was struggling with the same question. This is what I came up with after thinking it through.
With depth-first-search, you backtrack to a node that is a non-expanded child of your parent (or the parent of the parent when your parent has no more non-expanded children (and so on going up the tree)). So the space complexity is limited by your ancestors and the children of these ancestors. Which translates in m*b where m is the max path length (so max number of ancestors) and b is the branching factor (number of children per ancestor).
With greedy search when you backtrack you can jump to any evaluated but unexpanded node, you passed going down on paths earlier. So the algorithm, when backtracking, can make pretty random jumps throughout the tree leaving lots of sibling nodes unexpanded. You will have to remember the value of the evaluation function for all these unexpanded nodes though because possibly they are next up when backtracking occurs. So in a theoretical very worst-case scenario that could mean that almost the whole tree needs to be remembered. Hence O(b^m).
I know there are still gaps in the reasoning but intuitively this makes me understand it best.


Why is the target $r + \gamma \max_{a'} Q(s', a'; \theta_i^-)$ in the loss function of the DQN architecture?

This is the problem that reinforcement learning (RL) is trying to solve: What is the best way to behave when we don’t know what the right action is and only have a scalar (the reward (r) is a scalar) reward of how well we have done?

RL approaches this problem by utilizing temporal difference learning and makes predictions based on the previous experience. An RL agent is trying to maximize the sum of future discounted rewards, called the return.

The term $r + \gamma \max_{a'} Q(s', a'; \theta_i^-)$ is essentially saying "the reward I just saw + $\gamma$ * (my prediction of the return given that I take what I think is the best action in the current state and follow my policy from then on)".

By updating the Q function, the agent can better predict the consequences of its actions and can then choose the best action with greater probability.

The $\gamma$ (gamma) helps to balance between the immediate reward and future rewards. A $\gamma = 0$ makes the immediate reward the only thing that matters but usually the reward for a good action is delayed so values of gamma which put a higher importance on later rewards ($\gamma = .8, .9, .99$, etc) are used.


Suppose you are dealing with a classification problem you can simply define

def error_rate_classification(p, t):
    return np.mean(p != t)

where p and t stand for predictions and targets resp.

If you deal with a regression problem the you should modify a bit this function. For example :

def error_rate_regression(p, t):
    return np.mean(np.abs(p-t) < 1e-2) # the criteria is not general

Note that for regression problem assessing the accuracy of the prediction is not as simple as it is in classification problem

Edit

In regression case, you can use the R2 regression standard metrics. Suppose yi is the ground truth value of the i-th element of your test (or validation) set and f(xi) is your i-th input prediction. The R2 is then

R2 = 1 - sum( (yi - f(xi))**2 )  / sum( (yi - y_mean)**2 )

where y_mean is the mean of the ground truth values of the test ( or validation) test.  You can find other examples in the scikit-learn regression metrics here


What's the term for death by dissolving in AI?

I find the concept of the a Turing machine useful. In one dimension, everything is a string. All of the parts that are "not you" are merely a substrate, a medium for the program your_mind runs on top of. The you, your identity, the "metaphysical" component we think of as the mind, is a result of running the algorithm that is your_mind on the bioware of your body, or, the hardware (technically, "wetware".) So what we're really talking about is the software, and in that light I might use:

Translation

because the software is being translated to execute in a new environment, or

Migration

as in moving software from one system to another.

Philip Dick wrote a philosophical narrative, not technically sci-fi, called The Transmigration of Timothy Archer which is about identity moving between bodies. In a rare departure from his usual work about AI and the effects of a technological society on the human spirit, this book looks at the question of identity in the context of the soul, which opens up all kinds of philosophical questions surrounding the type of technology we're speculating on, particularly in relation to the self.

I value artistic insight, and Phillip K. is considered quite prescient, so perhaps

Transmigration

is most appropriate, as it carries both metaphysical and information technology meanings.

Re death by dissolution, it's worthwhile to look at the etymology of dissolve:

  late 14c. (transitive and intransitive) "to break up" (of material substances), from Latin dissolvere "to loosen up, break apart," from dis- "apart" (see dis-) + solvere "to loosen, untie," from PIE *se-lu-, from reflexive pronoun *s(w)e- (see idiom) + root *leu- "to loosen, divide, cut apart." Meaning "to disband" (an assembly) is early 15c. Related: Dissolved; dissolving.

I think you're on the right track with dilution, certainly per the modern usage, but it may be possible to get more precise.

It's not about breaking up, or washing away (except metaphorically); rather, it's about minimization as in the diminishment of the original software kernel (the self) in relation to an expanding algorithm.

  early 15c., from merger of two obsolete verbs, diminue and minish. Diminue is from Old French diminuer "make small," from Latin diminuere "break into small pieces," variant of deminuere "lessen, diminish," from de- "completely" + minuere "make small" (from PIE root *mei- (2) "small").

The Old French diminuer is apt, as is the Latin deminuere and deminuo, which also carries a meaning of "civil death" and "abatement". Abatement carries a meaning of mitigation, which can be defined fundamentally as "lessening of effect," in this case of the kernel of the original self in relation to the new aggregate.

It may be useful to think of it as a ratio 1/ℵ, with the self as the 1. Important to note the ratio is not literal--each number represent an aggregation of functions we call programs.

That's how I'd think of it mathematically, but metaphysically, Nat nails it by referencing Jung and the death of the ego.

Ego comes from the Latin noun for the self (I, me) which can also be plural (we, us).  It's also fun to note that the Latin verb of being is sum, because in English, sum has a mathematical meaning of an aggregate considered as a whole.

Because we're talking about what it means to be a person, but we also want to connote a function (a process of relative minimization) I am thinking:

Depersonalization

  (noun) the action of divesting someone or something of human characteristics or individuality.SOURCE: Google

The fun part is that there's a legal definition of person:

  In legal use, "corporate body or corporation having legal rights," 15c., short for person aggregate (c. 1400), person corporate (mid-15c.)

Which I think applies (loosely) to the aggregation of function that comprises the program we call the self, and the greater aggregation of functions that form a new self.

Because here we're talking about the loss of the original individual, depersonalization as opposed to repersonalization as a corporate entity.

Alternately, I might propose:

Deindividualization

with the definition "a loss in individual identity within a group" (also known as deindividuation).

(Charles Stross wrote about this in Accellerando, where a character chooses to be subsumed by a group intelligence as the only means of escaping a swarm of autonomous lawsuits;)


Is there a way to translate the concept of batch size into reinforcement learning?

Potentially.

If you do offline reinforcement learning, you're basically learning to approximate a function by sampling input/output pairs, rather than episode-by-episode. Here, your batch size could be set exactly as in an ordinary supervised learning problem.

If you do online learning, then it's not clear to me that the techniques used to set the learning rate in supervised learning can be directly applied though.

Both approaches are well covered in the RL chapter of Russell & Norvig (17? 18?).


Why do we use $X_{I_t,t}$ and $v_{I_t}$ to denote the reward received and the at time step $t$ and the distribution of the chosen arm $I_t$?

Isn't the distribution independent of the time the arm $i$ was chosen?

Yes, but you don't know which arm was chosen at time $t$, that is what $I_t$ represents. $v_i$ would represent the $i$th arms distribution, whereas you want the distribution of the arm that was chosen at time $t$, which is $v_{I_t}$.
$X_{I_t,t}$ is used to represent the arm you chose at time $t$ and the time you chose it. Imagine if we chose the following arms at respective time steps {1,2,5,2,4}, then the reward you get at time 3 (assuming in my example time started at 1) would be $X_{5,3}$. You need this notation because every time you pull arm $i$ you get a different reward, because the reward is a random variable (unless the assumption was a deterministic reward but that would not be interesting).
It has been a while since I read the papers but I assume that the distributions of the arms are stationary, however this notation is more general and would allow for non-stationary distributions.


Minimum number of perceptrons for an n-bit truth table?

This is a very dicey question. Logic functions can be thought of as mapping multiple inputs to a single output. Now each logic function create its own boundary. So if you are using a complex logical equation it is actually very hard to approximate the underlying function. Here I am treating the input Booleans as the input features.

From practical experience: I had a n-bit = 8-bit input, and it mapped to a single bit output. I used a 2 layer Neural Net and then added a final layer of single node (to have a 0 or 1 output). I was using the sigmoid activation function along with normal back-propagation learning.

Now, I varied the hidden layer nodes from 64 to 1024. The cost decreased, indicative of correct learning but the accuracy did not change, howsoever I tried (changing learning rates, using momentum, etc). It was either giving all 1's or all 0's, even though I was using the same set for training and validation. The reason I hypothesized for this behavior was:

The irrelevant input features are affecting the final output. Even after extensive training and testing on the same set with a huge number of hidden nodes.
Due to the 1 and 0 nature of the input the NN was not able to create a correct boundary. Since if it created an equivalent feature of x^3 using its nodes, it'll just output the same constant value again and again f(0) and f(1), where it should have been a nice little curve

So in short it might or might not work, you never can tell!


My suggestion is to go with 1st option. reason is you will get to know much about data and initial stage will find some challenges in developing the model, over a period of time you will get to better results after hypertunning. Please go through article , ignore you have already read this article


What is the most computationally efficient genetic algorithm?

First of all, for a lot of realistic problems, the fitness function evaluation is usually orders of magnitude greater in complexity than the rest of the genetic algorithm. This is not always true, but often is true (e.g. imagine trying to optimise a simulation where you need to execute the simulation completely to obtain the fitness). So optimising the GA itself might only be helpful when the fitness function is lightweight (e.g. it's a mathematical function as used in some competitions).
While your diagram shows a lot of operators, it doesn't show variations of the GA itself. Here are two:

generational: selects a new population at each iteration after performing mutation and crossover on the existing population to generate offspring (which are usually more in number than the population size)
steady state: maintains a single population and replaces only individuals during each iteration with their offspring according to selection/replacement rules, so uses less memory than a traditional GA

If you really wanted to maximise efficiency in the GA, a steady state approach would probably work best combined with tournament selection, as tournament selection does not require any sorting. The mutation/crossover operators you list are likely to all be quite efficient, but simpler methods are likely to to be the most efficient (e.g. bit flip mutation+1 point crossover). For a realistic problem, however, problem-specific operators usually work best.


How do you build a language model to predict the contextual similarity between two documents?

You can use an autoencoder for text. For example, you can refer to this example here: https://machinelearningmastery.com/lstm-autoencoders/

For comparing the contextual similarity, you can compare the encoded vectors for the distance maybe through a formula like mean squared error.

This works as the autoencoder compresses the input data into a vector of numbers, forcing the encoder and the decoder to learn specific features about text. The features cannot be understood by humans but have meanings.

Another approach will be using some word or document embedding like word2vec or GloVe. IT may also work well depending on your dataset size. You need to experiment through different methods to find out which is the best.

A supervised method can also be used if you have labels. For example, you can use a LSTM and train it like an Siamese network for LSTM using triplet loss. Here is an example: https://medium.com/@gautam.karmakar/manhattan-lstm-model-for-text-similarity-2351f80d72f1 Implementation: https://github.com/GKarmakar/deep-siamese-text-similarity

Hope this can help you and have a nice day


What's the difference between biological and artificial evolution?

Biological and artificial evolution work around pretty much the same principles.

Fitness and selection: In biology, the fittest organisms in an ecosystem are more likely to survive long enough to reproduce, passing on their genes in the process. In artificial evolution, our organisms are in fact solutions to our problem, which can be evaluated to determine how good they are (their fitness). We choose ourselves which solutions will be selected for reproduction (there are many ways to do this selection, but what is common among all of them is that the fittest solutions have a higher chance of being selected).

Crossover: In biology, an organism inherits a portion of each parent's genes, so is a sort of genetic hybrid of both parents. For artificial evolution, a new solution (a "child" solution) will inherit part of its parent's solutions (we take a partial solution from each parent, and glue those partial solutions together to construct a new solution).

Mutation: In nature mutations often occur at birth and this is why there are many different species. Harmful mutations make the individual less likely to survive long enough to pass them on to children, and in contrast helpful mutations make it more likely that the individual will survive long enough to pass them unto children. The same can be said for artificial evolution: A mutation randomly changes a small part of the solution, and if it makes that solution fitter, then that solution has a higher chance of being selected for reproduction.


$f(x) = x^2 + b$ is a polynomial (more precisely, a parabola) so it is continuous, thus, a neural network (with at least one hidden layer) should be able to approximate that function (given the universal approximation theorem).

After a very quick look at your code, I noticed you aren't using an activation function for your dense layers (i.e. your activation function is None). Try to use e.g. ReLU.


How do we express $q_\pi(s,a)$ as a function of $p(s',r|s,a)$ and $v_\pi(s)$?

Not quite. You are missing the reward at time step $t+1$.

The definition you are looking for is (leaving out the $\pi$ subscripts for ease of notation)

$$q(s,a) = \mathbb{E}[R_{t+1} + \gamma v(s') | S_t=s,A_t=a] = \sum_{r,s'}(r + v(s'))p(s',r|s,a)\;.$$

Because $q(s,a)$ relates to expected returns at time $t$, and returns are defined as $G_t = \sum_{b = 0}^\infty \gamma ^b R_{t+b+1}$, thus $R_{t+1}$ is also a random variable at time $t$ that we need to take expectation with respect to, not just the state that we transition into.


Why would the lookup table (of a table-driven artificial agent) need to store data at pixel precision?

A tabular system for agent decisions is a direct and simple map of percept to control choice. For each percept received, the agent looks up the percept and cross-references it to the action it should take. In order to construct this, you need to list all percepts in full detail, with the associated control choice.
Clearly that is not going to be feasible for the automated taxi example. No-one would think to build such a table to handle natural image inputs. That is the author's point.
However, a tabular structure is a reasonable theoretical construct for mapping an arbitrary discrete function, and also is practical for simple environments.
To answer your extended question:

Assuming $10^{600,000,000,000}$ is derived from $30 × 1080 × 720 × 24 × 8 × 3600$, what is the purpose of storing data in the look up table at pixel precision?

It is the only way to get a map from percept to control using a tabular system.
By proposing any kind of summarisation or approximation of the input-to-output function to solve this, you have gone beyond the capability of a tabular system. That again is the author's point.

Wouldn't storing higher level of details be enough to solve these kind of problems (ie, autonomous driving)?

If it is really obvious to you that this is the solution, then that's a good thing as you are thinking ahead. However, you should also consider what that means in terms of what you might be giving up, from a theoretical perspective. For instance, a tabular system can make radically different decisions based on very minor differences between percepts, whilst any form of processing of the inputs to make them easier to handle is necessarily going to remove information that might be important.


The short answer to your question is: you probably do not fully know your data. remember that ML is no magic wand. It needs your understanding of the data and the behavior of it. Although it is approved that neural networks with at most two hidden layers can approximate any model with an acceptable degree precision, setting up the structure of the neural network is still your task; also note that the cleaner your data is, the more acceptable the degree of precision will be for a structured NN.

the following more detailed explanation might also help. understanding your data that I pointed above, means:

the design of your machine learning pipeline is the first and most important thing to take care of. Are you sure you have handled training set and test set appropriately? are the ratios standard? have you performed random sampling well?
the size of the dataset is also a matter of consideration. remember a ratio of n/p>10 (n: the number of instances, p: the number of features) is necessary. So if you have 7 features, you should have more than around 100 instances. check how big is your dataset
the distribution of your dataset is also important. dealing with badly distributed data should be considered if you have an issue here. use plots and histograms to understand the distribution of your dataset over feature axes. There is a high risk of overfitting if your data is distributed badly
also take care of your output distribution. with a bad distribution over your output data (focused outputs in a zone) you loose the power of generalization of your model
check the relations between different features. check if there are any pairs of dependent variables. you can use scatter plot or calculating pairwise covariances. PCA or more complicated methods (like auto-encoders) will be of help if there is any issue here
one possibility is maybe your data doesn't carry any latent information at all. maybe the underlying pattern is so stochastic and nonlinear that it seems to be random. in other words: you can not predict what kind of food your grandma has cooked by looking at results of world cup games, no matter how big your dataset is.

Now about your output matrix:

reshaping your matrix to an array of 144 might be the best or the worst thing to do. it depends on the characteristics of your data. maybe the position of each element in the matrix is also important in predicting the value. so you might want to include the i,j of each element in the input array too. maybe each output also takes effect from neighbors. there are some other options to consider:

you can use 144 different MLP networks, one for each element (a little bit like what you have already done, when reshaping your matrix. since your library constructs a single MLP network that has 144 neurons in the output layer. so it seems like 144 networks that only vary in the output layer, and note that the weight correction step is different too, since the training error for all of the outputs propagate back in the layers simultaneously)
you can create 144 instances out of each instance, each of which has 7+2 inputs and 1 output. the 2 more inputs are the position of the output value (i,j). it might be a poor decision to do that but this method has helped me a couple of times, so test it
Do the second step, but include the neighbor values too. so you'd have for example 7+2+4 inputs (7 initial + 2 i,j + 4 neighbor values)
also consider using RNNs (recurrent neural networks). if you consider the matrix to be a sequence of values which has internal dependency plus the external dependency to those 7 features, then you can use a RNN network like LSTM.

feel free to update your post with more specific experiments and issues.


How do Artificial Neural Networks store data compared to Biological Neural Networks?

Second question first: Data is stored in an ANN in the form of weights in the adjacency matrix between neurons. During training, these weights are updated by a learning algorithm (such as backpropagation).

First question: according to award-winning neuroscientist Tim Bliss:

  “It’s been accepted really since the turn of the 20th century, since the time of the Spanish neuroscientist Ramón y Cajal, that really the only place where memories can be stored is at synapses, the junctions between nerve cells.

A protein called the NMDA receptor plays a key roll in the strengthening of synaptic connections (which is more broadly achieved by a form of Hebbian Learning).


Who is working on explaining the knowledge encoded into machine learning models?

I have one concept, which allows to structure knowledge gathered by a ML system in both categorical and algorithms-like structure.

The key idea here is that we contain in our minds some network-like structure in our minds, which helping us not only to classify some data, but also to form an abstract meaning about text or message.

Other than that, in opposite to our minds, it looks like this solution can produce these structures in readable for us forms.


Which library would you recommend to begin with deep learning?

Keras is a simple and high-level neural networks library, written in Python, that works as a wrapper for Tensorflow and Theano. It's easy to learn and use. Using Keras is like working with Lego blocks. It was built so that people can do quick experiments and proofs-of-concept before launching into a full-scale build process.

With that in mind, it was made to be highly modular and extensible. Now, it can be used for a lot more than just experiments. It can help with RNN, CNN, and combinations of both.

If you want to begin and make a prototype ready solution, then I will recommend you start with Keras. To know the details under the hood, then learn TensorFlow. It has huge active community and also very good resources are available, for example, this Youtube series.

See also https://blog.keras.io/keras-as-a-simplified-interface-to-tensorflow-tutorial.html.


Digital Rights and Agents talking to humans

Googling this throws a lot of debate on this issue of whether the call made to the restaurant for booking was legal or not. I found this article to put forward a lot of ideas for and against this. So it is for us to decide.

But one thing that I fully agree with you, which should be made into a regulation, is human should know that he/she is talking to a bot. This tweet by @traviskorte reasons it out why:

We should make AI sound different from humans for the same reason we put a smelly additive in normally odorless natural gas.


What is the difference between a stochastic and a deterministic policy?

A deterministic policy is a function of the form $\pi_{\mathbb{d}}: S \rightarrow A$, that is, a function from the set of states of the environment, $S$, to the set of actions, $A$. The subscript $_{\mathbb{d}}$ only indicates that this is a ${\mathbb{d}}$eterministic policy.
For example, in a grid world, the set of states of the environment, $S$, is composed of each cell of the grid, and the set of actions, $A$, is composed of the actions "left", "right", "up" and "down". Given a state $s \in S$, $\pi(s)$ is, with probability $1$, always the same action (e.g. "up"), unless the policy changes.
A stochastic policy is often represented as a family of conditional probability distributions, $\pi_{\mathbb{s}}(A \mid S)$, from the set of states, $S$, to the set of actions, $A$. A probability distribution is a function that assigns a probability for each event (in this case, the events are actions in certain states) and such that the sum of all the probabilities is $1$.
A stochastic policy is a family and not just one conditional probability distribution because, for a fixed state $s \in S$, $\pi_{\mathbb{s}}(A \mid S = s)$ is a possibly distinct conditional probability distribution. In other words, $\pi_{\mathbb{s}}(A \mid S) = \{ \pi_{\mathbb{s}}(A \mid S = s_1), \dots, \pi_{\mathbb{s}}(A \mid S = s_{|S|})\}$, where $\pi_{\mathbb{s}}(A \mid S = s)$ is a conditional probability distribution over actions given that the state is $s \in S$ and $|S|$ is the size of the set of states of the environment.
Often, in the reinforcement learning context, a stochastic policy is misleadingly denoted by $\pi_{\mathbb{s}}(a \mid s)$, where $a \in A$ and $s \in S$ are respectively a specific action and state, so $\pi_{\mathbb{s}}(a \mid s)$ is just a number and not a conditional probability distribution. A single conditional probability distribution can be denoted by $\pi_{\mathbb{s}}(A \mid S = s)$, for some fixed state $s \in S$. However, $\pi_{\mathbb{s}}(a \mid s)$ can also denote a family of conditional probability distributions, that is, $\pi_{\mathbb{s}}(A \mid S) = \pi_{\mathbb{s}}(a \mid s)$, if $a$ and $s$ are arbitrary. Alternatively, $a$ and $s$ in $\pi_{\mathbb{s}}(a \mid s)$ are just (dummy or input) variables of the function $\pi_{\mathbb{s}}(a \mid s)$ (i.e. p.m.f. or p.d.f.): this is probably the most sensible way of interpreting $\pi_{\mathbb{s}}(a \mid s)$ when you see this notation (see also this answer).
In the particular case of games of chance (e.g. poker), where there are sources of randomness, a deterministic policy might not always be appropriate. For example, in poker, not all information (e.g. the cards of the other players) is available. In those circumstances, the agent might decide to play differently depending on the round (time step). More concretely, the agent could decide to go "all-in" $\frac{2}{3}$ of the times whenever it has a hand with two aces and there are two uncovered aces on the table and decide to just "raise" $\frac{1}{3}$ of the other times.
A deterministic policy can be interpreted as a stochastic policy that gives the probability of $1$ to one of the available actions (and $0$ to the remaining actions), for each state.


Is there a way to ensure that my model is able to recognize an unseen example?

The comments already are giving you some good tips about how to improve what your model recognizes, but I think your question goes above that asking if there's a way to ensure that it will always recognize the cats.

The short answer is "no".

The slightly longer answer is "yes, but cheating".

Regardless, there are a lot of steps you might take to improve the generalization aspect of your model.

Long answer:

A drama: cat classification in three acts

Act I: Cat as texts

Let's start with an example. Say that your model is trained with these inputs, and learns to correctly recognize them as a cat or not a cat:

cat → yes!
Cat → yes!
ferret → no
cat. → yes!
Cat! → yes!
Three MC's and one DJ → no

Your goal is to train your model so that every new variation, even unseen ones, will correctly be identified.

With a good level of generalization, your model will correctly classify new inputs that it has never seen before:

skunk → no
cat? → yes!
dog → no
CAT → yes

With this scenario, let's say the model now finds this:

kat → ?

Is that a "cat" misspelled? Is that short for Katherine? What should the model do?

Act II: But surely this doesn't happen in real life

Leaving the analogy for a bit, will your model that's looking at domestic cats properly accommodate for Savannah Cats, or will it consider them out? (They kind of look like cheetahs.) What about Sphinx cats? (They look like raw chicken to me.) Elf cats? (They look like bats.) This is just an example, but you can probably figure out more.

And the reason behind this problem is that the distinction itself between different classifications (in real life) is not binary, but rather a transition between "yes, that's a textbook cat" and "that's a chair". Your model will output binary decisions (maybe accompanied by a confidence interval, but even with it, you'll make the call into deciding if it's a cat or not).

Setting specific boundaries will help. You can define that your model will only detect domestic cats, maybe no bigger than a certain size, only of certain colors, etc... This is limiting what the model will correctly recognize as a cat when we (humans) might disagree. For instance, I would still argue that flourescent cats are still cats.

Going back to the simple text analogy, this is similar to deciding that to be detected as a cat, it has to start with a "c". So now you've discarded ¡Cat!.

In this way, it's not possible to ensure (notice the word) that your model will detect all of these unknown variations. There will be always some room for error that needs to be accepted, as long as the errors are infrequent or rare enough that they can be accepted as a regular part of the model.

Act III: Concept drift, a cautionary tale

Finally, the problem becomes even harder as we might be dealing with concepts that change over time, outside of the knowledge of the model, and outside of the knowledge of the person that supervised the model learning.

As and the breeds of cat changes, your model will have to accommodate by what we (users of the model) consider a valid definition of cat. Which might change in really unexpected ways and not really "look" like a cat. And since your model can only learn from what "looks" like a cat, it's always held in a disadvantaged position.

This will happen with almost any machine learning model that is approximating a result, regardless of the technique/algorithm. Approximations include a level of error because reality is usually complex in ways that we either don't know about or that are too computationally expensive.


Why is a mix of greedy and random usually "best" for stochastic local search?

As an example of local/global minima, imagine being on a rugged, mountainous landscape, and you want to find the lowest point within some area. For a greedy search, every step you take will take you downhill. If you go downhill long enough, you'll eventually find a flat spot, which is a minimum - from here, there's no step you can take that will get you any lower. However, there's a nearby ridge, which if you crossed it, you could continue downhill to find an even lower spot, the global minimum (the true lowest point). Using your greedy approach, you'll never go uphill to cross the ridge, so you'll be stuck in the local minimum forever. If you occasionally take random steps (other than directly downhill), you have the opportunity to cross ridges that separate local minima, and you have a better chance of finding the global minimum. You are correct that in many cases, the random step won't help you cross a ridge, and will just take you up a mountain in the wrong direction, which is a waste of time. But unless we allow the algorithm to "explore" a bit, it will be content that the first minimum it finds is the best one, and will never get to the bottom.


Do specific units exists for measuring the intelligence of a machine?

One of the challenges of AI is defining Intelligence.
If we could precisely define general intelligence then we could program it into a computer. After all an algorithm is a process so well defined that it can be run on a computer.

Narrow AI can be evaluated on its success at achieving goals in an environment. In domains such as computer vision and speech recognition narrow AI algorithms can be easily evaluated.

Many universities curate narrow AI tests. Fei-Fei Li a professor at Stanford who directs the Artificial Intelligence lab there organises the annual ImageNet Challenge. In 2012 Geoffrey Hinton famously won the competition by building a Deep Neural Network that could recognize pictures more accurately than humans can.

To my knowledge the testers commonly use Precision and recall evaluation metrics


Why is this simple neural network not training?

You should try bigger learning rate / more epochs.

What you see is basically output of randomly initialized weights. Default learning rate for SGD is 0.001, it's definitely not enough for weights to change quick enough with single batch epochs and 9 examples.

I never used TF.js, but in keras (that's look very similar) you can set it like this

sgd = keras.optimizers.SGD(lr=1)
model.compile(sgd, 'mse')

model.fit(xs, ys, epochs=200)

Another approach would be to use adaptive learning rate optimizer, for example adadelta. It still requires quite a lot of epochs, but much less, then SGD with standard learning rate and you don't need to set LR manually.

model.compile('adadelta', 'mse')
model.fit(xs, ys, epochs=2000)

Here you can read how different optimizers work and converge

Edit:
Another thing with tf.js (and javascript code in general) is it doesn't guarantee order of operations. So, in your case it prints evaluation before it finishes training.

This code should perform fine. Take note that we evaluate model in then clause.

const model = tf.sequential();
model.add(tf.layers.dense({ units: 5, activation: 'sigmoid', inputShape: [1]}));
model.add(tf.layers.dense({ units: 1, activation: 'sigmoid'}));
model.compile({loss: 'meanSquaredError', optimizer: 'adadelta'});
const xs = tf.tensor1d([1, 2, 3, 4, 5, 6, 7, 8, 9]);
const ys = tf.tensor1d([0, 0, 0, 0, 0, 1, 1, 1, 1]);
model.fit(xs, ys, {epochs: 1000}).then(h => {
   console.log("Loss: " + h.history.loss[0]);
   model.predict(xs).print();
});

You can also use async/await approach, for example as it described in this answer


Is it better to adjust the natural lighting (while recording the video) or to subsequently apply filters on the original video?

Personally, I'd say as long as the object is visible don't do either. If the model has been well built and if lighting changes would help, the convolution operation weights would learn an operation similar to contrast or brightness changes.

On the other hand if the object visibility is an issue, then natural lighting changes would be better, due to the lack of potential artefacts a filter would create.

So overall, I'd say natural lighting changes should be more helpful (Assuming model is built well) and brightness filters would not be very helpful as the convolution operations would learn them if they were useful, also there would be  artefacts in the input which can lead to the model learning irrelevant details.

Hope this helped!


What should the range of the output layer be when performing classification?

For this particular classification problem, I would recommend you using a softmax function whose output range is [0,1].
The sum of all outputs should be 1, so an advantage of using a softmax function is that you get a percentage of how confident the network is in this classification.

Side note: As DuttaA has commented, cross entropy loss is a better loss function than the quadratic mean squared error.


In AlphaZero, do we need to store the data of terminal states?

I'm not 100% sure whether or not they added any data for terminal game states, but it's very reasonable to indeed make the choice not to include data for terminal game states. As you rightly pointed out, we don't have any meaningful targets to update the policy head towards in those cases, and this is not really a problem because we would also never actually make use of the policy output in a terminal game state. For the value head we could provide meaningful targets to update towards, but again we would never actually have to make use of such outputs; if we encounter a terminal game state in a tree search, we just back up the true value of that terminal game state instead of making a call to the network to obtain a value function approximation.
In theory, I could imagine some cases where training the value head on terminal game states might be slightly beneficial despite not being strictly necessary; it could enable generalisation to similar game states that are not terminal (but close to being terminal), and speed up learning for those. For example, if you have a game where the goal is to complete a line of $5$ pieces, training the value head on terminal states where you actually have a line of $5$ pieces and have entirely won the game might generalise and speed up learning for similar game states where you may not yet have $5$ pieces in a line, but are very close to that goal. That said, intuitively I really don't feel like this would provide a big benefit (if any), and we could probably also come up with cases where it would be harmful.

In the $\pi^{\text{T}}$ notation, $\pi$ is a vector (for any arbitrary time step, the time step is not specified here) containing a discrete probability distribution over actions (visit counts of MCTS, normalised into a probability distribution), and the $\text{T}$ simply denotes that we take the transpose of that vector. Personally I don't like the notation though, I prefer something like $\pi^{\top}$ which is more clearly distinct from a letter $T$ or $\text{T}$.
Anyway, once you understand that to denote the transpose, you'll see that $\pi^{\top}\log(\mathbf{p})$ is a dot product between two vectors, which then ends up being a single scalar.


Bayes' theorem relates conditional probabilities:

$$P(A \mid B) = \frac{P(B \mid A) P(A)}{P(B)}$$


Why is KL divergence used so often in Machine Learning?

This question is very general in the sense that the reason may differ depending on the area of ML you are considering. Below are two different areas of ML where the KL-divergence is a natural consequence:

Classification: maximizing the log-likelihood (or minimizing the negative log-likelihood) is equivalent to minimizing KL divergence as typical used in DL-based classification where one-hot targets are commonly used as reference (see https://stats.stackexchange.com/a/357974). Furthermore, if you have a one-hot vector $e_y$ with $1$ at index $y$, minimizing cross-entropy $\min_{\hat{p}}H(e_y, \hat{p}) = - \sum_y e_y \log \hat{p}_y = - \log \hat{p}$ boils down to maximizing the log-likelihood. In summary, maximizing the log-likelihood is arguably a natural objective, and KL-divergence (with 0 log 0 defined as 0) comes up because of its equivalence to log-likelihood under typical settings, rather than explicitly being motivated as the objective.
Multi-armed bandits (a sub-area of reinforcement learning): Upper confidence bound (UCB) is an algorithm derived from standard concentration inequalities. If we consider MABs with Bernoulli rewards, we can apply Chernoff's bound and optimize over the free parameter to obtain an upper bound expressed in terms of KL divergence as stated below (see https://page.mi.fu-berlin.de/mulzer/notes/misc/chernoff.pdf for some different proofs).

Let $X_1, \dots, X_n$ be i.i.d. Bernoulli RVs with parameter $p$.
$$P(\sum_i X_i \geq (p+t)n) \leq \inf_\lambda M_X (\lambda) e^{-\lambda t} = \exp(-n D_{KL}(p+t||p)).$$


OpenAI Gym interface when reward calculation is delayed? (continuous control with considerable reaction time)

It's not on your end, as a creator of flight simulator, to worry about what action should get the credit for the reward that happened some time after the action was taken. You should return the reward when the actual event happens not when the action that caused it happened. It's the job of the reinforcement learning agent to figure that out. For example if you want to give a reward when airplane nose is at 45 degrees from the horizontal axis, you should return the reward when that event actually happens, RL agent should figure out that the crucial action happened some time ago. This may be difficult for the agent but its up to the user to use a proper algorithm and proper exploration strategy to solve the problem.


Assuming that you do have a dataset (images + labels/bounding boxes) in a format that's required by the training model, you can fine-tune your existing model. You can choose to unlock the final few layers or leave all the layers unlocked during the training process. When I had performed such an experiment with RetinaNet I chose to unlock final few conv layers and was able to achieve slightly higher accuracy.


Yes, it is a very common practice to use some RNN when your input data is a sequence.
Besides, your network has some shape issue if your input data is 2D. You should, at least flatten your input data to a vector to be able to forward propagate to the dense layer, but instead of this use some kind of RNN.
To the best of my knowledge, that dropout value doesn't good. It seems too low.


Can I do image classification with Multi Layers Perceptron (MLP)?

Multi Layers Perceptron(MLP) can be used for image classification, but it has a lot of deficiency than Convolutional Neural network(CNN). But if you compare MLP and Fisher Faces , the better one is MLP, because Fisher Faces will be increasingly difficult if adding more individuals or classes. You can make a simple MLP model, because it just has 3 layers which are an input layer, hidden layer and output layer, here a source code that you can try:

https://towardsdatascience.com/multi-layer-neural-networks-with-sigmoid-function-deep-learning-for-rookies-2-bf464f09eb7f

if you make a model, it will be based on training data, I think if make data trained composed based on noise in background and different light shadows on your image I think it will have a better performance, but remember if you are using MLP for image classification it can just predict an image on one spot, for example:
"if you train a model with the object in the middle of an image, your model can not predict it when the image is moved to the different spot".

here is pdf to see Fisher Faces performance:

https://core.ac.uk/download/pdf/84801048.pdf


How to use Genetic Algorithm for varying lengths of solutions

Before I answer your question, it is important to frame what a GA is, so please allow me to cover some history.

Friedman and Fraser performed some of the earliest evolutionary computation experiments. Fraser presented the case of diploid organisms represented by binary strings of a given length. Each bit in the string represented a gene. Fraser proposed the process of single-point crossover for binary string reproduction.

American computer scientist John Henry Holland and his colleagues at the University of Michigan formalised the evolutionary algorithms originally implemented by Bremermann, Fraser, Friedman and Friedberg.

Holland's formal algorithm is known as a Genetic Algorithm (GA). A GA has a fixed length chromosome structure. Traditionally it was binary based. However due to hamming distances of various bit strings a GA using a binary encoded chromosome string can get stuck at local minima. Holland experimented with integer based GAs too.

Getting back to your question, theoretically and in its strictest sense, a Genetic Algorithm has a fixed length chromosome vector. The primary reason for this is that each gene represents something. If it was variable then the meaning of the gene must be encoded into the chromosome, which makes evolution more difficult. However, there are other chromosome structures that are variable length. The chromosome structure itself implies meaning.

Koza experimented with an alternative version of a GA called a genetic programming (GP). A GP replaces the fixed length vector structure used in a genetic algorithm with a variable length, non-linear, hierarchical abstract data structure, known as a tree.

The tree consists of branch nodes and leaf nodes. Leaf nodes have no children and implement a number or constant from a terminal set. A leaf node is also known as a terminal node. A branch node is connected to other branch nodes or leaf nodes and implement an operator from a functional set. A branch node is also known as a non-terminal node.

The tree structure naturally accommodates a game rule. A tree is not restricted to a fixed length. A tree is a natural structure to represent hierarchical decisions rules, and a tree can store both binary, logical, and numeric functions within its nodes. Logical functions allow the GP to evolve conditional execution of game play functions not easily implemented using a fixed length vector. Tree does need a traversal path and grammar (syntax and semantics).

In short to answer your questions:

A GA (fixed length) is generally not used for games, but rather a GP (variable with decisions).
A GA is not the most suitable method for this kind of problem as it is fixed
length, and each gene corresponds to a rule or value.

I would implement a GP, where the operators of mutation and crossover are adapted for a GP. I would also use the game itself as a fitness function, where a pool of individuals either using a single population, or co-evolution evolve players that gradually increase their score.

The paragraphs above were taken from Nicholls' MSc thesis. His thesis was on stock market trading rule generation which is similar to game rule generation. His thesis is available here (Only available after April 2019).


Is it possible for a NN to reach the same results as CNNs?

NNs won't be able to reach the performance of CNNs, in general.

By a Neural Network, I am assuming you're pointing to 'vanilla' vector based neural networks.

Let's take the MNIST dataset, it performs almost similar in both NNs and CNNs; the reason being digits of almost same size, and similar spatial drawings. To put it easy, all the digits roughly take up the same area as the other 60k. If you were to zoom in our zoom out these digits, the trained NN may not perform well.

Why?

Plain NNs lack the ability to extract 'position independent' features. A cat is a cat to your eye, no matter whether you saw it in the center of an image, or left corner. CNNs use 'filters' to extract 'templates' of cat. Hence CNNs can 'localize' what it is searching for.

You could say, NNs look at the 'whole' data to make a sense, CNNs look at the 'special' features at certain parts to make a sense.

This is also a reason why CNNs are popular. NNs are suited for their own applications. Every input node in a vectorized Neural Networks represent a feature, and that feature is tied to a purpose throughout training, i.e. its position is fixed throughout the training.


What is the current state of AGI development?

The state of AGI research is pursuing the few problems that we have been able to break off from the gigantic research problem. These are terms which can be more thoroughly looked into.

A few of the main focuses are:

One Shot learning - You know how a person can sometimes learn to do something by something by seeing literally 1 example of it? Well current learning methods on the whole are not able to accomplish this to the extent that we can easily take for granted. Work is being done to find ways to approach this feat of learning and it’s on its way to becoming much more influential.
Transfer Learning - If you have ever played a side scroller like Mario, and then I gave you a slightly different game like Sonic, odds are you’d could learn to play Sonic faster than it took you to learn to play Mario. This is because of learning “savings” you have by transferring your Mario knowledge to the new Sonic domain. This is a much more popular research arm than one-shot, probably because it’s easier to think about but also because there have been promising results of pretraining a network on one set of data and focusing it an another task.
Creativity/curiosity - Although one could say that GANs have really changed how humans can be more creative, it is difficult to quantify curiosity and creativity This paper gives an okay overview. Moreover, allowing an agent to take the chances and make some mistakes as is the nature of creativity concerns many people who are focused on AI Safety.
Understanding concepts - This is subtle but very very important. Current AI methodologies struggle with imbuing AI with the ability to have concepts. By concepts, I don’t mean “it kinda looks like this neuron in the second last layer is sensitive to tires”. I mean that understanding what a tire looks like is just a small part of understanding what a tire is, what it is used for, what it affords someone to do etc. This research direction is in it’s infancy but will be much more influential as more theories and ideas are brought forward.

Despite the progress made in these fields and in the many other areas in AI, there is still much to be done and understood before we can finally have s̶k̶y̶n̶e̶t̶ Wall-E.


Machine learning algorithm for xml manipulation

Terminology

There are two uses of the word map in this discussion.

Road maps are construed below as images of road maps.
Mapping input to desired output is the skill the system must learn.

The set of examples used to teach the system from an existing mapping of input to output is called a labeled data set and the associated type of learning is called supervised.

Training Resources

Unreliable labeled road map examples and access to more also unreliable labeled road map examples
A large number of unlabeled road map examples and access to more road maps
A smaller number of labeled road map examples

Inputs

Virtual game map as an image
Starting position
Target ending position

Output

The fastest route in sufficient detail to fully direct car movement — Assuming that the route should be optimized for soonest arrival time, not for fuel conservation, minimal tire wear, safety, or minimal distance, since the car was identified as a race car.

Accurate Valuation of the Resource Inventory

The low quality routes from the black box service provide neither examples of desired system behavior nor examples undesirable system behavior.  The former case would be good training data.  The later would be good to use in an adversarial architecture based on the design of GANs and their variants.  The uncertain quality of the labeled data from the black box service makes it ambiguous and, from an information science point of view, of zero value.

Comparing the low quality routes and the high quality routes may be interesting, but not particularly useful given the current state of machine learning.  The objective of your initial project phase is to teach the machine to generate a high quality route from the low quality route, not produce a comparison report.  To do that without processing the images a substantially large overlap between these two sets would be required.

The smaller number of labeled road map examples for which the labels were created manually
Corresponding labeled road map examples for which the labels were created by the black box service

That approach would require you to correct bad routes to create a sufficient training set.  Without overlap, you have no training data for artificial network training.  It may be possible to create an algorithm of GAN style that learns how to correct the black box service output using a concept called cross-entropy, but it would require processing the images and would likely be more difficult to do so than to replace the black box service altogether.

If your ultimate goal is to create a working system that generates routes from images and start and and positions, I suggest discounting the external service altogether, and discard the idea of creating a route improver sub-system.  That you wish to improve upon the black box route generator's algorithm is noble but ultimately a time-consuming distraction to reaching the ultimate goal.

Replace the unreliable thing with the reliable thing you design and build and to which you will have full and open access to improve.  Your system will ultimately have to deal with the images in your code either way, and that's the most challenging aspect of the overall task.  Just learn about CNNs and RNNs and get right to it.  That's my advise anyway.

A Note on Feedback

For any non-trivial route, the car will not likely stay on the road if given only turn, acceleration, and deceleration instructions.  Detection of the edges of the road would normally be needed as a feedback mechanism to accompany the route instructions.  The only way around this is to make the visualization sufficiently accurate that cumulative errors never exceed the distance that would put the car off pavement.

A Note on Data Representation

JSON has a more flexible and terse structure when it comes to homogeneous arrays than XML, and it is easy to convert XML to JSON.  Furthermore, a transportation route is often represented as a directed graph, and there are many algorithms already conceived in graph theory that have been implemented in graph libraries in most languages.  For instance, shortest path, path equivalence, path concatenation, and detection of rings (driving in circles) are one line calls to these libraries.  Because JSON, for the reason given, has overtaken XML in many domains, the number of graph libraries that can read and write JSON directly has overtaken the number that can read or write XML directly.  The tooling for JSON analysis and visualization has surpassed that of XML at this point too.


Maybe.

AI has a long history of encountering mathematical impossibilities and then working around them already. While the individuals who solved these problems don't get as much press as Newton, Einstein, or Hawking, a case could be made that their contributions to human knowledge are on a similar scale. Unfortunately, their results don't relate to physical systems, so they can be harder to explain to the layperson.

Something to keep in mind is that your question assumes the correctness of the "Great Man Theory" of science history, which holds that science advances by the efforts of exceptional people, and that we need to wait around for more such people to appear for science to advance. This view of scientific history is overly simplistic, and probably wrong. For example, most (all?) of Newton's discoveries were likely to have been made by someone else, around the same time, if he didn't make them (see, e.g. Leibnitz, who actually did discover calculus at the same time), so a better view might be of a large community of researchers who gradually develop more advanced models based on each others' work.

To answer your question, I've listed out some past examples of problems that were overcome, and some outstanding problems that might require the development of new mathematical tools to solve properly. Keep in mind though, that we can't know for sure whether new tools are needed: maybe existing tools are sufficient, but no one has applied them in the right way yet!

Some past problems that plagued AI and required the development of better mathematical approaches:

The combinatorial explosion problem appeared to preclude having an AI system reason about probabilities and causation in a logically correct way. This problem was solved by the development of Bayesian networks and causal networks, with the work being led by Judea Pearl and his students. Pearl won a Turing award for this, but has received very little coverage in the popular press.
Many optimization problems that AI systems need to solve are NP-hard. This means that no general purpose algorithms exist that can give exact solutions to these problems in a polynomial number of steps. Because of this, the problems were initially viewed as intractable. The development of PTAS algorithms, and a deep understanding of phase-changes in NP-Hard problems (pioneered by Cheeseman et al.) led to an ability to identify exactly what makes these problems hard, to identify which subparts cause the hardness, and to practical algorithms that solve all sorts of problems in these domains.
Much effort in AI was spent designing new classification methods, and arguing about why one might or might not be better than another. The No Free Lunch Theorems, along with other work in computational learning theory provided a clear mathematical framework for understanding how systems can learn, and where their limits are.
As a more recent example, games involving chance, like Poker, have state spaces so vast that they cannot be searched through effectively, even with heuristics. Phrasing these games in the language of Game Theory, and proving the convergence of the counter-factual minimax regret algorithm provided solutions. Bowling et al. were at the center of this work. It got some press coverage, but most of the coverage focused on "computers can play poker", and not the exiting, more generalizable, lessons AI researchers had learned in the process.

Some problems that are still outstanding, and might require new mathematical techniques:

Are there efficient algorithms for solving problems in the complexity classes PPAD, NP, or #P? Many AI problems fall into these categories. We do not know the answer definitively, and although most researchers suspect that no such algorithms exist, understanding why not seems like it could also provide major research advantages. Existing proof techniques do not seem likely to crack this problem.
We have no good mathematical models of subjective experience. While some researchers (like Churchland) think this is not likely to play a major role in the development of intelligent systems, a concrete model could either support or refute that view, and might provide solid frameworks for current problems in AI like the study of motivation.
The original project of the social sciences was to provide mathematical models of individual humans, and then of human societies. This has mostly been abandoned (Psych is still at it, but Economics prefers to study rational agents instead of humans, and most of the others gave up mathematical modeling entirely to pursue the methodologies of the humanities instead of the sciences). Nonetheless, the lack of sound mathematical descriptions of human behaviours is becoming a major topic within AI, with work on norms, trust, emotion, and other topics. If a mathematical framework were developed to describe the actions of human societies or of individual humans, then AI would be advanced significantly (along with many other fields!).


Putting constraints on output of deep neural network

There are many ways of constraining the network's output.
Using an activation layer is a good one. If you sigmoid the output layer, the output is constrained between [0,1] and you can multiply that by 180 to adapt to your output. Since the layer is part of the optimization process, gradients will be learned correctly.
However, there are a few issues with it, namely sigmoid is not a linear curve, and is quite steep around 0. Maybe you can consider something like a ReLU, but with a maximum as well. This makes your output linearly between [0,180], and is also learned during optimization.
You could also just clip the output without any activation layers. This is simple and works, but the optimization process sees no difference between outputting 180 or 180000. This can lead to problems, depending on your problem.


What are the most instructive movies about artificial intelligence?

Here are my suggestions

Her, the AI part (movie spoiler):

   The AI can define a user's profile just by hearing his short story, and "act" based on the user's profile. The AI makes the user comfort with it (her). It shows a very advance user profiling.

Ex Machina the AI part (movie spoiler):

   This movie will show you how an AI learn to trick someone. The AI can express her feelings, and make you trust their feeling.

Eagle Eye, the AI part (movie spoiler):

   A movie about a general story "AI that want to kill". This movie can show you how The AI can compile a lot of information for its purpose.

Big Hero 6, the AI part (movie spoiler):

   Baymax is a very good example of a very complex expert system, he has a "knowledge chip" and a very smart way to diagnose people


Intuitively, why can the training of a neural network be formulated as a probability estimation problem?

Consider the case of binary classification, i.e. you want to classify each input $x$ into one of two classes: $y_1$ or $y_2$. For example, in the context of object classification, $y_1$ could be "cat" and $y_2$ could be "dog", and $x$ is an image that contains one main object.
In certain cases, $x$ cannot be easily classified. For example, in object classification, if $x$ is a blurred image where there's some uncertainty about the object in the image, what should the output of the neural network be? Should it be $y_1$, $y_2$, or maybe it should be an uncertainty value (i.e. a probability) that lies between $y_1$ and $y_2$? The last option is probably the most reasonable, but also the most general one (in the sense that it can also be used in the case there's little or no uncertainty about what the object is).
That's the reason why we can model or formulate this (or other) supervised learning problem(s) as the estimation of a probability value (or probability distribution).
To be more concrete, you can formulate this binary classification problem as the estimation of the following probability
\begin{align}
P(y_1 \mid x, \theta_i) \in [0, 1] \label{1}\tag{1}
\end{align}
where $y_1$ is the first class (or label), $(x, y) \in \mathcal{D}$ is a labeled training example, where $y$ is the ground-truth label for the input $x$, $\theta_i$ are the parameters of the neural network at iteration $i$, so, intuitively, $P(y_1 \mid x, \theta_i) $ is a probability that represents how likely the neural network thinks that $x$ belongs to the class $y_1$ given the current estimate of the parameters. The probability that $x$ belongs to the other class is just $1 - P(y_1 \mid x, \theta_i) = P(y_2 \mid x, \theta_i)$.  In this specific case, I have added a subscript to $\theta$ to indicate that this probability depends on the $i$th estimate of the parameters of the neural network.
Once you have $P(y_1 \mid x, \theta_i)$, if you want to perform classification, you will actually need to choose a threshold value $t$, such that, if $P(y_1 \mid x, \theta_i) > t$, then $x$ is classified as $y_1$, else it is classified as $y_2$. This threshold value $t$ can be $0.5$, but it can also not be.
Note that, in the case above, $P(y_1 \mid x, \theta_i)$ is a number and not a probability distribution. However, in certain cases, you can also formulate your supervised learning problem so that the output is a probability distribution (rather just a probability). There are also other problems where you don't estimate a conditional probability but maybe a joint probability, but the case above is probably the simplest one that should give you the intuition behind the idea of formulating machine learning problems as the estimation of probabilities or probability distributions.