How can "any process you can imagine" be thought of as function computation?

A function is simply a procedure that maps a particular input to a particular output. You put in $X$, and the function computes $Y$. Those $X$ and $Y$ can take many different forms. It could be mapping one number to another number (convert miles to kilometres), mapping sound to text (name that tune), mapping text to text (translate languages), mapping a video to text (review this movie), or mapping text to an image (draw a picture of $X$). Anytime you have a procedure that produces a fixed output based on a fixed input, it's a function.
Universality theorems guarantee that a neural network can produce an arbitrarily good approximation of any possible function. That doesn't mean it's easy, though - finding the right function that maps $X$ to $Y$ is the hard part.


Is the emergence of the field "psychology of artificial intelligence" necessary?

And by trying to answer my own question, and how I see it, yes - there is some requirement like it stated in this particular article: https://medium.com/womeninai/psychology-of-artificial-intelligence-ca0f0a9f3d7c

Because yes - there are multiple occurrences of an AI being biased, hateful, racist, sexist, etc.

Also, humans themselves tend to act towards an AI in the insolent ways, so the human factor also needed to be accounted for.

So yeah - I think it will be a branch, or an extension of an engineering psychology, which exactly addresses primarily a human factor and relationships between the human and a machine.

But other than that it will address the factors of how should it be built, how we can expect it to behave, and how should it behave, because it should follow the strict moral code, to be not the threat to us.


Is it possible to separately evolve a part of the population?

There have been extensive studies within evolutionary computation in the area of island models and niching for doing exactly this.
The advantages of this approach include greater population diversity (which is particularly useful when the problem is multiobjective) and the potential for concurrent execution of each separate population.
See also the answers to the question What is the niching scheme?.
With specific reference to genetic programming, here is a recent paper that uses a parallel island model.


how to recognise handwriting and convert into text?

What you described sounds to me like Optical Character Recognition(OCR).

If you want to implement your own, I would say read through how an open source OCR like Tesseract was implemented.

Otherwise just google for OCR and you will find a list of OCR engines, both comercial and open-source. To list but a few:

Tesseract: https://github.com/tesseract-ocr/tesseract
Matlab Computer Vision Toolbox's OCR: http://www.mathworks.de/de/help/vision/ref/ocr.html

Most OCRs makes use of a combination of a segmantation and image classification(usually done with Convolutional Neural Networks).

See here for more information.


Why is sampling non-uniformly from the replay memory an issue? (Prioritized experience replay)

The problem is not that we need importance sampling because the learning is off-policy -- you are correct in that for one step off-policy algorithms such as $Q$-learning we don't need importance sampling, see e.g. here for an explanation why. The reason we need the importance sampling is due to the loss used to train the network.
In the original DQN paper, the loss is defined as
$$L_i(\theta_a) = \mathbb{E}_{(s,a,r,s') \sim \mbox{U}(D)} \left[ \left( r + \gamma \max_{a'} Q(s',a' ; \theta_i^-) - Q(s,a;\theta_i) \right)^2 \right ]\;.$$
You can see here the expectation over the loss is taken according to a uniform distribution over the replayed buffer $D$. If we started randomly sampling non-uniformly, as is the case in PER, then the expectation wouldn't be satisfied and would introduce bias. Importance sampling is used to correct this bias.
Note that in the paper they mention that the bias isn't as much of an issue at the start of learning and hence they use a decaying $\beta$ that only makes the importance sampling weights the 'correct' weights to use at the end of learning - this means that the estimate of the loss is asymptotically unbiased.


What are episodic and non-episodic domains in reinforcement learning?

Assuming that continuing means non terminating, what does non-episodic or episodic domain mean ?

Non-episodic means the same as continuing. The quote you found is not listing two separate domains, the word "continuing" is slightly redundant. I expect the author put it in there to emphasise the meaning, or to cover two common ways of describing such environments.

Episodic domain problems are ones that terminate, or otherwise naturally split into groups of time steps that can be considered separately.


Does changing the order of the convolution layers in a CNN have any impact?

Conventionally, CNN layers downsample over and over, which enables them to capture details at different levels of abstractions. Usually, it is observed that the initial layers do nothing more than detecting edges, or filtering color channels; the combinations of these edges are what we perceive as 'features'.

If you reverse the order, you essentially are changing sampling modes down the line.
CNNs detect by 'downsampling' the inputs and therefore 'extracting' features.

It may not work as expected!


Is there any difference between a control and an action in reinforcement learning?

There's no difference. As they too often do, ML researchers take concepts from other disciplines, conveniently forget to cite sources and change the terminology, leading to much confusion. RL is a textbook example (pun intended). Optimal control researchers have been studying very similar problems long before RL ones, and used standard symbols and terms ($x$ for states, $u$ for controls). Then RL researchers came and changed just about everything.

See the paper A Tour of Reinforcement Learning: The View from Continuous Control (2018), by Benjamin Recht, which discusses reinforcement learning from a control and optimization perspective.

See also this tweet https://twitter.com/beenwrekt/status/1134536093980864514?s=21 (by Benjamin Recht) regarding the presentation of Sham Kakade.


You can always train your network with higher resolution images. There is nothing preventing you to do that if you don't have any restriction about inference time.

Also, the paper is actually mentioning how to upsample. Check for the phrase "Upsample using deconvolutional layers".

The most common ways to upsample are either using the deconvolutional layer or just by simple resize methods (like image resize).


So as far as I understood your problem is of mathematical nature, especially about back-propagation derivatives. Just a quick clarification, the squared loss function is not the only cost function, there might be other types of cost function. Anyways, coming to the first part of the question:

You can just actually apply chain rule directly from L2 to wherever you want. The weights are independent variables, they don't depend on each other. So dw5/dw1 will turn out to be 0 i.e you are differentiating a constant, in this case w5 with respect to w1. Why do we write this

instead of this

is because we can differentiate the loss function L2 w.r.t to a single independent variable only (of our choice), you cannot differentiate it w.r.t to an independent variable and still go on differentiating w.r.t to another independent variable. Its purposeless since you want to find the variation of L2 w.r.t to a particular weight only.
Here are 2 links for better understanding of partial derivatives:
Intuitive reasoning behind the Chain Rule in multiple variables? and Chain Rule Intuition

Yes, you certainly have to add the derivatives of all the branches. Basically first we look from the frame of reference of o1 and o2. From this frame o1 and o2 are independent variables and L2 is dependent on them and so we differentiate L2 w.r.t them. Then we move a step back and check what is o1 and o2 dependent on. How you go about doing this simply experience and an understanding of the thing you are doing. There is no steadfast algorithm from a simple point of view.
The vectorized implementation requires you follow a certain framework, it will be very lengthy here to explain the framework, but I am linking you to certain lecture videos which will make you aware of the convention being followed for vectorized implementation. Just check week 2 and week 3.

Also the thing I was talking about having different L2's in the begining is important in the sense is that in the future we may come up with weights which are dependent on other weights and so now we may have to do a dw_n/dw_m step since w_n is dependent on w_m, but lets keep that for another day.
Hope this helps!

EDIT: this might give you a sense of why I am  calling variables as independent variables How do I know if my back-propagation is implemented correctly?


How do I calculate the gradient of the hinge loss function?

Hinge loss is difficult to work with when the derivative is needed because the derivative will be a piece-wise function. max has one non-differentiable point in its solution, and thus the derivative has the same. This was a very prominent issue with non-separable cases of SVM (and a good reason to use ridge regression).

Here's a slide (Original source from Zhuowen Tu, apologies for the title typo):

Where hinge loss is defined as max(0, 1-v) and v is the decision boundary of the SVM classifier. More can be found on the Hinge Loss Wikipedia.

As for your equation: you can easily pick out the v of the equation, however without more context of those functions it's hard to say how to derive. Unfortunately I don't have access to the paper and cannot guide you any further...


What types of applications qualify as "compound intelligences"?

Swarm intelligence, compound intelligence, or group intelligence may emerge as an important concept as AI develops toward higher complexity.  Whether these terms should be considered synonymous is doubtful.

Compound features in biology are the result of control in differentiation during the development of an organism from a single cell.  Compounding in biology is a single function performed across like elements.

Swarms are a result of distinct organisms operating in proximity.  Swarming in biology is similarity in complex independent behaviors that appear coordinated but are generally engaged in as a defense from predators.

Group intelligence may be distinct from compound intelligence in that adjacent units may be in agreement or opposition, as can be case with intelligent beings in groups.  The agreement in the group that a variety of models is permissible.  We call them opinions, but they are distinct matches of models to problems that produce different projections and suggest different selections from among the group's options.

The terms general and strong appear in this question and many others and it appears that, mainly for historical reasons, may continue to frustrate clarity.

All intelligence is general in that it learns a generality from specific experiences and then applies that generality to future scenarios to achieve objectives within those scenarios.  What makes the application of the generality intelligent is that it is expected to work because it has been working for similar scenarios.

All intelligence is specific in that it is limited to scope of what generalities have been discovered.  Certain techniques are significantly more general that others because they are allegedly domain independent.  We call this mathematics.

Consequently, the development of artificial intelligence is not a path from specific to general but one of discovering generalities and applying them to specifics.  One could say that the most general thinking of mathematics applied to the computer is the primary activity of applied artificial intelligence, and the capabilities that emerge are a more sophisticated as more sophisticated mathematics is represented in working software and hardware.

The generalities take on greater complexity so that they can apply to a greater number of specific scenarios.  That is not a gain in strength but a gain in the breadth of potential application.

Physalia physalis is a symbiotic colony of organisms of four types, the pneumatophore (or float), dactylozooids (long tentacles), gastrozooids (feeding tentacles), and gonozooids which produce reproductive gametes.  To avoid breaking the historic conception of animals, these organisms are called polyps.  The venom of the colony does not come from any of those organisms but rather from another symbiotic organism, cnidocytes, which attach to the tentacles and are released under strictly controlled conditions.

In addition to all this symbiosis, there are several species of fish that use the colony for shade and protection that the colony lets swim among the tentacles and that have formed partial immunity to the cnidocytes.  These complex symbiotic networks are not fully understood, but they seem to operate well and create sustainable inter-specie systems.

It's not likely that the organisms, within their lifetimes adapt or remember, but the DNA of each organism has involved in a way that resembles intelligence in that the colony and its symbionts have adapted to the ocean's surface and its biology.

The idea that the system of the biosphere is the first example of broad intelligence is likely, in that design excellence has emerged from an evolutionary processes.  It is not altogether ridiculous to propose that human intelligence is a higher speed approach in neurons to the slower speed of DNA replication in larger organisms.  Because of the metabolic requirements of growth slows evolution for larger organisms with a greater cell count, these larger organisms may have needed to develop a way to achieve the nimble adaptivity of their lower ancestors.

Neurology facilitates the approximation of some aspects of evolution and may have been the most attainable natural solution to reacquire nimble adaptivity.

Attempting to apply these various ideas to the current and ongoing development of autonomous vehicles reveals a gap in understanding.  We don't yet have the mathematics developed to understand how compound, swarm, or group intelligence can be used in the laboratory to accomplish well defined problems in controlled execution scenarios.  That is probably a prerequisite to using these ideas in vehicle control.

The system designs of future cars may be like a colony of independent or semi-independent components that each have a role and purpose.  Compound, group, and symbiotic designs are likely to develop.  The current activities of vehicles on the road or in the air near airports is like a swarm, so the application of that idea is obvious: Avoid collisions.


Consider the execution order, 5 will have an invalid value because it hasn't been set form 3 yet. However the second time around it should have a value set. The invalid value should falloff after sufficient training.

0 -> 5
1 -> 5
5 -> 2
2 -> 3
3 -> 4
3 -> 5
RESTART
0 -> 5
1 -> 5


Convolutional Sequence to Sequence Learning kernel parameters

They are doing a matrix multiplication: consider $y = Ax, y \in \mathbb{R}^m, x \in \mathbb{R}^n, A \in M_\mathbb{R}(m,n)$. In the paper $x$ is a concatenation of $k$ elements of $\mathbb{R}^d$, so $x$ is long $kd$; $y$ is long $2d$.


Understanding lemma 2 of the "Trust Region Policy Optimization" paper

We can start with equation (30):

$$
\bar{A}(s) = P(a \neq \tilde{a}) \mathbb{E}_{(a,\tilde{a})\sim(\pi,\tilde{\pi}|a\neq\tilde{a})} [A_\pi(s, \tilde{a}) - A_\pi(s, a)]
$$

Taking the absolute value of both sides, the equality remains true. We can pull the probability term out of the absolute value since it is guaranteed to be nonnegative.

$$
|\bar{A}(s)| = P(a \neq \tilde{a}) |\mathbb{E}_{(a,\tilde{a})\sim(\pi,\tilde{\pi}|a\neq\tilde{a})} [A_\pi(s, \tilde{a}) - A_\pi(s, a)]|
$$

By Definition 1, $P(a \neq \tilde{a}) \leq \alpha$. Substituting this definition in, we get:

$$
|\bar{A}(s)| \leq \alpha \cdot |\mathbb{E}_{(a,\tilde{a})\sim(\pi,\tilde{\pi}|a\neq\tilde{a})} [A_\pi(s, \tilde{a}) - A_\pi(s, a)]|
$$

By Jensen's Inequality, we can take the absolute value inside the expectation.

$$
|\bar{A}(s)| \leq \alpha \cdot \mathbb{E}_{(a,\tilde{a})\sim(\pi,\tilde{\pi}|a\neq\tilde{a})} [|A_\pi(s, \tilde{a}) - A_\pi(s, a)|]
$$

The expectation of a random variable is always upper bounded by the max value of that variable.

$$
|\bar{A}(s)| \leq \alpha \cdot \max_{a,\tilde{a}|a\neq\tilde{a}} |A_\pi(s, \tilde{a}) - A_\pi(s, a)|
$$

This part is a little strange, and I'm not sure if this is the logic that the authors followed, but it is still true. For any $a,b$, we have that $|a - b| \leq |a| + |b|$.

$$
|\bar{A}(s)| \leq \alpha \cdot \max_{a,\tilde{a}|a\neq\tilde{a}} (|A_\pi(s, \tilde{a})| + |A_\pi(s, a)|)
$$

For $a \neq \tilde{a}$, it must be that either $|A_\pi(s, \tilde{a})| \geq |A_\pi(s, a)|$ or vice versa. We can use this to replace the pair of advantage functions with 2 times the max of the two.

$$
|\bar{A}(s)| \leq \alpha \cdot 2 \max_{a} |A_\pi(s, a)|
$$

Now, if we take the max over $a$ and $s'$, our inequality still holds, since we are taking the maximum over a set that contains $s$. Making this substitution gives us (31).

$$
|\bar{A}(s)| \leq \alpha \cdot 2 \max_{a,s'} |A_\pi(s', a)|
$$


How do you find the homography matrix given 4 points in both images?

To understand homographies and how to find them, you will need a good dose of projective geometry. I will briefly describe some preliminary concepts that you need to know before trying to find the homography, but don't expect to understand all these concepts with one reading iteration and only by reading this answer, if you are not familiar with them, especially, if you don't even know what homogenous coordinates are. For more details, I suggest you read the book Multiple view geometry in computer vision (2004) by Richard Hartley and Andrew Zisserman, in particular, chapter 4.

The projective space $\mathbb{P}^2$

$\mathbb{P}^2$ is the projective space of $\mathbb{R}^2$, so it is $\mathbb{R}^2$ augmented with lines and points at infinity. All points and lines of $\mathbb{P}^2$ actually belong to $\mathbb{R}^3$, i.e. they are vectors of three components, because they are homogenous representations of the counterparts in $\mathbb{R}^2$. To emphasize, in $\mathbb{P}^2$, both points and lines can be represented by a vector in $\mathbb{R}^3$, which is the homogenous representation of the counterpart vector in $\mathbb{R}^2$ (if it exists, e.g. points at infinity do not exist in $\mathbb{R}^2$).

What is a homography?

A homography (aka projectivity, collineation or projective transformation) is an invertible map $h$ from the projective space $\mathbb{P}^2$ to itself, $$h: \mathbb{P}^2 \rightarrow \mathbb{P}^2,$$ such that $\mathbf{x}_1, \mathbf{x}_2, \mathbf{x}_3$ lie on the same straight-line if and only if $h(\mathbf{x}_1), h(\mathbf{x}_2), h(\mathbf{x}_3)$ do.

This property is called collinearity. In practice, this property means that projective transformations can map straight-lines in one image to straight-line to another image, but it cannot map e.g. straight-lines to parabolas (and vice-versa). So, if you have a distorted image, a homography cannot convert it to a non-distorted image.

All and only linear maps in $\mathbb{P}^2$ are homographies

All linear maps in $\mathbb{P}^2$ are homographies and only linear maps can be homographies. Hence, when you want to find a homography, you are looking for a linear map in $\mathbb{P}^2$.

A homography can be represented as $3 \times 3$ matrix

Given that homographies are linear maps they can be represented as an invertible matrix

$$\mathbf{H}
=
\begin{bmatrix}
h_{11} & h_{12} & h_{13} \\
h_{21} & h_{22} & h_{23} \\
h_{31} & h_{32} & h_{33}
\end{bmatrix}
\in \mathbb{R}^{3 \times 3},
$$
such that, $\forall \mathbf{x} \in \mathbb{P}^2$, the following equation holds
$$
h(\mathbf{x}) = \mathbf{H}\mathbf{x}
$$

Homographies map points in $\mathbb{P}^2$ to points in $\mathbb{P}^2$

Given that homographies map points in $\mathbb{P}^2$ to other points in $\mathbb{P}^2$ and there's a matrix $\mathbf{H} \in \mathbb{R}^{3 \times 3}$ for each homography $h$, then it follows that, $\forall \mathbf{x} \in \mathbb{P}^2$,
\begin{align}
\mathbf{x}'
&=
\mathbf{H}\mathbf{x} \\
\begin{bmatrix}
\mathbf{x}'_1 \\ \mathbf{x}'_2 \\ \mathbf{x}'_3
\end{bmatrix}
&=
\begin{bmatrix}
h_{11} & h_{12} & h_{13} \\
h_{21} & h_{22} & h_{23} \\
h_{31} & h_{32} & h_{33}
\end{bmatrix}
\begin{bmatrix}
\mathbf{x}_1 \\ \mathbf{x}_2 \\ \mathbf{x}_3
\end{bmatrix}
\tag{1}\label{1}
\end{align}
where $\mathbf{x}' \in \mathbb{P}^2$, for some homography $\mathbf{H} \in \mathbb{R}^{3 \times 3}$.

In practice, this means that you can transform points in an image $I_1$ to points in image $I_2$ by matrix multiplication.

A homography has $8$ degrees of freedom

Although the matrix $\mathbf{H}$ has $9$ entries, it has only $8$ degrees of freedom, which means that, in practice, there are only $8$ variables that you need to find. This property comes from the fact that only the ratio between the elements of $\mathbf{H}$ actually counts. This means that equation \ref{1} can actually be written as
\begin{align}
\mathbf{x}' = \lambda \mathbf{H}\mathbf{x} \tag{2}\label{2}
\end{align}
for all $\lambda \in \mathbb{R} \setminus \{0 \}$.

How to estimate a homography from point correspondences?

We want to estimate a homography $\mathbf{H} \in \mathbb{R}^{3 \times 3}$ from point-to-point correspondences $\{(\mathbf{x}^i, \mathbf{x}'^i) \}_{i=1}^N$, where $N \geq 4$, such that $\mathbf{x}'^i = \mathbf{H}\mathbf{x}^i, \forall i$.

First, recall that the matrix $\mathbf{H}$ has $8$ degrees of freedom (i.e. variables we want to find).

Each point-to-point correspondence represents $2$ constraints

Each point-to-point correspondence $(\mathbf{x}^i, \mathbf{x}'^i)$ accounts for $2$ constraints, i.e. $\mathbf{H}\mathbf{x}^i$ maps to the point $\mathbf{x}'^i$, which has $2$ degrees of freedom (even if it's defined by 3 components) because it's represented in homogenous coordinates and so, as for equation \ref{2}, it's defined "up to scale". In other words, these $2$ degrees of freedom represent $2$ constraints.

$4$ point-to-point correspondences are necessary to estimate a homography

Given that $1$ point-to-point correspondence represents $2$ constraints, then $4$ point-to-point correspondences corresponds to $8$ constraints. Given this and given that homographies have $8$ degrees of freedom, at least $4$ point-to-point correspondences are necessary to estimate a homography.

The equation \ref{2} can actually be written as

$$
\mathbf{x}' \times \lambda \mathbf{H}\mathbf{x} = \mathbf{0} \label{3} \tag{3}
$$
where $\times$ is the cross-product.

Direct linear transformation

If you manipulate equation \ref{3}, then you will end up with another equation (if you ignore the scaling factor $\lambda$, which we can ignore because the homography converts points independently of their magnitude)

$$
\mathbf{A}_i \mathbf{h} = \mathbf{0}
$$
where $\mathbf{A}_i \in \mathbb{R}^{2 \times 9}$ is the design matrix (i.e. the matrix that contains the input data to estimate the homography) that contains the elements of the the point-to-point correspondence $(\mathbf{x}^i, \mathbf{x}'^i)$ and $\mathbf{h} \in \mathbb{R}^{9}$ is a vector that contains the unknown elements of $\mathbf{H}$. The details of this manipulation can be found in section 4.1 of the book Multiple view geometry in computer vision.

Now, the idea is that you can vertically stack $N \geq 4$ equations of the form $\mathbf{A}_i \mathbf{h} = \mathbf{0}$ to build the final linear system that you would like to solve
$$\mathbf{A} \mathbf{h} = \mathbf{0} \tag{4}\label{4}$$
where $\mathbf{A} \in \mathbb{R}^{2N \times 9}$.

The solution to this system is the vector $\mathbf{h} \in \mathbb{R}^{9}$, that is, your homography!

If you know something about linear algebra, you know that the solutions to $\mathbf{A} \mathbf{h} = \mathbf{0}$ are elements of the null space of $\mathbf{A}$.

Then, to find $\mathbf{h}$, you will typically use singular value decomposition (SVD). See e.g. How is the null space related to singular value decomposition? for a possible explanation of why you can use SVD to find an element of the null space.

This algorithm (which is described on page 91 of the cited book, i.e. Algorithm 4.1) is called direct linear transformation.


Why neural networks tend to be trained to recognize multiple things instead of just one?

In practice you never want to classify just a single digit rather than series. In such case you have to pass a patch of image to multiple network, which would make it inconvenient. If you built different accurate models, training parameters will not significantly reduced. For example sloppy written 6, in a single model the probability of being 6 and 0 would be close, not same if you consider likelihood you may get closest answer. While with different models the probability may vary in a greater scale and you may not have good generalization as you have in single model. At the end everything boils down to generalization and in my experience neural networks trained with multiple things have good generalization property that a single.


Can an AGI convince another AGI to modify its code?

An AI takes the decision based on the output of their utility function. This is just a fancy word for the calculations that AI perform to compare profit and loss of taking a certain decision.

There is always a tight analogy between an AGI and a human. You can juxtapose the utility function to how we make decisions by considering marginal profits of doing something over the other.

Now your question. The answer is a big NO. This is because an AI (or a human) never takes an action that gives a low score on its utility function or is against the fundamental view of the machine (or human). AI only cares about the utility function and goal state (plus the instrumental goals), nothing else. Our AI outside the box has been created with the purpose of not opening the box. For it, the action of opening the box has a very low score, or a negative score.

Now you might be wondering why an AI can't be convinced of doing something that gives a low score in its utility function. Keep reading.

Consider this for a second. Suppose an AGI was created, assigned with the task of copying the handwriting of others and improving its own writing. You can think of it as a smart writing hand. Now, what this AI might come up with is that, in order to practice writing, it needs more pages and thus it needs to cut more trees for that. No matter what you do, AI will not stop from cutting more and more trees. It might even replicate itself into machines that will cut trees for him. Now, the only thing that can be done is to turn off the AGI. But, a smart machine would have known the possibility of being turned off and thus will have transferred itself to many other machines over the globe. The important question to ask here is, why it is doing all this? It is not because the machine wants to live, as we humans want to. The only reason why it wants to live is to fulfill its goal.

You simply can't change the fundamental view of the machine. The change can result in, the machine not able to attain its goal or not to that extend (optimal profit). This is the reason why you can't convince a machine to do something that it was asked not to do as its primary task.

You, as a person, are living a life now, and have some fundamental beliefs. Let us consider that you believe in not killing someone. Now, suppose that I give you a pill, and tell you that after taking this pill it will rewire your brain and you will kill the first four people you see. But, after that, you will achieve pure satisfaction and happiness. Now you will definitely not take that pill as it conflicts with something that you believe in now. Also, you will try your best and fight back not to take that pill. The same thing applies to an AGI too. It doesn't matter what your future version will feel or attain after rewiring of the brain (changes in code), it is what and who you are now matters. This video link will help.

I hope it answers your question. There are lots of things to consider here. I have assumed a few things and tried to answer according to that.

There is one more thing. We don’t tell the machines how to do something, instead, we only tell them what to do (at least in the case of an AI). This is because sometimes we don't know the optimal way of solving certain problems. In the case of your question, we don't know what these two machines will do or say to each other. It will be a very interesting thing to hear or watch.


How can we define common sense in an AI agent?

I came up with a few ideas I would argue are valuable in motivating the idea of common sense for a machine learning model.

Common sense is retrospective. We define it in terms of a past (sensible) actions and conditions, and we can say someone has good common sense on the basis of their behavior, which we can view as the sum of their historical actions and the degree to which they were sensible.
The actions alone are not sufficient for demonstrating common sense; the mental model(s) that generated those actions is important as well. Why did the individual take those sensible actions? Is their reasoning behind their action sensible as well, or did they merely get lucky (put another way, does their generally not sensical mental model produce sensible actions in certain cases and non sensical actions in most others)? This hints that common sense is contingent on the rationalization of the actions themselves.
Given the former, common sense is contingent on the enumeration of possible actions and choosing the right one, given the context of the situation. For example, I’m walking on a path and see a snail. What are some actions? I could keep walking, stop and admire it for a while, step on it, or eat it. The first two options are sensible if I value the snail as a living being. The first option is sensible if I’m in a rush. The second and fourth may be sensible if I’m a chef exploring nature for new potential ingredients. The third option if I recognize the snail as invasive. We say someone has common sense when, given the context, the actions chosen are sensible and we can reason about them intuitively.

My guess is that, the intuition behind the perceived sense of an action is what you’re after. I’d argue that, ultimately, the intuition of common sense is defined by the person developing the model and will have to do with the formulation of the model (e.g it’s assumptions, the objective function, etc). After all, common sense is subjective and context specific.

Concretely, a model can have common sense if the developer bakes it in and we can use inferential methods to demonstrate this. For example, in a Word2Vec model, we might see that $\mathsf{Paris} \mapsto \mathsf{France}$ and would expect that $\mathsf{Tokyo} \mapsto \mathsf{Japan}$. To interrogate this, we might do some vector math and find that $\mathsf{Paris} - \mathsf{France} + \mathsf{Tokyo} = \mathsf{Japan}$. How and if the AI model develops the larger association between $\mathsf{Capital} \mapsto \mathsf{Country}$, however, comes down to how the developer built and trained the model to recapitulate their own common sense.


What is the derivative function used in backpropagration?

Overview

The derivatives of functions are used to determine what changes to input parameters correspond to what desired change in output for any given point in the forward propagation and cost, loss, or error evaluation &mdash whatever it is conceptually the learning process is attempting to minimize. This is the conceptual and algebraic inverse of maximizing valuation, yield, or accuracy.

Back-propagation estimates the next best step toward the objective quantified in the cost function in a search. The result of the search is a set of parameter matrices, each element of which represents what is sometimes called a connection weight. The improvement of the values of the elements in the pursuit of minimal cost is artificial networking's basic approach to learning.

Each step is an estimation because the cost function is a finite difference, where as the partial derivatives express the slope of a hyper-plane normal to surfaces that represent functions that comprise forward propagation. The goal is to set up circumstances so that successive approximations approach the ideal represented by minimization of the cost function.

Back-propagation Theory

Back-propagation is a scheme for distribution of a correction signal arising from cost evaluation after each sample or mini-batch of them. With a form of Einsteinian notation, the current convention for distributive, incremental parameter improvement can be expressed concisely.

$$ \Delta P = \dfrac {c(\vec{o}, \vec{\ell}) \; \alpha} {\big[ \prod^+ \! P \big] \; \big[ \prod^+ \!a'(\vec{s} \, P + \vec{z}) \big] \; \big[ c'(\vec{o}, \vec{\ell}) \big]} $$

The plus sign in $\prod^+\!$ designates that the factors multiplied must be downstream in the forward signal flow from the parameter matrix being updated.

In sentence form, $\Delta P$ at any layer shall be the quotient of cost function $c$ (given label vector $\vec{\ell}$ and network output signal $\vec{o}$), attenuated by learning rate $\alpha$, over the product of all the derivatives leading up to the cost evaluation. The multiplication of these derivatives arise through the recursive application of the chain rule.

It is because the chain rule is a core method for feedback signal evaluation that partial derivatives must be used. All variables must be bound except for one dependent and one independent variable for the chain rule to apply.

The derivatives include three types.

All layer input factors, the weights in the parameter matrix used to attenuate the signal during forward propagation, which are equal to the derivatives of those signal paths
All the derivatives of activation functions $a$ evaluated at the sum of the matrix vector product of the parameters and signal at that layer plus the bias vector
The derivative of the cost function $c$ evaluated at the current output value $\vec{o}$ with the label $\vec{\ell}$

Answer to the Question

Note that, as a consequence of the above, the derivatives of both the cost (or loss or error) function and any activation functions are necessary.

Redundant Operation Removal for an Efficient Algorithm Design

Actual back propagation algorithms save computing resources and time using three techniques.

Temporary storage of the value used for evaluation of the derivative (since it was already calculated during forward propagation)
Temporary storage of products to avoid redundant multiplication operations (a form of reverse mode automatic differentiation)
Use of reciprocals because division is more costly than multiplication at a hardware level

In addition to these practical principles of algorithm design, other algorithm features arise from extensions of basic back-propagation. Mini-batch SGD (stochastic gradient descent) applies averaging to improve convergence reliability and accuracy in most cases, provided hyper-parameters and initial parameter states are well chosen. Gradual reduction of learning rates, momentum, and various other techniques are often used to further improve outcomes in deeper artificial networks.


Your approach will give a similar result.
The difference with using the constraint is that the constraint allows the network to essentially confirm that it doesn't need the connections before it reallocates them (i.e. if the weight is lessened by the constraint and it results in something not working, then the weight will start increasing).


Is a decision tree less suitable for incremental learning than e.g. a neural net?

I will try to answer all three questions to the best of my ability.

Is this basically true? Quick googling just brought me to a lot of papers trying to fit decision trees into incremental learning.

The problem with decision trees in an online learning setting is that the model should be able to update when experiences (i.e. state, action, reward, new state) of the environment are collected. Any data that does not reflect the structure of the tree will cause the decision tree to fall apart and requires you to rebuild the model again. This is not necessarily a problem, but it is something you have to take into consideration when building a model.

What other algorithms fall under this category?

Other algorithms which assumes that the data is known beforehand (offline). For example K-NN or Learning Vector Quantization (LVQ). However it should be noted that these algorithms can be adapted to work in an online fashion.

Are neural nets good for incremental learning? What other algorithms would be good?

Neural networks can capture changes from online experiences fairly well. The reason is that each weight in such a network is adapted when a new experience comes in. The weights are adapted with regards to an error function and this allows to update the model relatively easy.

In 2016, Venkatesan, et al proposed a novel progressive learning technique for multi-class classification. You might want to look into this as the system that they use allows for online learning. Other techniques exists as well and should be found when googling for online learning methods.


How much computing power does it cost to run GPT-3?

I can't anwser your question on how much computing power you might need, but you'll need atleast a smallgrid to run the biggest model just looking at the memory requirments (175B parameters so 700GB of memory). The biggest gpu has 48 GB of vram  I've read that gtp-3 will come in eigth sizes, 125M to 175B parameters. So depending upon which one you run you'll need more or less computing power and memory. (https://lambdalabs.com/blog/demystifying-gpt-3/)
For an idea of the size of the smallest, "The smallest GPT-3 model is roughly the size of BERT-Base and RoBERTa-Base."


Is the Q value the same as the state-action pair value?

I don't understand your question very clearly.

Q-value of a particular state-action pair (s,a) under policy $\pi$ is the total reward you would expect to collect if you start from the state s, take the action a, and follow policy $\pi$ from then on.

In the literature, this is referred to as state-action values.


How can we use linear programming to solve an MDP?

This question seems to be addressed directly in the slides of the deck you linked to in the comment under your question.

The basic idea is:

Assume you have a complete model of the MDP (transitions, rewards, etc.).
For any given state, we have the assumption that the state's true value is reflected by:

$$V^*(s) = r + \gamma \max_{a \in A}\sum_{s' \in S} P(s' | s,a) \cdot V^*(s')$$

That is, the true value of the state is the reward we accrue for being in it, plus the expected future rewards of acting optimally from now until infinitely far into the future, discounted by the factor $\gamma$, which captures the idea that reward in the future is less good than reward now.

In Linear Programming, we find the minimum or maximum value of some function, subject to a set of constraints. We can do this efficiently if the function can take on continuous values, but the problem becomes NP-Hard if the values are discrete. You would usually do this using something like the Branch & Bound algorithm. These are widely available in fast implementations. GLPK is a decent free library. IBM's CPLEX is faster, but expensive.
We can represent the problem of finding the value of a given state as:
$$\text{minimize}_V \ V(s)$$
subject to the constraints:
$$V(s) \geq r + \gamma\sum_{s' \in S} P(s' | s,a)*V(s'),\; \forall a\in A, s \in S$$
It should be apparent that if we find the smallest value of $V(s)$ that matches this requirement, then that value would make exactly one of the constraints tight.

If you formulate your linear program by writing a program like the one above for every state and then minimize $\sum_{s\in S} V(s)$, subject to the union of all the constraints from all these sub-problems you have reduced the problem of learning a value function to solving the LP.


Mathematical Analysis of the Loss function of GAN

I guess the issue is you lost track of where the samples came from and since you requested a math explanation I'll try to go step by step using my notation and without checking other material to avoid being biased by how other authors present it
So we start from
$$ L(D,G) = E_{x \sim p_{r}(x)} \log(D(x)) + E_{x \sim p_{g}(x)}\log(1 - D(x)) $$
then you apply the definition of $E_{\cdot}(\cdot)$ operator in the continuous case
$$ L(D,G) = \int_{x} \log(D(x)) p_{r}(x)dx + \int_{x}\log(1 - D(x))p_{g}(x)dx $$
then you Monte Carlo sample it to approximate it
$$ L(D,G) = \frac{1}{n} \sum_{i=1}^{n} \log(D(x_{i}^{(r)}))  + \frac{1}{m} \sum_{j=1}^{m}\log(1 - D(x_{j}^{(g)})) $$
As you can see here I have kept the samples from the 2 distributions separated and used a notation that allows to track their origin so now you can use the right label in the Cross Entropy
$$ L(D,G) = \frac{1}{n} \sum_{i=1}^{n} L_{ce}(1, D(x_{i}^{(r)}))  + \frac{1}{m} \sum_{j=1}^{m} L_{ce}(0, D(x_{j}^{(g)})) $$
But you could also have decided to merge the 2 integrals before to have
$$ L(D,G) = \int_{x} \left( \log(D(x)) p_{r}(x) + \log(1 - D(x))p_{g}(x) \right) dx $$
which is mathematically legit operation, however the issue is when you try to
discretize this with Monte Carlo sampling.
You can't just replace the integral with one sum since you are Monte Carlo sampling and here, contrary to what we have done above, you do not have 1 distribution per integral to sample but in the same integral you have 2 distributions and for each sample you have to say what distribution it comes from which is where the issue is in your notation since you lost track of this information and it seems all the samples come from one distribution


Are Genetic Algorithms suitable for problems like the Knuth problem?

Before trying to answer your question more directly, let me clarify something.
People often use the term genetic algorithms (GAs), but, in many cases, what they really mean is evolutionary algorithms (EAs), which is a collection of population-based (i.e. multiple solutions are maintained at the same time) optimization algorithms and approaches that are inspired by Darwinism and survival of the fittest. GAs is one of these approaches, where the chromosomes are binary and you have both the mutation and cross-over operation. There are other approaches, such as evolution strategies or genetic programming.
As you also noticed, EAs are meta-heuristics, and, although there is some research on their convergence properties [1], in practice, they may not converge. However, when any other potential approach has failed, EAs can be definitely useful.
In your case, the problem is really to find a closed-form (or analytical) expression of a function, which is composed of other smaller functions. This really is what genetic programming (in particular, tree-based GP) was created for. In fact, the Knuth problem is a particular instance of the symbolic regression problem, which is a typical problem that GP is applied to. So, GP is probably the first approach you should try.
Meanwhile, I have implemented a simple program in DEAP that tries to solve the Knuth problem. Check it here. The fitness of the best solution that it has found so far (with some seed) is 4 and the solution is floor(sqrt(float(sqrt(4)))) (here float just converts the input to a floating-point number, to ensure type safety).  I used the difference as the fitness function and ran the GP algorithm for 100 generations with 100 individuals for each generation (which is not a lot!). I didn't tweak much the hyper-parameters, so, maybe, with the right seed and hyper-parameters, you can find the right solution.
To address your concerns, in principle, you could use that encoding, but, as you note, the GA could indeed return $4$ as the best solution (which isn't actually that far away from $5$), which you could avoid my killing, at every generation, any individuals that have just that value.
I didn't spend too much time on my implementation and thinking about this problem, but, as I said above, even with genetic programming and using only Knuth's operations, it could get stuck in local optima. You could try to augment my (or your) implementation with other operations, such as the multiplication and addition, and see if something improves.


Is it possible to classify songs by genres based on spectrograms?

It is possible to use spectrograms for genres classification. See relevant articles Music genre recognition using spectrograms or blog post here. The later link did use CNN for this task.

If you need training data, GTZAN or FMA could be your starting point.


Policy invariance under affine transformations of the reward function?

This statement:

  (it is well-known that the optimal policy is invariant to positive affine transformation of the reward function).

is, as far as I know, and as you summarise, incorrect because simple translations to reward signal do affect the optimal policy, and the affine transform of a real number $x$ can be given by $f(x) = mx + c$

It is well known that optimal policy is unaffected by multiplying all rewards by a positive scaling, e.g. $f(x) = mx$ where $m$ is positive.

It is also worth noting that if an optimal policy is derived from Q values using $\pi(s) = \text{argmax}_a Q(s,a)$, then that policy function is invariant to positive affine transformations of action values given by $Q(s,a)$. Perhaps that was what the paper authors meant to write, given that they go on to apply normalisation to Q values.

The impact of the mistake is not relevant to the the rest of the paper as far as I can see (caveat: I have not read the whole paper).


Divide classes into truncated and non-truncated objects

It could work. I think itll be hard to find labelled bounding box data describing full/truncated cars so this is a good idea for self labelling.

Theres probably also some other ways for you to get labelled data without the crowd sourcing or self labelling. Take a large labelled car dataset with bounding boxes (COCO has some, and if your good with aerial imagery theres alot more out there), and just enforce that any car bounding boxes that are touching the boundary are truncated, while ones fully encompassed in the imaged are not, since with high probability that will be the case.

Good luck!


Reinforcement Learning (RL) how to obtain $p(s',r|s,a)$

The function $r(s,a,s')$ gives the expected reward in each scenario, but not the distribution of rewards that lead to values $r_{search}$ and $r_{wait}$

The text explains that reward is $+1$ for each can found, and that different distributions of numbers of cans are expected when waiting as opposed to searching. However, it does not give any description of the actual distributions, just summarises them as the two expected rewards, and suggests $r_{search} \gt r_{wait}$

You have two main ways to answer the exercise:

Invent some parameters for the distributions of $r_{search}$ and $r_{wait}$ in order to split up single values of $p(s'|s,a)$ into multiple values of $p(s', r|s,a)$. E.g you could decide that $r_{search}$ consists of $0  \eta_0 + 1  \eta_1 + 2 \eta_2 + 3 \eta_3$ where $\eta_0, \eta_1, \eta_2, \eta_3$ are probabilities that sum to $1$ - each row that currently has $r_{search}$ as the output of $r(s,a,s')$ would then split into 4 rows with reward 0, 1, 2, 3 to complete the new table . . . $r_{wait}$ would need a different set of parameters.
Ignore the details of the distribution, move column $r(s,a,s′)$ to the left and call it $r$, changing $p(s|s,a)$ to $p(s', r|s,a)$. It might be all that's expected given the lack of information.

My personal opinion is that the authors want you to think about solution 1 - the only issue is that it requires you to invent some new parameters that were not provided. The ones I name are only a suggestion, they do not represent a specific "correct" answer in terms provided by the book, because the book omits those details.

As an example to start with, if you start solution 1, and use parameters as I have labelled them, you will end up with a first row looking like this:

$s\qquad \qquad a\qquad  \qquad  s'\qquad  \qquad   r \qquad       p(s', r| s, a)$
$high  \qquad \quad search  \qquad  high  \qquad \quad   0  \qquad   \alpha \eta_0$


Can a neural network work out the concept of distance?

Answer

I'm going to take your question at face value, and go really deep into this topic.

Yes, they can. The typical human mind can. But consider the human mind. Millions, if not billions, of neurons. In fact, one can consider distance as a human concept, simply a theory developed from interactions with the world.

Therefore, given a year or two, with a ton of neurons on your hand, you could replicate this scenario. That is if your computer is as parallel as the human mind. The short explanation is that the human mind is very parallel.

However, it would be simpler to calculate the distance with a program, not an AI, and simply feed the result to the AI that would make the decisions.

Consider the amount of time you have spent looking at a screen. If you can tell the (approximate) distance between two pixels, so can a Neural Network, as you are one. However, add the amount of time you have spent alive and learning into the equation, and it becomes a disaster.

Further reading

The human brain is parallel

This is a result of the fact that all of the neurons in the human brain are independent of each other. They can run true simultaneous actions, thus making the action of interpreting images and such much easier, as blocks of neurons can "think" independent of the operations of the others, limiting what would be "lag" to a minuscule amount.


Why can't we combine both training and validation data, given that both types of data are used for developing the model?

Usually we are splinting the training dataset because we want to fine tune, find the best hyper-parameters, for our model. If we combine the validation and training dataset given a network complex enough we could achieve perfect performance for the given task. But having very good performance on the training dataset does not mean our model is useful. It might be the case that our model is very good on the training dataset, but it fails to generalize, so when given samples outside of the training dataset the performance will significantly drop (this is called over-fitting). By splitting the dataset, we are testing how well our model is performing on new/unseen data.


Will Q-learning converge to the optimal state-action function when the reward periodically changes?

No, it will not converge in the general case (maybe it might in extremely convenient special cases, not sure, didn't think hard enough about that...).

Practically everything in Reinforcement Learning theory (including convergence proofs) relies on the Markov property; the assumption that the current state $s_t$ includes all relevant information, that the history leading up to $s_t$ is no longer relevant. In your case, this property is violated; it is important to remember whether or not we visited $s$ more recently than $s'$.

I suppose if you "enhance" your states such that they include that piece of information, then it should converge again. This means that you'd essentially double your state-space. For every state that you have in your "normal" state space, you'd have to add a separate copy that would be used in cases where $s$ was visited more recently than $s'$.


This is a simple digital image processing task. I am proposing a quite efficient solution without any error correction mechanism, so this is not a production quality approach. But as a first proof of concept it should do the trick.

The easiest approach would be splitting the image in two channels - one for the color red and one for the color black. You can neglect the rest outside a certain threshold. If the lighting is good, this should also deal with the background.

Step 1: Crop away everything outside the gauge to simplify your life. The easiest way to do this is just putting the camera close enough that it only captures the gauge. That way you don't have to implement it in your algorithm.

Step 2: The center of your image will be the red dot in the middle of the gauge. You can easily identify it with a filter looking for the largest concentration of red in the red channel of your image. Calculate the center and remember the coordinates.

Step 3: Do the same for the largest concentration of black in the black channel. This will be the thick end part of the black needle. Calculate the center as well and remember the coordinates.

Step 4: Calculate the vector between those two coordinates. The angle of the vector will tell you the angle of the needle (take care of the direction of the vector). Once you have the angle of the needle, you can easily derive the number it is pointing to.

There are obviously much more elegant approaches to solve this problem, but this quick and dirty algorithm should do the trick without a lot of hassle. You need to make sure that the camera is positioned in the right angle, because we haven't implemented any error corrections in our algorithm. You also need to make sure the lighting is sufficient and that there aren't any other significant influences on the image quality. As long as the orientation is stable and the image quality high, this should enable you to read the gauge with good precision.


How to load an image into tensorflow.js code which reads handwritten numbers and clasify them

To test the model, you have to achieve a Tensor with the same dimensions as the training data. This Tensor should be the same for the same image, no matter how many times you remake the Tensor with the same image. That being said, you can attempt to reverse-engineer how the Tensors are being made. Hint: The dimensions of the input tensor should be (28, 28, 1).


How does an LSTM output the correct dimensions for classes?

After some more research, I have found the answer.

An LSTM is comprised of 1 LSTM cell that is continuously updated by passing in new inputs, the hidden state and the previous output. All nodes inside the LSTM cell are of size hidden_units. That means the output of the activation gate, input gate, forget gate and output gate are all of size hidden_units. As an extension of this, the size of the hidden state is also equal to hidden_units.

The problem arises when the desired output is not equal to that of hidden_units. This is fixed simple and easily, by slapping a basic feed forward net onto the end of the output of the LSTM, so even if it outputs 256, you can convert it to n nodes. These final output nodes are typically put through a softmax with cross-entropy loss applied.

I came to the conclusion by looking at the code included here: https://towardsdatascience.com/lstm-by-example-using-tensorflow-feb0c1968537

In which you can see the final output has its own separate weights and bias'.


The output of YOLO is (x,y,w,h,confidence,class). The confidence value presents whether the rectangle holds an object, the rectangle is non-classed when confidence is low.
The class value will be used, only when confidence is high.


How to perform back-propagation in Decoupled Neural Interfaces?

I kid you not when I tell you I just started with decoupled networks this morning. But the issue is terribly worded by most computer scientists. The way a decoupled neural interface works is not by sending the error backwards, but by storing it in a form of learned error inside a separate network. So not only do we have our nodes and weights, but also the Synths. The Synths modify the weight tables using past error. So instead of passing error back like the article suggests, it actually is passing it forward in a kind of "hey don't do this." What's really fun is that to you never have to mix up your thinking to reverse a network for backprop if you're using a decoupled network. The amazing part is that it is actually easier to code this decoupled network, than it is to write a "simple" ANN, even better still is that it functions a lot like an LSTM network without all the programming mumbo-jumbo. You can see for yourself two different forms of the same network from here.

The code from the demo in the repository is from https://towardsdatascience.com/only-numpy-implementing-and-comparing-combination-of-google-brains-decoupled-neural-interfaces-6712e758c1af it was a fundamental resource for me, and it may be for you. It also helps you with loss, or cost.


A neural network to learn the connection between two images

It is possible to have both input and output be images that differ in a predictable way. For example, architectures similar to autoencoders have been used to remove blur, change weather conditions, change between day and night photos etc. In these architectures, the training data is matching pairs of images. If your goal is to replicate some image enhancement, then often the input is artificially processed e.g. to reduce its quality in a hard to reverse way. A good example of this would be to remove distortion or noise from an image.

You can also use generative models. These are  harder to get working, but can be more flexible in that you don't need image pairs in order to train, just a set of images labelled with the traits that you want to learn. Converting an image using a generative model involves using an encoder stage to get its embedding, altering the embedding based on label you require and then feeding the new embedding into the decoder stage. This is how you might alter a face portrait from male to female, or young to old, because it is not possible to find good natural image pairs for that task.


Best way to create an image dataset for CNN

I found this solution and will be happy about any improvements or suggestions:

First, I create a random dataset of images, which are 28x28 pixels, and corresponding random labels (just for sake of clarification, I have another image dataset, this is just for explaining). Then I use a sklearn module for splitting the data:

import numpy as np
from sklearn.model_selection import train_test_split

# create the data and labels
def pixel_dataset(n_data=10, dpi = 28):

    ary = np.zeros((n_data, dpi, dpi))
    label = []

    for i in np.arange(n_data):
        labels =['a', 'b', 'c']
        ary[i,::] = np.reshape(np.array(np.random.randint(0, 255, 28*28)),(28,28))
        label.append(np.random.choice(labels, 1)[0])

    return ary, label

# create the test train split
def mydata(test_size=0.3):

    X, y = pixel_dataset()
    X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = test_size)
    return (X_train, y_train), (X_test, y_test)


Can an evolving AI be programmed not to lie?

If a machine learning based AI is "sufficiently smart enough" to be able lie then there is nothing preventing it from lying. This does not mean it can't be persuaded from lying.

So just make the AI simple enough to not be able to lie.

The reasoning here is that in order for a system to be able to lie, a system must be able to recognize an incentive to lie. Recognizing this incentive is a challenging function and would be impossible to code manually into a computer. Machine leaning can be applied to problems such as these where the function is hard to code manually. Although there has been promising work on understanding what the representations/features learned by machine learning actually represent, it may not be possible in general to have an understanding of what a lie in the agent's representation looks like. Because of this, having a hand coded rule to catch when an agent is lying is not possible and thus being able to prevent an agent from (or catch an agent when) lying isn't possible when using machine learning.


What are the keys and values of the attention model for the encoder and decoder in the "Attention Is All You Need" paper?

BUT there are 2 different attention layers and one of which do not use the encoder’s output at all. So, what are the keys and values
now?

The first attention layer in the decoder is the "Masked Multi-Head Attention" layer and is the self-attention layer, calculating how much each word is related to each word in the same sentence. However, our aim in the decoder is to generate the next French word and so for any given output French word we can use all the English words but only the French words previously seen in the sentence. We, therefore, "mask" the words that appear later in the French sentence by transforming these to 0 so the attention network cannot use them.

How these 3 words are converted to 4 words

The second attention block in the decoder is where the English to French word mapping happens. We have a query for every output position in the French sentence and a key/value for every English input word. We calculate relevance scores from the dot product of the query and key and then obtain output scores for each predicted word from multiplying the relevance and value. The following diagram is useful to visualise how, for each predicted word, we can have relevance scores that can predict that one English word can be translated to multiple, or no French word.

In summary, the encoder discovers interesting things about the English sentence whilst the decoder predicts the next French word in the translation. It should be noted they use "Multi-Head Attention", meaning that a number (8 in the original paper) of attention vectors are calculated to learn attention mechanisms to pay attention to different things, for example, grammar, vocabulary, tense, gender, and the output is a weighted average of these.


As complexity rises, precise statements lose meaning and meaningful statements lose precision. ( Lofti Zadeh ).

Fuzzy logic deals with reasoning that is approximate rather than fixed and exact. This may make the reasoning more meaningful for a human:

Fuzzy logic is an extension of Boolean logic by Lotfi Zadeh in 1965 based on the
mathematical theory of fuzzy sets, which is a generalization of the classical set theory.
By introducing the notion of degree in the verification of a condition, thus enabling a
condition to be in a state other than true or false, fuzzy logic provides a very valuable
flexibility for reasoning, which makes it possible to take into account inaccuracies and
uncertainties.

One advantage of fuzzy logic in order to formalize human reasoning is that the rules
are set in natural language. For example, here are some rules of conduct that a driver
follows, assuming that he does not want to lose his driver’s licence:

Intuitively, it thus seems that the input variables like in this example are approximately
appreciated by the brain, such as the degree of verification of a condition in fuzzy
logic.

I've written a short introduction to fuzzy logic that goes into a bit more details but should be very accessible.


How to find optimal mutation probability and crossover probability?

As @Oliver Mason says, picking the parameters that control the behavior of a GA (which are sometimes called "hyperparameters") is historically more of an art than a science.

The evolutionary computation literature has many theories about the merits of high vs. low mutation, and high vs. low crossover. Most practitioners I have worked with use either high crossover, low mutation (e.g. Xover = 80%, mutation = 5%), or moderate crossover, moderate mutation (e.g. Xover = 40%, mutation = 40%).

In more recent years, the field of hyperparameter optimization has emerged and focuses on developing automatic approaches to picking these parameters. A very simple example of hyperparameter optimization is the GridSearchCV function in ScikitLearn. This systematically tries every combination of, say, 10 crossover values with evey one of 10 mutation values, and reports on which one works best. It uses Cross Validation to prevent overfitting during this process. A more complex approach is Bayesian Hyperparameter Optimization, which performs a sort of optimal experiment design to uncover the best values using as few tests as possible. This approach has been quite successful in tuning the hyperparamters of deep neural networks, for example.


How to implement RL policies learned on a finite horizon?

A normal way to deal with training on infinite horizon (aka "continuing" or "non-episodic") problems is to use TD learning or other bootstrapping methods (of which Q-learning in DQN is one example), and to treat the cutoff at $T$ for pseudo-episodes as a training artefact.
If the state at time $T$ was really a terminal state, the TD target would be just $r$ because $q(s^T,\cdot) = 0$ by definition, but that doesn't apply in your case.
So always use the bootstrapped TD target - e.g. $r + \gamma \text{max}_{a'} \hat{q}(s',a',\theta)$ for single step TD target with a Q estimate having $\theta$ as learned params - and don't treat the horizon data any differently.
If you do this, then your concerns about under-estimates should not be an issue. Your pseudo-episodes do need to be long enough to observe the impact of multiple requests in the long term is the main issue (setting $T$ too low so that the system does not reach any kind of equilibrium would be a problem).
You could also use an average reward setting and differential value functions. It is slightly better from a theoretical standpoint, but Q-learning and DQN is fine if you don't want to be bothered with that. The same basic rule applies - ignore "episode end" for constructing TD targets because it is just a training artefact. Also ensure you set $T$ high enough that the long term impacts of a policy are observable.
If your starting state is special (e.g. cars all in fixed places, and no requests in progress) this could also be an issue, because the real world system will rarely be in that state but you will have many episode starts with it. If you cannot start the system in a reasonable random initial state for your problem, you may also want to discard data from episode starts and allow a run-in time before using data for training from the pseudo-episodes.


What are the risks associated with regulating AI?

I don't think regulating something necessarily causes that regulation to defacto become a "risk".

Regulation - including overregulation - may, in fact, aid in the dialogue between practitioners, which may end up educating the regulators, the public and the practitioners themselves.

My answers to your survey would most likely be "it depends...", or "no risk", which isn't to say it's not an impediment, but just not a "risk", per se.


Agents meeting in a directed connected graph

It's not possible to solve version 1) of the problem in general. To see why, consider a graph with 2 cities, and 2 agents, where the agents start in opposite nodes. Since both agents need to move every turn, they will never meet in the same city.

For version 2), I'm going to make some assumptions that aren't completely clear from your text:

Agents have only local information. They can't see the global structure of the graph, and just all agree to meet at city X, because they don't know that X exists. They also can't see where other agents are initially.
Cities are labelled, or otherwise identifiable to the agents. That is, an agent can tell whether they've visited a city before or not.
Agents can tell how many agents are in their current city, and in all adjacent cities.
There are a finite number of cities, and it's always possible to get from city A to city B, for any pair of cities (A,B).
Agents have no way to communicate with each other, except by being present in the same space or not.

In this setup, the following heuristic will always work:

If you've never seen another agent, move to a random adjacent city, or stay in place, with a uniform probability of taking each possible action.
If you've never seen another agent before, but you see one or more agents now, then switch permanently to the following strategy:

If you can see an adjacent city with more agents than your city, move there now. If you can see several such cities, move to the largest one. Pick randomly in case of a tie.
If an adjacent city has the same number of agents as yours, flip a coin to decide whether to go or stay. Eventually, one city ends up with more than the other, and then step 1 causes all agents to end up in the same city.
Otherwise, if you can see other adjacent cities with fewer agents, just stay put. They'll come to you.
If all adjacent agents are empty and there's a total of $n$ agents you can see in your city right now, then remember $n$.
As long as you still can't see more than $n$ agents in total, stay put with probability $(n-1)/n$ and move to a random adjacent city with probability $1/n$.
If you stayed put per step 5, and you still can only see n agents, and all but one of them is in your current city, move to the city with the single agent who moved. Otherwise, if 0 or more than 1 agent moved, stay put. The agents who moved come back, and we go to step 5.
After successfully moving, go to step 1 again.

You can see that once agents meet up in a group, that group never shrinks. Further, groups of agents are always trying to move around together. You can prove that a group moves an average of about $\frac{1}{2e}$ every step. It may take a while, but eventually every group will bump into every other group, resulting in all the agents being in the same city.


Create optimizer object using the tf.keras.optimizers.get function

In order to have a configurable optimizer and configurable hyperparameters for it you need to make the function call in the following way:

opt = tf.keras.optimizers.get({"class_name": hyperparams['optimizer'],
                               "config": {"learning_rate": hyperparams['learning_rate']}})

where class_name is the name of the optimizer you want to use and the config dictionary contains the hyperparameters and their values for the given optimizer.


Minimax deals with two kinds of values:

Estimated values determined by a heuristic function.
Actual values determined by a terminal state.

Commonly, we use the following denotational semantics for values:

A range of values centered around 0 denote estimated values (e.g. -999 to 999).
A value less than the smallest heuristic value denotes a loss for max (e.g. -1000).
A value more than the biggest heuristic value denotes a win for max (e.g. 1000).
The value 0 denotes either an estimated draw or an actual draw.

The advantage of this denotational semantics is that comparing values is the same as comparing numbers (i.e. you don't need a special comparison function). We can extend this denotational semantics to incorporate optimality of winning and losing as follows:

A range of values centered around 0 denote estimated values (e.g. -999 to 999).
A range of values less than the smallest heuristic value denote loss (e.g. -2000 to -1000).
A range of values more than the biggest heuristic value denote win (e.g. 1000 to 2000).
The value 0 denotes either an estimated draw or an actual draw.
A loss in n moves is denoted as -(m - n) where m is a sufficiently large number (e.g. 2000).
A win in n moves is denoted as (m - n) where m is a sufficiently large number (e.g. 2000).

Using this denotational semantics for values requires only a small change to the minimax algorithm:

function minimax(node, depth, max)
    if max
        return negamax(node, depth, 1)
    else
        return -negamax(node, depth, -1)

function negamax(node, depth, color)
    if terminal(node)
        return -2000

    if depth = 0
        return color * heuristic(node)

    value = -2000

    foreach child of node
        v = -negamax(child, depth - 1, -color)

        if v > 1000
            v -= 1

        if v > value
            value = v

    return value

Incorporating optimality for draws is a lot more difficult.


Why is symbolic AI not so popular as ANN but used by IBM's Deep Blue?

You might also ask if there's any particular reason why we would use a neural net. If we're to train a neural net to play chess, we need to be able to:
1. Feed it positions as input vectors (easy enough),
2. Decide on an output format. Perhaps a distribution over possible moves (but then, how to represent that such that the meaning of a specific output cell doesn't change drastically based on the board state?  Or perhaps instead, we let the resulting board state after a candidate move be the input, and let the output be a score that represents the desirability of that state. That'll require exponentially more forward/backprop passes, though.
3. Provide it with an error signal to whatever output vector it produces. This is the really tricky bit, since we don't know whether a given move will result in victory until the very end.
Do we play the game to the very end, storing decisions as we go, and then at the end, replay each input, feeding it an error signal if we lost? This will give the same error to the good moves as to the ones that actually lost the game. With enough games, this will work, since the good moves will get positive feedback a bit more often than negative, and vice versa for the bad ones. But it'll take a lot of games. More than a human is going to be willing to play.  We can have different networks learn by playing against each other, but not on 1996 hardware.
Do we instead provide a score based on another heuristic of the board state? In that case, why not just use minimax? It's provably optimal for a given heuristic up to however many moves deep we look, and it doesn't need training.

Add to this the fact that if we don't choose a good representation at each of these steps, there's a good chance that the network will only learn the positions it's specifically been trained on, rather than generalizing to unseen states, which is the main reason for using a neural network in the first place.
It's certainly possible to use neural nets to learn chess (DeepMind's approach can be found here, for instance), but they're not a natural fit to the problem by any means. Minimax, by contrast, fits the problem very well, which is why it was one of the techniques used by Deep Blue. Neural nets are an amazing tool, but they're not always the right tool for the job.
Addendum:
I didn't stress this point much, since K.C. already brought it up, but training large neural nets require us to perform a huge number of matrix vector multiplications, and this wasn't especially practical before GPUs got powerful and cheap.


How could we solve the TSP using a hill-climbing approach?

I will give you a basic idea of an approach.

The basic idea behind hill climbing algorithms is to find local neighbouring solutions to the current one and, eventually, replace the current one with one of these neighbouring solutions.

So, you first need to model your problem in a way such that you can find neighbouring solutions to the current solution (as efficiently as possible). But what is a solution to a TSP? It is a sequence of nodes (or vertices), such that the first node is equal to the last one (given that the travelling salesman needs to return to its initial position), no other vertex is repeated, and all vertices of the graph are included.

So, given a sequence of nodes $x_1, x_2, \dots, x_{n}, x_1$, how can you create a neighbouring solution that is valid, that is, there is not repeated vertex (apart from the initial and the last one) and all vertices are included?

Hint: recall that, in the case of TSP, we can assume that every node (or city) is connected to every other node.


What are the main differences between skip-gram and continuous bag of words?

So as you're probably already aware of, CBOW and Skip-gram are just mirrored versions of each other. CBOW is trained to predict a single word from a fixed window size of context words, whereas Skip-gram does the opposite, and tries to predict several context words from a single input word.

Intuitively, the first task is much simpler, this implies a much faster convergence for CBOW than for Skip-gram, in the original paper (link below) they wrote that CBOW took hours to train, Skip-gram 3 days.

For the same logic regarding the task difficulty, CBOW learn better syntactic relationships between words while Skip-gram is better in capturing better semantic relationships. In practice, this means that for the word 'cat' CBOW would retrive as closest vectors morphologically similar words like plurals, i.e. 'cats' while Skip-gram would consider morphologically different words (but semantically relevant) like 'dog' much closer to 'cat' in comparison.

A final consideration to make deals instead with the sensitivity to rare and frequent words. Because Skip-gram rely on single words input, it is less sensitive to overfit frequent words, because even if frequent words are presented more times that rare words during training, they still appear individually, while CBOW is prone to overfit frequent words because they appear several time along with the same context. This advantage over frequent words overfitting leads Skip-gram to be also more efficient in term of documents required to achieve good performances, much less than CBOW (and it's also the reason of the better performances of Skip-gram in capturing semantical relationships).

Anyway, you can find some comparisons in the original paper (section 4.3).
Mikolov et al. 2013

About the architecture, there's not much to say. They just randomly initialised word embedding for each word, then a projection matrix NxD (number of context words times embeddings dimension) is generated at each iteration, there is no hidden layer, the vectors are just averaged together and then fed into an activation function to predict index probabilities in a vector of dimension V (the size of the vocabulary).

For a more specific explanation (even Mikolov's paper lack some detail) I suggest checking this blog page (Words as Vectors), even though the model described there do apply a hidden layer, unlike the original architecture.


How to source training data in ML for information security?

You have implicitly assumed that supervised learning is being used, given the assumption that labels are needed. But this might lead to the following potential problems:

Log file data tends to be huge, and it may be infeasible to label due to the time/expertise required;
Then there's the class imbalance problem, in that attack examples are far far rarer than "normal" behaviour, and this can mess up supervised models during both learning and evaluation stages;
And finally even if the data is labelled, a supervised model is unlikely to be useful in detecting completely novel attacks because it has not been trained to recognise these.

I think a far easy way to approach these kinds of problem would be unsupervised learning: model normal patterns and behaviours in the logs, and then flag any deviations from normality. It may be an attack or it may be new normal behaviour. In the latter case, the model can be updated. There are various approaches here such as clustering, outlier detection and possibly self-supervised learning that might be useful. Dimensionality techniques might also be useful to visualise clusters of "normal" behaviour that can be compared to abnormal patterns.


Factors that causing totally different outcomes from an exactly same model and datasets

By default, Keras sets shuffle argument True, so you should set the numpy seed before importing Keras.
CPU
from numpy.random import seed
seed(25)
from keras.models import Sequential

GPU
tf.random.set_seed(seed)


Zero shot learning available labels in testing set

The formal definition of zero-shot learning is that given labeled training instances $D_{tr}$ belonging to the seen classes $S$, the aim is to learn a classifier $f^u(·):X→U$ that can classify testing instances $X_{te}$ (i.e., to predict $Y_{te}$) belonging to the unseen classes $U$.
The general idea of zero-shot learning is to transfer the knowledge contained in the training instances $D_{tr}$ to the task of testing instance classification. So it is considered a transfer learning method, and, more specifically, heterogeneous transfer learning with different label spaces.
However, since no labeled instances belonging to the unseen classes are available,
to solve the zero-shot learning problem, some auxiliary information is necessary. Such auxiliary information should contain information about all of the unseen classes. This auxiliary information usually contains some semantic information about the unseen classes, and its representation belongs to a space that is often referred as $semantic$ $space$. In the semantic space, each class has a corresponding vector representation, which is referred to as the class prototype or $prototype$ for short.
So, you should be aware of classes of your problem, in order to construct the semantic space and their class prototypes. I also suggest you read the survey paper "A survey of zero-shot learning: settings, methods and applications".


Is a state that includes only the past n-step price records partially observable?

An environment is partially observable if the agent cannot fully observe the current state but it only partially observes it. More specifically, in fully observable MDPs (FOMDPs), the agent knows the current state of the environment, which can or not (for example, depending on whether the state is Markov or not) contain all the information required to theoretically take the optimal action. In a partially observable MDPs (POMDPs), the state is not available or observable, so the agent might not possess all the required information to theoretically take the optimal action.

For example, in the game of poker, there are hidden and observable cards. In this game, the state of the environment could be defined as all the hands of all players, the cards on the table and the next cards that will be drawn from the deck until the end of the round. If a player had access to all this information, then the environment would be fully observable, but, usually, this information is not available, so poker is usually considered a partially observable game.

You defined the state as follows

$$s_t = \{ p_{t:t-n}, E_t\}$$

So, if you have access to the past $n$ price records, $p_{1:n}$, and the energy level, $E_t$, then your environment is fully observable. However, this does not mean that it is a Markov environment, that is, intuitively, that your current state is sufficient to theoretically take the optimal action.

To conclude, partial observability depends on the definition of state in a specific problem. The concept of Markov property is also related to the concept of partial observability. However, in the case of MDPs, the concept of Markov property has a specific definition: given the current state and an action, the probability of the next state is conditionally independent of all previous states and actions or, more formally

$$
P(S_{t+1} \mid S_t, A_t) = P(S_{t+1} \mid S_t, S_{t-1}, \dots, S_1, A_t, A_{t-1}, \dots, A_1)
$$

where $S_{t+1}$ is the next state, $S_t$ the current state, $A_t$ the current action, $S_{t-1}, \dots, S_1$ the previous states and $A_{t-1}, \dots, A_1$ the previous actions.


Should deep residual networks be viewed as an ensemble of networks?

Imagine a genie grants you three wishes. Because you are an ambitious deep learning researcher your first wish is a perfect solution for a 1000-layer NN for Image Net, which promptly appears on your laptop.

Now a genie induced solution doesn't give you any intuition how it might be interpreted as an ensemble, but do you really believe that you need 1000 layers of abstraction to distinguish a cat from a dog? As the authors of the "ensemble paper" mention themselves, this is definitely not true for biological systems.

Of course you could waste your second wish on a decomposition of the solution into an ensemble of networks, and I'm pretty sure the genie would be able to oblige. The reason being that part of the power of a deep network will always come from the ensemble effect.

So it is not surprising that two very successful tricks to train deep networks, dropout and residual networks, have an immediate interpretation as implicit ensemble. Therefore "it's not depth, but the ensemble" strikes me as a false dichotomy. You would really only say that if you honestly believed that you need hundreds or thousands of levels of abstraction to classify images with human accuracy.

I suggest you use the last wish for something else, maybe a pinacolada.


Would it take 1700 years to run AlphaGo Zero in commodity hardware?

Although the above statement holds important analogies to communicate the technical advances made by deep mind in the development of Alpha Go. It is inaccurate and should be taken skeptically.

Firstly, although Alpha go was trained on specialized hardware such as high end NVidia GPU's and custom google TPU's. It should be noted that it can run on a regular desktop although it won't be as powerful as the distributed version of Alpha Go. Additionally anyone with a laptop can access a similar amount of computing resources in the cloud with the touch of a button.

Although a version of Alpha Go indeed ran on 1202 CPUs and 176 high end GPU's as reported by Nature magazine. Which roughly translates to a system 1000 times as powerful as a commodity laptop. We need to consider other important factors such as moores law which postulates that we could have a computer as fast as 1202 CPU's in our pockets within 20 years and not several millennia.

Furthermore, the latest version of Alpha Go, Alpha Go Zero, trained on a single server with only 4 TPU's. Considering that the latest generation of smartphones such as iPhone X and Huawei's Kirin come packed with specialised AI chips. I will not be surprised if a similar reduction of form factor is achieved in commodity desktop computers once new models packed with AI chips are introduced.

I respect and acknowledge the technical achievements of Demis Hassabis and the deep mind team in developing a system as powerful as Alpha Go. However I believe analogies such as the one used above are inaccurate and mis-represent facts.


What is chaotic behavior and how it is achieved in non-linear regression and artificial networks?

It looks like you have some common misconceptions about AI and neural networks.

First, AI programs generally do not try to imitate the human behaviour of a human brain. Instead, they try to imitate some higher-level behaviour. For example, they might imitate the reasoning process that you go through when you make a plan. In this context, the building-blocks (silicon or flesh) don't matter too much.

Second, artificial neural networks are also (mostly) not intended to imitate the human brain. Although they are inspired by the arrangement of neurons in a human brain, the networks used in most ANN systems have very little to do with real brains. The main similarity is that both systems have a lot of simple little computational units connected in patterns such that signals passed from one to another lead to interesting computations. However, real neurons produce lots of different kinds of signals, are connected in arbitrary ways, and randomization and transmission times play a significant role. Artificial neurons generally are deterministic, produce only one kind of signal (or sometimes a couple different kinds), are connected in extremely regular ways, and usually simulate instantaneous transmissions between neurons.


How important is true randomness in AI designs?

Randomness is typically the best one can do with ignorance, rather than a source of strength in its own right.

For example, the primary use of randomness in statistics is random assignment (A/B testing, randomized controlled trials, etc.). The reason to do this is to make the influence of confounders independent from the influence of the factor under investigation.

But randomness only works for this in expectation. If we actually knew what the confounders were, we could do a paired assignment (or a similar scheme) that ensured the various groups were matched as well as possible, instead of us just not knowing ahead of time which way the bias went.

There are some cases where pseudorandomness, rather than full randomness, will impair training AI designs. A simple example would be a case where you want to randomly initialize weights in a network where the number of parameters exceeds the periodicity of the RNG; this means that while you have as many possible networks as there are possible unique seeds, you can't actually visit the entire weight space that you wanted to sample over.

I don't think any of those cases are limiting factors, however. Having truly random stochastic gradient descent instead of pseudorandom stochastic gradient descent doesn't seem like it would make a serious difference in the trajectory of AI designs.


How to find proper parameter settings for a given optimization algorithm?

How to find the best configuration for an algorithm is an open research question in AI. The topic in general is known as `hyper-parameter optimization' and there are a range of possible methods:

One of the most popular is IRace, but other possibilities include:

Spearmint: uses wrappers in Matlab or Python. It uses MongoDb,
and Bayesian optimisation algorithms.
SMAC requires a python wrapper for the algorithm to be optimized
and has a command line interface.
Hyperopt: a Python library which uses Random Search and Tree of
Parzen Estimators.

This paper argues that Spearmint performs the best, compared with SMAC and Hyperopt, but with significantly longer running times in some cases.


How do you interpret this learning curve?

Depends on what does 1 represent in your task.
If you are trying to predict household prices and 1 represents \$1, I think the average validation loss is good. If 1 represents \$10000 in this case, probably something is not right. But remember that there are 2 parts contributing to the overall loss. The mse loss and the l2 penalty loss. (Also remember that most optimizers already implement l2 penalty as weight decay. So you do not need to add it separately)

Some suggestions.

Check if your data has any outliers/ anomalies. Based on your task you should know as to what you can do with these data points. Also see if your dataset has high variance.
If you are worried about over-fitting, think about your data again. Less data + more parameters often leads to over-fitting. If your dataset is too small, you need to think again.
Try to adjust the number of hidden units and observe the results.
Try using cross validation.
Alternatively, try using different optimizers and see what happens (Try Adam).


Is there a shortage of machine learning and artificial intelligence professionals?

In one word – yes, there is!

If you are following the industry reports then you must know that more than 500,000 jobs are lying vacant because there is a huge shortage of qualified talent in the industry.  It is also quite interesting to note that the market has almost double the number of jobs available than the required talented ML and AI professionals – just in case – there is someone out there.

Another thing to note is that the countries like US, China, India, Israel, Germany, Switzerland, Canada, France, Spain and Singapore —  have seen a consistent rise in demand for skilled ML and AI professionals.


Which model should I use to determine the similarity between predefined sentences and new sentences?

If this is a simple syntax comparison, neural networks is not the best way to achieve this.
If it's semantic comparison, then you can take a look at models used in the SNLI dataset for example.
From your question, it looks like just a syntax comparison.
Consider the 2 sentences :

She likes playing guitar
She likes listening guitar

The 2 sentences have almost the same words, but the meaning is different.
Now consider these 2 sentences :

The bird is taking a bath in the fountain
Birdie wash himself with water in public place

These 2 sentences have almost no words in common, but the meaning are very similar.

So if your use case need to return a high-score for the first example, give up neural network (it is possible, but pointless).
If your use case need to return a high-score in the second example, take a look at SNLI leaderboards, there is plenty of models that can works.


What happens to the channels after the convolution layer?

Answer to my question is that values obtained from convolutions among different channels sum up together, therefore 3 channels after convolution with one filter give one output.

Best explanation delivered by Andrew: https://www.coursera.org/lecture/convolutional-neural-networks/convolutions-over-volume-ctQZz


Does Musk know what gradient descent is?

I think Musk was using the terminology correctly though perhaps with hyperbole.

I believe this was tweeted in the context of the botnet attacks on name-resolution services that broke Netflix and a large number of other internet services for a time.  He was expressing the idea that you could train a botnet-based system to attack the internet by giving it a toolbox of targets and hacks and using well-known machine-learning techniques to optimize the effectiveness of using the attacks in combination to take down the internet.

Gradient descent itself isn't harmful; it is using gradient descent to train a botnet that could theoretically result in a really devastating attack.

I also don't think he is necessarily implying that there is an actual machine learning system at work; rather that the hackers were training themselves with a methodology analogous to gradient descent.


Library for rendering neural network NEAT

I've never used it, but, if you are using Python, have a look at neat-python, which is a Python package that implements NEAT and also provides a module, visualize, to plot the best and average fitness vs. generation, plot the change in species vs. generation, and to show the structure of a network described by a genome. See this Stack Overflow thread, if you encounter any issue while attempting to use this module. See also this example.

Have also a look at this web page, which lists links to many NEAT implementations in different languages, where some of them provide visualization tools.


This is known as the intelligence explosion hypothesis or recursive self-improvement.