text:
\section{Future development}

The main weakness of the classier in its current form is its processing speed. With the aid of the machine learning algorithm, the speed of the URL classification can be improved. At the moment, the pattern-matching algorithm needs to match a given URL with every single URL in the URL database, which containing both malicious and benign URLs. Malicious and benign URLs can be converted into the form of a vector. These two vectors allow additional machine learning methods such as Support Vector Machine (SVM) or Multi-Layers Perceptron (MLP) to be applied. This reduces the need for matching a given URL to every single URL within the URL database. However, converting the URL to a form of a vector is very challenging in terms of implementation effort because the structure of a URL does not have a repetitive pattern.

\paragraph{}
The second development for the classifier is to use different classification algorithms and then to compare and contrasts their performance. Three machine learning technique algorithms are suggested for implementation, namely; Naïve Bayes, Support Vector Machine (SVM) and Decision Tree. The implementation of these three algorithms requires two datasets; a training and a testing dataset. A list of URLs can be used as training dataset for the classifier and a separate list of URLs is required for the testing dataset. URLs in the testing dataset should be completely different from those URLs in the training dataset. The important aspect involves the selection of URL features for classification. Many features can be considered such as; length of the URL, number of forward slashes '/'or the number of dots '.'' used in the URL structure. Features extracted from the URL’s structure are used as input for the classifier. The classifier will use extracted features to distinguish a malicious URL from the benign URL.

\paragraph{}
In the current system the HTML malware scanner uses a URL and a keyword as the input and based on the frequency of the keyword in the web page of the given URL decides whether the URL is malicious or clean. An alternative design for the HTML scanner involves continuously updating a database with a feed of new trendy keywords so that the HTML scanner’s only input requirement is a given URL. The challenging part of this model is designing of a trendy keyword database, with each trendy keyword belonging to a group of keywords that have the same topic. The database will be connected to a list of selected trusted website such as BBC News. In turn, the database can be updated with any new trendy keywords being used on the News website, with old keywords being removed from the database after a predefined time period e.g. one month. In this model, the scanner decision is not only based on one trendy-keyword, it is based on a group of keywords connected to the trendy keyword topic within the database. Hence, ensuring a more reliable result is provided by the malware scanner.

\paragraph{}
Another option for the design of the HTML malware scanner involves considering the features of the web page related to each URL. In this method, a list of features of the webpage is extracted. This method is very similar to the one already suggested for the classifier. Again, there is a list of training dataset and testing dataset for the classifier, but instead the extracted features from web page will used as an input for the classifier. The web page features may be the number of hyperlinks in the webpage, number of lines, word count and functions (JavaScript keywords).

After training the classifier, some features may provide more useful results such as the use of JavaScript function in the web page. Hence, the features that produce the most accurate results will be selected for classification. Although the latter model does not consider the relationship between the trendy keyword term and the malicious URLs, a powerful system with an updated database and classifier can be created by combining the above two models. As a result, the database would contain a list of trendy keyword terms updated from trusted websites and the classifier would be capable of identifying unknown malicious URLs relating to the trendy keyword term.

The malware detection system that employs machine-learning techniques has a few advantages. The system can overcome the problem of detecting an unknown malware and also polymorphic malwares. In addition, it is very efficient and resilient in comparison with other malware detection techniques. Moreover, the adapted techniques in this system make it difficult for malware writers to bypass the detection system.

max_stars_repo_name: chrissorchard/malucrawl

max_issues_repo_name: chrissorchard/malucrawl

max_forks_repo_name: chrissorchard/malucrawl


text:
\documentclass[review,authoryear,3p]{elsarticle}
\usepackage{mystyle}

\begin{document}
\linenumbers
\input{./tex/title.tex}
\input{./tex/abstract.tex}
\maketitle
%\tableofcontents
%\listoffigures
%\listoftables

\input{./tex/introduction.tex}
\input{./tex/studyarea.tex}
\input{./tex/method.tex}
\input{./tex/results.tex}
\input{./tex/discussion.tex}
\input{./tex/conclusion.tex}

\section*{Acknowledgments}
This work is part of the eEcoLiDAR project, eScience infrastructure for Ecological applications of LiDAR point clouds \citep{kissling2017eecolidar}, funded by Netherlands eScience Center (\url{https://www.esciencecenter.nl}).

% Bibliography:
%\clearpage
\section*{References}
\bibliography{./tex/bibliography}

% Appendix:
% \clearpage
% \input{./tex/appendix.tex}

\end{document}

ext: tex

max_stars_repo_name: chrislcs/linear-vegetation-elements

max_issues_repo_name: chrislcs/linear-vegetation-elements

max_forks_repo_name: chrislcs/linear-vegetation-elements


text:
\section{Education}

\begin{EntriesTable}{0.18}{0.02}{0.80}
2012--2018 & Ph.D in Geophysics, University of Science and Technology of China, Hefei, China \\
2008--2012 & B.S. in Geophysics, University of Science and Technology of China, Hefei, China
\end{EntriesTable}

max_stars_repo_name: seisman/cv

max_issues_repo_name: seisman/cv

max_forks_repo_name: seisman/cv


text:
\documentclass[11pt, a4paper, twoside]{article}
\usepackage{graphicx}
\usepackage{amsmath}
\usepackage[margin=0.8in]{geometry}
\usepackage{listings}
\usepackage{float}
\usepackage{fancyhdr}
\usepackage{indentfirst}
\usepackage[inline]{enumitem}
\usepackage{xcolor}
\usepackage{minted}
\usemintedstyle{vs}
\usepackage[belowskip=0pt,aboveskip=0pt,font=small,labelfont=small]{caption}
\captionsetup{width=0.9\linewidth}
\setlength\intextsep{0pt}

\setlist[itemize]{noitemsep, topsep=0pt}
\fancyhead[RO,LE]{EE2703: Assignment 4}
\fancyhead[LO,RE]{Akilesh Kannan}
\cfoot{\thepage}

\title{EE2703: Assignment 4}
\author{Akilesh Kannan(EE18B122)}
\date{\today}

\pagestyle{fancy}

\begin{document}
    \maketitle
    \section{Abstract}
         In this assignment, I look at approximating a few functions using the Fourier series. I will employ two methods to find the Fourier approximation of $e^x$ and $cos(cos(x))$: \begin{enumerate}[label=\textbf{(\alph*)}, noitemsep, topsep=0pt]
            \item Direct Integration
            \item Least Squares Method
        \end{enumerate}

        I shall also compare the Fourier coefficients I have got in the two methods and see how close the Fourier approximation is to the true function.

    \section{Theory}
        \textit{Any periodic function can be expressed as a linear combination of various sinusoids.} This is the underlying principle of the Fourier Approximation. We can also approximate any function in the domain [0,2$\pi$) by slightly `tweaking' the mathematics of the Fourier series by extending copies of the function between [0,2$\pi$) till infinity, thus creating a periodic function.

        For a function $f(x)$, it's Fourier series is given as:
        \begin{equation}
            f(x) = a_0 + \sum_{k=1}^{\infty}a_kcos(kx) + b_ksin(kx)
        \end{equation}
        where,
        \begin{equation}
            a_0 = \frac{1}{2\pi}\int_{0}^{2\pi}f(x)dx
        \end{equation}
        \begin{equation}
            \begin{split}
                a_k = \frac{1}{\pi}\int_{0}^{2\pi}f(x)cos(kx)dx\\
                b_k = \frac{1}{\pi}\int_{0}^{2\pi}f(x)sin(kx)dx
            \end{split}
        \end{equation}

        The above equations describe the \textit{Direct Integration} method of finding the Fourier approximation. We shall also use the \textit{Least Squares} method we learnt in the last assignment to find the Fourier approximation.

    \section{Tasks}
        I shall now walk through the tasks that were asked to be performed and also include the necessary portions of the code and any associated plots.
        \subsection{Question 1}
            \begin{itemize}[label=-]
                \item Define Python functions for the two functions $e^x$ and $cos(cos(x))$ which return a vector (or scalar) value.
                \item Plot the functions over the interval $[-2\pi,4\pi)$.
                \item Discuss periodicity of both functions.
                \item Plot the expected functions from Fourier series.
            \end{itemize}
            \subsubsection{Code}
                \inputminted[linenos, breaklines]{python}{Code/q1.py}
            \subsubsection{Plots}
                \begin{figure}[H]
                    \centering
                    \setlength\tabcolsep{2pt}
                    \begin{tabular}{cc}
                        \includegraphics[scale=0.5]{Plots/Figure 1.png} &
                        \includegraphics[scale=0.5]{Plots/Figure 2.png}\\
                    \end{tabular}
                \end{figure}
            \subsubsection{Discussions}
                \begin{enumerate}
                    \item From the above plots, you can easily see that while $cos(cos(x))$ is periodic with period $2\pi$, the latter, $e^x$ isn't periodic and rises monotonically.
                    \item From the Fourier series, we expect that the Fourier approximation of the function would be repetitions of the function between $[0, 2\pi)$. We can thus say that the Fourier approximation of the function $f(x)$ is $f(x\mod2\pi)$\footnote{It is actually $f(x\mod p)$, where $p$ the minimum among the period of the periodic function and $2\pi$.}.
                \end{enumerate}
        \subsection{Question 2}
            \begin{itemize}[label=-]
                \item Obtain the first 51 coefficients i.e., $a_0$, $a_1$, $b_1$,\ldots for $e^x$ and $cos(cos(x))$ using \texttt{quad} function.
                \item Calculate the function using those coefficients and compare with original functions graphically.
            \end{itemize}
            \subsubsection{Code}
                \inputminted[linenos, breaklines]{python}{Code/q2.py}
            \subsubsection{Discussions}
                The function \mintinline{python}{calc51FourierCoeffs(f)} takes in the name of a function as argument and returns the first 51 coefficients of the function \texttt{f}, in order, as an array. One advantage of hard-coding the number 51 is that it becomes faster, while the hard-coding makes it difficult for the function to be generalised.

        \subsection{Question 3}
            \begin{itemize}
                \item[-] Two  different  plots  for  each  function  using  \texttt{semilogy}  and  \texttt{loglog} and plot the magnitude of the coefficients versus n.
            \end{itemize}
            \subsubsection{Code}
                \inputminted[linenos, breaklines]{python}{Code/q3.py}
            \subsubsection{Plots}
                \begin{figure}[H]
                    \centering
                    \setlength\tabcolsep{2pt}
                    \begin{tabular}{cc}
                       \includegraphics[scale=0.5]{Plots/Figure 3.png} &
                       \includegraphics[scale=0.5]{Plots/Figure 4.png}\\
                       \includegraphics[scale=0.5]{Plots/Figure 5.png} &
                       \includegraphics[scale=0.5]{Plots/Figure 6.png}\\
                    \end{tabular}
                \end{figure}
            \subsubsection{Discussions}
                \begin{enumerate}[label=(\alph*)]
                    \item As we can see from the above plots, almost all $b_n$ coefficients are very close to 0 for $cos(cos(x))$. This is expected - we find by hand integration that the $b_n$ coefficients are 0 (due to the odd nature of the integrand between $[-\pi,\pi)$.

                    It does not become exactly zero due to approximations used in the implementation of the \texttt{quad} function.

                    \item Rate of decay of Fourier coefficients is determined by how smooth the  function is - if a function is infinitely differentiable then its Fourier coefficients decay very fast. But if the $k^{th}$ derivative of the function $f$, denoted by $f^{(k)}$, is discontinuous, then the rate of decay of the Fourier coefficients is only $\frac{1}{n^k}$.

                    Since $cos(cos(x))$ is an infinitely differentiable function, it's Fourier coefficients decay very fast, while that of $e^x$ decay very slowly due to the discontinuity in the Fourier approximation of the function at $2n\pi$.

                    \item
                \end{enumerate}
        \subsection{Questions 4 \& 5}
            \begin{itemize}
                \item[-] Use a \textit{Least Squares} approach to find the Fourier coefficients of the functions, using \texttt{scipy.linalg.lstsq}.
                \item[-] Build the coefficient matrix $A$ and the constant matrix $b$.
            \end{itemize}

            \subsubsection{Code}
                \inputminted[linenos, breaklines]{python}{Code/q4.py}
        \subsection{Question 6}
            \begin{itemize}
                \item[-] Compare the coefficients obtained through the \textit{least squares method} and the \textit{direct integration} method.
                \item[-] Find the maximum deviation between the coefficients obtained in the two methods.
            \end{itemize}
            \subsubsection{Code}
                \inputminted[linenos, breaklines]{python}{Code/q6.py}
            \subsubsection{Plots}
                \begin{figure}[H]
                    \begin{tabular}{cc}
                        \includegraphics[scale=0.5]{Plots/Figure 3.1.png} &
                        \includegraphics[scale=0.5]{Plots/Figure 4.1.png}\\
                        \includegraphics[scale=0.5]{Plots/Figure 5.1.png} &
                        \includegraphics[scale=0.5]{Plots/Figure 6.1.png}\\
                    \end{tabular}
                \end{figure}
            \subsubsection{Discussion}
        \subsection{Question 7}
            \begin{itemize}
                \item[-] Computing $A\cdot c$ from the estimated values of $c$ by \textit{Least Squares Method} and plotting them.
            \end{itemize}
            \subsubsection{Code}
                \inputminted[linenos, breaklines]{python}{Code/q7.py}
            \subsubsection{Plots}
                \begin{figure}[H]
                    \begin{tabular}{cc}
                        \includegraphics[scale=0.5]{Plots/Figure 7.png} &
                        \includegraphics[scale=0.5]{Plots/Figure 8.png}\\
                    \end{tabular}
                \end{figure}
            \subsubsection{Discussion}

            \begin{itemize}[label=-]

            \item
              As we observe that there is a significant deviation for \(e^{x}\) as
              it has discontinuities at \(2n\pi\) which can be observed in the figure,
              and so there will be \textbf{Gibbs phenomenon} i.e, there will be
              oscillations around the discontinuity points and their ripple
              amplitude will decrease as we go close to discontinuity. In this case
              it is at \(2\pi\) for \(e^{x}\).
            \item
              As we observe that ripples are high initially and reduces and
              oscillate with more frequency as we go towards \(2\pi\). This
              phenomenon is called \textbf{Gibbs Phenomenon}
            \item
              Due to this, the original function and one which is reconstructed using
              least squares will not fit exactly.
            \item
              And as we know that Fourier series is used to define periodic signals
              in frequency domain and \(e^{x}\) is a aperiodic signal so you can't
              define an aperiodic signal on an interval of finite length (if you
              try, you'll lose information about the signal), so one must use the
              Fourier transform for such a signal.
            \item
              That is why there are significant deviations for \(e^{x}\) from original
              function.
            \item
              For\(\cos(\cos(x))\) the curves fit almost perfectly because
              the function itself is a periodic function and it is a continuous
              function everywhere, so we get very negligible deviation and
              able to reconstruct the signal with just the Fourier coefficients.
            \end{itemize}
    \section{Conclusion}
    We see that the Fourier estimation of \(e^x\) does not match
significantly with the function close to \(0\), but matches near
perfectly in the case of \(\cos(\cos(x))\). This is due to the
presence of a discontinuity at \(x=0\) for the periodic extension of
\(e^x\). This discontinuity leads to non-uniform convergence of the
Fourier series, with different rates for both the functions.

The difference in the rates of convergence leads to the \textbf{Gibb's
phenomenon}, which is the ringing observed at discontinuities in the
Fourier estimation of a discontinuous function.
This explains the mismatch in the Fourier approximation for \(e^x\).

Thus we can conclude that the Fourier Series Approximation Method works extremely well
for smooth periodic functions, but gives bad results for discontinuous periodic functions
\end{document}
\end{document}

max_stars_repo_path: Week 4/report.tex

max_issues_repo_path: Week 4/report.tex

max_forks_repo_path: Week 4/report.tex