About artifical neural networks

Artificial Neural Networks (ANN) are computing systems inspired by the biological neural networks that constitute animal brains. The network itself is not an algorithm. It is basically something like a framework to put all Machine Learning (ML) algorithms together. Such systems "learn" to perform tasks by considering examples, generally without being (hard) programmed with any task-specific rules.

For example, in image recognition, they might learn to identify images that contain cats by analyzing example images that have been manually labeled as "cat" or "no cat" and using the results to identify cats in other images. The most important part is, that they do this without any prior knowledge about cats. So they "automatically" generate indentifying characteristics form learning material that they process.
We gonna built a little and simple neural network. In this we want that the "AI" learns to use the following pattern without any hard coded knowledge:


1 1 0 = 0
0 0 1 = 1
0 1 0 = 0


As you can see we take 3 inputs and validate between 0 and 1. The output is 1 when input (1) and input (2) are 0's. Others are 1's.
We don't tell that to the computer. We want that he learns and recognizes the pattern.

Modelling the network

A Network with more than 3 layers of neurons is a deep neural network. To make it as simple as possible, we gonna use just 1 layer with 1 neuron.
It will have 3 input and 1 output.

Training

We must train the NN, that it "automatically" recognizes the pattern. So we give each input a weight, which can be a positive or a negative number. An input with a large positive or a large negative weight would have a strong effect on the NN's output.
Before we start, we set to each weight a random number (through that the NN has a start definition and it will configure itself with the right algorithms). Then we do the following training process:


(1) Take the inputs from the training set, adjust them by the right weights, and pass them through a "formula" to calculate the neurons's output.
(2) Calculate the error, which is the difference between the output and desired output in the training set.
(3) Depending on the direction of the error (+ / -) adjust the weights.
(4) Repeat this process 10'000 times, that the "AI" fully learns it.

This process is called "back propagtion". In data science (the subject of ML/CNN/ANN) there are many different ways to build actually a NN. Through that there are several different algorithm ideas.

Start with programming

We gonna use JavaScript (ES6+ and a sprinkling of nodeJS) to solve this task. JavaScript is next to python one of the most used AI languages. For example if you use C++, than you could use C++ to modelling the network or (the more common way) you gonna use JavaScript for that task and connect it to your C++ application.

Let's start with creating a new *.js file. Create in your IDE a neuralnet.js file and if you want, copy the following code in it.

// Created by Luca Marceca
// (c) - valenzelektron.com

/*
*	        _                    _      _    _                   
*	       | |                  | |    | |  | |                  
*   __   ____ _| | ___ _ __  _______| | ___| | _| |_ _ __ ___  _ __  
*   \ \ / / _` | |/ _ \ '_ \|_  / _ \ |/ _ \ |/ / __| '__/ _ \| '_ \ 
*    \ V / (_| | |  __/ | | |/ /  __/ |  __/   <| |_| | | (_) | | | |
*     \_/ \__,_|_|\___|_| |_/___\___|_|\___|_|\_\\__|_|  \___/|_| |_|
*  
*   valenzelektron (c) - valenzelektron.com
*
*   @author: Luca Marceca
*   @title: Neural Network Tutorial
*   @programmingLanguage: JavaScript 
*   @commentLanguage: en
*   @version: 1.1.1
*   @update: 11-05-2018 [DD-MM-YYYY]
*   @updateBy: Luca Marceca (LuM)
*   @updateDescription: release
*/

"use strict";



Now we also make a markup file something like neuralnet.html. The reason why we use HTML and no other GUI based system is, that we gonna make it as simple as possible (and HTML is fairly easy :)).
Copy the following code into your markup file.

<html lang="en">

<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta http-equiv="X-UA-Compatible" content="ie=edge">

<title>neural network tutorial</title>

<script src="neuralnet.js"></script>
</head>
<body>
<div class="final-network">
    <input id="number1" type="number" placeholder="0 or 1"><br />
    <input id="number2" type="number" placeholder="0 or 1"><br />
    <input id="number3" type="number" placeholder="0 or 1"><br />
    <button onclick="main()">test</button></div>
</div>
</body>

</html>



What this code actually does is, that it will generate the input form and also define a new onclick event that it activate our JS application.
So now we can concentrate us on the real app.

// config
const /* int */ numberQuantity = 3, /* double */ trainingQuantity = 10000;
var weights = [], numbers = [], adjustments = [];
var sum = 0, loggedError = 0, loggedAnswer = 0, output = 0; 

var sigmoidFunction = (s) => { return 1 / (1 + Math.E**(-1 * (s))); }; // normalize inputs in dependence to the weighted sum between 0 and 1
var errorFuntion = (err, num, out, iteration) => { return err * num[iteration] * out * (1 - out); }; // diffrence between the calculated output and the real output; its used to change the weights and actually train the net



The above code defines our main variables (copy this to your *.js file). The variable numberQuantity is equal to the amount of input fields, trainingQuantity is the loop number for the training sets.
Through the 3 arrays, we will save the numbers for the 3 inputs in it. Then there are also this arrow functions. The sigmoid function is a common formula to normalize the weighted sum of the inputs between 0 and 1. Its fairly interesting how this formula actually works for a good understanding I recommend to google it and take a look at the curve in the coordinate system.
So we get indeed the input dependence to the weighted sum, but that doesn't help us to actually change the "AI's" knowledge. We must take through the error function the difference between the "real" output and the weighted (sigmoid) output and calculate it into our main function area.
Depence on the neural network, it will have way more formula (i.e. a activation function). But we don't need that. This 2 formulas are enough. It's like going to a supermarket - you could choose what you want but you just buy what you need.

Let's start with a neural network class.

class NeuralNetwork {
    constructor() {
        this.numbers = numbers;
        this.weights = weights;
        this.output = output;
        this.adjustments = adjustments;

        this.loggedAnswer = loggedAnswer;
        this.loggedError = loggedError;

        this.sum = sum;

        // set initial weights
        for(var j = 0; j < numberQuantity; j++) {
            this.weights.push((Math.random() * 10) - 5); // set a big range
        }
    }
}



In it we defined a constructor (main structure method) with the main config variables. We also loop through the weights and give them for the start random values as I said.

    get training() {
        // set initial weights
        for(var a = 0; a < numberQuantity; a++) {
            this.weights.push((Math.random() * 10) - 5); // set a big range
        }
        for(var j = 0; j < trainingQuantity; j++) {
            // set random numbers for the first setup
            for(var i = 0; i < numberQuantity; i++) {
                this.numbers.push(Math.floor(Math.random() * 2)); // set a small range
            }

            // define the neural network rule
            if(this.numbers[0] === 0 && this.numbers[1] === 0) this.loggedAnswer = 1;
  
            this.sum = 0; // reset
            
            for(var c = 0; c < this.numbers.length; c++) {   
                this.sum += this.weights[c] * this.numbers[c]; // one variant => calculate them together for the main e function   
            }

            this.output = sigmoidFunction(this.sum); // calculate diffrence with the std exp. function
            this.loggedError = this.loggedAnswer - this.output; 

            if(this.output !== 0.5) {
                for(var d = 0; d < this.numbers.length; d++) {
                    this.adjustments.push(errorFuntion(this.loggedError, this.numbers, this.output, d)); // calculate all together
                }
        
                for(var o = 0; o < this.numbers.length; o++) {
                    this.weights[o] += this.adjustments[o]; // log the new calculated weights
                }
            } // redefine weights
            
            // log it public
            output = this.output;
            weights = this.weights;
            numbers = this.numbers;

            // reset
            this.numbers.length = 0;
            this.adjustments.length = 0;
            this.loggedAnswer = 0;
            
            // debugger;
        }
    }



Put the above method into your NeuralNetwork class next to the constructor. It will define the training section.
So the first part will define the same thing as the constructor: random values for the weights. After that part it will loop 10'000 times to find the right weights. In the first step it will create a training set. So now we have an example training set which the computer must find out. In other NN's this can be also something not computer generated like a really big database with predefined situations. After that it will define the rule. Note that this rule is basically not in the real NN: its simulated. So for a better unterstanding you can say that there are 2 different computers. 1 which generates training patterns (he is precoded and knows the pattern) and 1 which recognizes the pattern with a NN (he is not precoded and must findout ther pattern himself). They are independent.
Now it has a training set. Through that it can calculate the weights and the input numbers together and evaluate it with the sigmoid function as defined above. After that it has a "loggedError" variable which takes the difference between the answer of the "AI" and the real answer (0 or 1).
If the calculated output is not equal to 0.5 (because if so the result would be perfect), then loop through the inputs and calculate the deviation with the loggedError and the predefined error function. After that we log those as new weight variables. Finally we gonna reset the values, so we could make another training.

// application start config
var neuralNetwork = new NeuralNetwork(); // generate new neural network
window.onload = function() {
    neuralNetwork.training; // train network
}



For the next step copy the above code outside to your class (after the class is defined). So its a fairly easy setup, because we use HTML and we can just make a window onload function.

    get errorCalculation() {
        this.sum = 0; // reset

        for(var i = 0; i < this.numbers.length; i++) {
            this.sum += this.weights[i] * this.numbers[i]; // one variant => calculate them together for the main e function
        }

        this.output = sigmoidFunction(this.sum); // calculate diffrence with the std exp. function
        this.loggedError = this.loggedAnswer - this.output; 
        
        // log it public
        output = this.output;
        weights = this.weights;
        numbers = this.numbers;
    }



Now we will need a realtime validator to evaluate the inputs. That's what the above code does. Put it as a method before the training method in your NeuralNetwork class.

/* export */ function main() {
    // get the input fields
    for(var i = 0; i < numberQuantity; i++) {
        numbers[i] = parseInt(document.getElementById("number" + (i + 1)).value);
    }
    
    // evaluate the inputs
    if(isNaN(numbers[0]) || isNaN(numbers[1]) || isNaN(numbers[2])) { /* ... */ } 
    else { 
        neuralNetwork.errorCalculation; 

        if(output <= 0.75) { // <= 0.7
            console.log("right ans: 0")
        }
        else {
            console.log("right ans: 1")
        }
    }

    // reset
    output = 0;
    loggedAnswer = 0;
    sum = 0;
}



Last but not least, we gonna setup the main function which is the event of the test button. In the first part we loop through the inputfields and put them into our number array. After that we will evaluate those numbers with the errorCalculation method. If the output is greater or equal to 0.75, than the answer is 1. The reason why we used 0.7 is just for testing reasons. The best numbers for that are those between 0.5 and 1, because all numbers greater then 1 are clearly defined and so also all less then 0.5. You can change 0.75 to 0.5 or 1 and see what happens :).

Wow! you have built a neural network. With the follwoing input field you can actually test this application.