Welcome to mini_net’s documentation!¶
 Author: Cole Howard
 Email: uglyboxer@gmail.com
network.py is a basic implementation of a one layer linear neural network, to examine an implementation of backpropagation. It is based on the basic model of the Perceptron. Information on that can be found at: https://en.wikipedia.org/wiki/Perceptron
The intent of this specific project is to alter the Perceptron’s
decision function to a logistic function and add a “backpropagation” step at the end of each vector’s pass through the neuron.
There are several methods included that are currently passed, as they are plans to make it extensible as possible as a tool for exploring neural nets under more general circumstances.
 Dependencies:
numpy.dot() : for a fast implementation of the dot product of two vectors sklearn.datasets : (optional) for running this as a script on the
scikitlearn digits dataset neuron : the class definition of an individual neuron, also included in
 mini_net
 Usage:
It is currently set up to run a training set of input (along with the associated answers) and a set of similar, but distinct, input (without) the answers, and have the machine guess an answer to each of those cases based on information it gathered during the training run.
To import the network for testing on other data:
download the package mini_net, then include in your script:
from network import NetworkTo execute as is, from the command line, while in the linear_neuron/mini_net/ directory, input:
$ python3 network.pyThis will pull the learning and test data from scikitlearn run both and return a success count, total count of the unseen test data, and the success rate that equals.
First output and success ratio will be based on the first set of testing vectors. The second set will represent the same for the validation set. The visualization (see below) that pops up, just close that window for the script to finish running.
 Alternate data sets:
 Alternate training and testing data sets can be swapped out in the first section of main() below. See those notes for specifics.
 Visualization:
 Pyplot is included to provide a visual representation of a member of the dataset.

class
network.
Network
(neuron_targets, train_set, train_answers, epochs, test_set, test_answers, validation_set, validation_answers, images=None)¶ A Network instance will create layers of neurons for the implementa tion of neural network.
Parameters:  images (list) – Corresponding images of the dataset
 neuron_targets (list) – The possible final output values
 vector_size (int) – Size of the individual input vectors
 train_set (list) – Set of vectors for the learning portion
 train_answers (list) – Correct answers that correspond to the train_set
 epochs (int) – Number of times the learn_run will run for a given train_set
 test_set (list) – Set of vectors, discrete from the train_set to have the machine guess against
 test_answers (list) – Correct answers for the test_set, to compare the machine’s guesses against
 validation_set (list) – A validation set to compare answers in a second run
 validation_answers (list) – Answer for the above

neurons
¶ Class Neuron
Instances of the Neuron class, one for each of possible correct answers

append_bias
(vector)¶ Takes a list of n entries and appends a 1 for the bias
Parameters: vector (list) – 
num_of_training_vectors
¶ int
This is to adjust the size of the training set when all of the data is provided as large list. Breaking the training data into a training set, testing set, and a validation set. Picking this number is a balance between speed (lower number) and overfitting the data (a higher number)
Returns: The input vector with a one appended to the end of the list, as a bias Return type: list 

gradient_descent
(vector, vector_index)¶ Calculates the gradient_descent
Parameters:  vector (list) – A single input, comprised of floats
 vector_index (int) –

learning_rate
¶ float
Determines how much of the error is applied to the weights in each iteration
Returns: Represents the error to be used to update the weights of the neurons. It should approximate a gradient descent in topology of the outputs Return type: float

learn_run
()¶ Runs an iteration through the neuron sets and adjust the weights appropriately. It then follows up with a second weight adjusment accross all neurons with an estimate of the gradient descent function

report_results
(guess_list, validation=False)¶ Reports results of guesses on unseen set
Parameters: guess_list (list) –

run_unseen
(validation=False)¶ Makes guesses on the unseen data, and switches over the test answers to validation set if the bool is True
For each vector in the collection, each neuron in turn will either fire or not. If a vector fires, it is collected as a possible correct guess. Not firing is collected as well, in case there an no good guesses at all. The method will choose the vector with the highest dot product, from either the fired list or the dud list.
Parameters: validation (bool) – Runs a different set of vectors through the guessing process if validation is set to True Returns: a list of ints (the guesses for each vector) Return type: list
Author: Cole Howard Email: uglyboxer@gmail.com neuron.py is a basic linear neuron, that can be used in a perceptron Information on that can be found at: https://en.wikipedia.org/wiki/Perceptron
It was written as a class specifically for network ()
 Usage:
From any python script:
from neuron import Neuron API:
 update_weights, fires are the accessible methods usage noted in their definitions

class
neuron.
Neuron
(vector_size, target, sample_size, answer_set)¶ A class model for a single neuron
Parameters:  vector_size (int) – Length of an input vector
 target (int) – What the vector will associate with its weights. It will claim this is the correct answer if it fires
 sample_size (int) – Total size of sample to be trained on
 answer_set (list) – The list of correct answers associated with the training set

threshold
¶ float
The tipping point at which the neuron fires (speifically in relation to the dot product of the sample vector and the weight set)

weights
¶ list
The “storage” of the neuron. These are changed with each training case and then used to determine if new cases will cause the neuron to fire. The last entry is initialized to 1 as the weight of the bias

expected
¶ list
Either 0’s or 1’s based on whether this neuron should for each of the vectors in the training set

guesses
¶ list
Initialized to 0, and then updated with each training vector that comes through.

fires
(vector)¶ Takes an input vector and decides if neuron fires or not
Parameters: vector (list) – A sample vector Returns:  bool – Did it fire? True(yes) or False(no)
 float – The dot product of the vector and weights

train_pass
(vector, idx)¶ Passes a vector through the neuron once
Parameters:  vector (a list) – Training vector
 idx (an int) – The position of the vector in the sample
Returns: Return type: None, always

update_weights
(error, vector)¶ Updates the weights stored in the receptors
Parameters:  error (int) – The distance from the expected value of a particular training case
 vector (list) – A sample vector

l_rate
¶ float
A number between 0 and 1, it will modify the error to control how much each weight is adjusted. Higher numbers will train faster (but risk unresolvable oscillations), lower numbers will train slower but be more stable.
Returns: Return type: None