Programming a Perceptron in Rust

Creating a new project

First, let's create a new Rust project with Cargo:

$ cargo new perceptron-rs
$ cd perceptron-rs

Then add the rand crate as a dependency in Cargo.toml:

[dependencies]
rand = "0.3"

Helper functions

For our implementation, we need some helper functions:

• An implementation of the Heaviside Step Function
• A function to calculate the dot product of two 3-tuples

For this example we won't bother writing generic code, so the implementations are quite straightforward:

/// Heaviside Step Function
fn heaviside(val: f64) -> i8 {
    (val >= 0.0) as i8
}

/// Dot product of input and weights
fn dot(input: (i8, i8, i8), weights: (f64, f64, f64)) -> f64 {
    input.0 as f64 * weights.0
    + input.1 as f64 * weights.1
    + input.2 as f64 * weights.2
}

Structs

Let's define a simple struct to hold the training data and the expected output. As in the original blogpost, we'll train the perceptron to do the boolean OR function.

A TrainingDatum can hold an input value (implemented as 3-tuple) and an expected output value (a simple integer).

struct TrainingDatum {
    input: (i8, i8, i8),
    expected: i8,
}

The first two values of the input field are the two inputs for the OR function. The third value is the bias.

Initialization

First we need to load the rand library and initialize a random number generator instance:

extern crate rand;

use rand::Rng;
use rand::distributions::{Range, IndependentSample};

// ...

let mut rng = rand::thread_rng();

The training data is provided as an array of TrainingDatum instances:

let training_data = [
    TrainingDatum { input: (0, 0, 1), expected: 0 },
    TrainingDatum { input: (0, 1, 1), expected: 1 },
    TrainingDatum { input: (1, 0, 1), expected: 1 },
    TrainingDatum { input: (1, 1, 1), expected: 1 },
];

We then initialize the weight vector with random data between 0 and 1:

let range = Range::new(0.0, 1.0);
let mut w = (
    range.ind_sample(&mut rng),
    range.ind_sample(&mut rng),
    range.ind_sample(&mut rng),
);

The learning rate is set to 0.2 and the iteration count to 100.

// Learning rate
let eta = 0.2;

// Number of iterations
let n = 100;

Now we can start the training process!

// Training
println!("Starting training phase with {} iterations...", n);
for _ in 0..n {

    // Choose a random training sample
    let &TrainingDatum { input: x, expected } = rng.choose(&training_data).unwrap();

    // Calculate the dot product
    let result = dot(x, w);

    // Calculate the error
    let error = expected - heaviside(result);

    // Update the weights
    w.0 += eta * error as f64 * x.0 as f64;
    w.1 += eta * error as f64 * x.1 as f64;
    w.2 += eta * error as f64 * x.2 as f64;
}

After 100 iterations, our perceptron should have learned how to behave like an OR function.

// Show result
for &TrainingDatum { input, .. } in &training_data {
    let result = dot(input, w);
    println!("{} OR {}: {:.*} -> {}", input.0, input.1, 8, result, heaviside(result));
}

Testing

Let's run the resulting program!

$ cargo build
$ target/debug/perceptron
Starting training phase with 100 iterations...
0 OR 0: -0.07467118 -> 0
0 OR 1: 0.69958573 -> 1
1 OR 0: 0.82801196 -> 1
1 OR 1: 1.60226888 -> 1

Yay for Rust!

Feedback

I'm still quite new to Rust and not an expert on AI either. So if you have any feedback, suggestion or improvement, please leave it below or on Hacker News!

The code is on Github. If you have improvements concerning the code itself, you may also create a pull request.