Single Neuron / Perceptron

Category: Deep Learning
Difficulty: Beginner
Time Complexity: O(n)
Space Complexity: O(1)

Overview

The perceptron is the fundamental building block of neural networks. A single neuron takes multiple inputs, multiplies each by a learned weight, sums the results, adds a bias term, and passes the total through an activation function. This visualization walks through each computation step, showing how inputs are transformed into an output, and how changing weights and bias affects the decision.

Try It

Web: Open in Eigenvue →

Python:

import eigenvue
eigenvue.show("perceptron")

Default Inputs

{
  "inputs": [
    0.5,
    0.8
  ],
  "weights": [
    0.6,
    -0.3
  ],
  "bias": 0.1,
  "activationFunction": "sigmoid"
}

Input Examples

AND gate (step function)

{
  "inputs": [
    1,
    1
  ],
  "weights": [
    0.5,
    0.5
  ],
  "bias": -0.7,
  "activationFunction": "step"
}

OR gate (step function)

{
  "inputs": [
    1,
    0
  ],
  "weights": [
    0.5,
    0.5
  ],
  "bias": -0.2,
  "activationFunction": "step"
}

Three inputs (sigmoid)

{
  "inputs": [
    0.3,
    0.7,
    0.5
  ],
  "weights": [
    0.4,
    -0.2,
    0.8
  ],
  "bias": -0.1,
  "activationFunction": "sigmoid"
}

Code

Pseudocode

function perceptron(inputs, weights, bias, activation_fn):
  z = 0
  for i in range(len(inputs)):
    z += inputs[i] * weights[i]    // weighted sum
  z += bias                         // add bias
  output = activation_fn(z)          // apply activation
  return output

Python

import numpy as np

def perceptron(x, w, b, activation='sigmoid'):
    z = np.dot(x, w) + b
    if activation == 'sigmoid':
        return 1 / (1 + np.exp(-z))
    elif activation == 'relu':
        return max(0, z)
    elif activation == 'step':
        return 1 if z >= 0 else 0

JavaScript

function perceptron(inputs, weights, bias, activation) {
  const z = inputs.reduce((sum, x, i) => sum + x * weights[i], 0) + bias;
  if (activation === 'sigmoid') return 1 / (1 + Math.exp(-z));
  if (activation === 'relu') return Math.max(0, z);
  if (activation === 'step') return z >= 0 ? 1 : 0;
}

Key Concepts

Weighted Sum

Each input is multiplied by its weight. The weight controls how much influence that input has.

Bias Term

The bias shifts the activation threshold. It allows the neuron to fire even when all inputs are zero.

Activation Function

Without activation, a neuron is just a linear function. Activation functions introduce non-linearity for learning complex patterns.

The Perceptron’s Limitation

A single perceptron can only learn linearly separable patterns. It can learn AND and OR but NOT XOR.

Common Pitfalls

Weight Initialization Matters: If all weights start at zero, all neurons compute the same thing.
Sigmoid Saturation: When |z| is large, sigmoid derivative is nearly 0 — the vanishing gradient problem.

Quiz

Q1: A neuron has inputs [1, 0], weights [0.5, 0.5], and bias -0.7. Using a step activation function, what is the output?

A) 0
B) 1
C) 0.5
D) -0.2

Show answer

Answer: A) 0

z = 1×0.5 + 0×0.5 + (-0.7) = -0.2. Since z < 0, step(-0.2) = 0.

Q2: Why can’t a single perceptron learn XOR?

A) XOR requires more than 2 inputs
B) XOR is not a linear function — no single line can separate the outputs
C) The step function can’t handle XOR
D) XOR requires negative weights