OSS4AI AI Stack · Season1 · #5
by Yujian Tang

A historical perceptron sums weighted inputs and applies a threshold. Open full-size image
Start with a weighted decision
An artificial neuron is simpler than its biological name suggests. It takes numbers, multiplies each input by a weight, adds a bias, and applies a function to the result. The weights control how strongly each input contributes. The bias shifts the point at which the neuron responds.
A simple threshold perceptron makes a decision: output 1 if the weighted sum is above zero, otherwise output 0. You can picture a judge adding points from a scorecard. The point values are weights; the threshold determines whether the total passes.
This is a useful starting point for neural networks, but not a claim that mathematical neurons work like cells in a brain.
One neuron can implement AND
Consider two binary inputs, x1 and x2. Give each a weight of 1 and choose a bias of -1.5. The score is x1 + x2 - 1.5.
For inputs (0,0), the score is -1.5. For (0,1) and (1,0), it is -0.5. All three produce 0. Only (1,1) gives a positive score, 0.5, and produces 1. That is the AND operation.
On a two-dimensional plot, the threshold corresponds to a straight decision boundary. In more dimensions, it becomes a hyperplane. A single perceptron can learn some useful separations, but its geometry limits what it can represent.

With weights1,1 and bias−1.5, only input(1,1) crosses the threshold. Open full-size image
Why XOR needs more
XOR outputs 1 when exactly one input is 1. The positive examples, (0,1) and (1,0), occupy opposite corners of the binary-input square. The negative examples occupy the other diagonal. No single straight line separates the two groups.
Changing the weights does not solve that structural problem. The training procedure cannot discover a boundary the model is unable to represent. The useful engineering lesson is to distinguish a learning problem from a capacity problem: extra training cannot change the set of functions a fixed architecture can represent.
Add a hidden layer and the situation changes. One hidden neuron can compute OR, another can compute AND. A final neuron combines them as "OR but not AND."
A tiny network you can check
Using step(z) = 1 when z is positive and 0 otherwise, define:
h1 = step(x1 + x2 - 0.5)h2 = step(x1 + x2 - 1.5)out = step(h1 - h2 - 0.5)
At (0,0), both hidden outputs are 0, so the final output is 0. At either single-positive input, h1 is 1 and h2 is 0, giving output 1. At (1,1), both hidden outputs are 1 and the final output returns to 0.
Two neurons feeding a third have created a distinction that one neuron could not make. These are hand-chosen teaching weights, not the result of a training benchmark.

The XOR positives occupy opposite corners, so a single linear boundary cannot separate them. Open full-size image
The nonlinear part is essential
Simply stacking weighted sums is not enough. Composing linear transformations with biases still produces an affine transformation. Without nonlinear functions between layers, the whole stack collapses into one affine map. If a threshold is then applied, the resulting boundary is still linear in the original features.
Modern networks use nonlinear functions rather than relying on this hard threshold everywhere. The step function here supplies historical intuition, not a literal description of every transformer activation. Logistic regression is a useful related baseline: a weighted sum passed through a sigmoid. Its probability-like output differs from a perceptron's binary threshold, but its basic decision boundary is still linear in the input features.
For a practical exercise, plot AND and XOR, evaluate the equations above, then compare a single-layer model with a model containing a nonlinear hidden layer. The goal is not to make the biggest network. It is to understand when added layers change what your model can represent.
Representation is not training
This example shows that suitable weights exist. It does not explain how to find them from data. The original perceptron updates weights based on prediction errors; differentiable networks commonly use gradient-based training and backpropagation.
Keep those questions separate: architecture determines the available functions, while training tries to find useful parameters within that space. More layers do not guarantee better generalization. The dataset, objective and evaluation still decide whether the learned function is useful outside the examples it saw.
