How a neural network differs from a regular program

A regular program is a set of instructions. A programmer writes: "if the total is over a thousand, apply a five percent discount." The computer obediently does it. Everything is predictable, and the program won't do anything beyond what's written.

Now try writing instructions for "how to spot a cat in a photo." Triangle ears? So does a fox. Whiskers? So does a seal. The cat's curled up in a ball, hiding behind a curtain, shot from above, and every rule falls apart. You can't solve problems like this with rules. That's where a neural network comes in: nobody explains to it what a cat looks like, they just show it cats.

How a neural network learns

Inside a neural network is a huge number of connections, and each has a "weight": how strongly it affects the answer. At first the weights are random and the network guesses blindly. It's shown a photo, says "dog," and gets told: wrong, that's a cat. The weights get nudged a tiny bit so the error is smaller next time.

Repeat that millions of times and the network starts recognizing cats in photos it has never seen. The number of these adjustable connections is called parameters. Large language models have billions of them, which is why training takes months and costs a fortune.