Rule #0) Dress in layers. The rest of the rules won't work without this pre-requisite.
Rule #1) Stay dry. Your outermost layer should block water but also be breathable so your sweat can escape. Your innermost layer should wick away water. Wool or polyester work well for that.
Rule #2) Keep out the wind. Your outermost layer should also block wind. Try to keep it tight where different layers meet. For instance, an elastic waist in a jacket helps keep the wind from billowing up underneath and circumventing the outer layer. Before you go out, stretch your arms around to make sure your wrists are waist aren't exposed.
Rule #3) Keep in the heat. Have a collar that blocks heat escape like an elastic band or scarf. Your middle layer should be insulating like wool, fleece, or even down.
Rule #4) Be mindful of changing conditions and ajust accordingly. If you're building up sweat, take off a layer or loosen it so you get better ventilation. If you start to feel cold somewhere, change something. Stand up if your butt gets cold. Pull your hands inside your sleeves or even all the way to your torso if you're losing feeling in your fingers. Go change your socks if your toes get cold.
Keep your head covered. Wear long underwear. Keep moving.
Keep your head covered, it's where most heat escapes. Also high risk areas like ears, fingers, and toes. I layer moisture wick long underwear, then a thicker liner like fleece or the "old style" long johns. I also think your core is important, so on top I'll wear a vest. From then on, it's all about conditions. Wet or not, but I usually wear tough canvas pants and a fleece, hood, and knit cap over the ears. Add gortex if it's wet, and get good gloves and boots. I use the $1.00 jersey cloth gloves under water resistant mittens and army surplus boots that have always kept,my feet warm. Yes, there are electric socks and gloves you could use. Or the Hot Packs, which sometimes I put behind my head. They even used to make "pocket warmers", but I guess that someone bursting into flames ended That.(Too bad, I'd love a couple) However, when it is really cold, you already are in 10 pounds of bulky clothes, so I don't want batteries or stuff in my pockets.
Engineer Toast and the others have provided great suggestions. I have also found that wearing a pair of snow pants (like those used for snowmobiling or skiing) makes a huge difference when you need to be outside for long duration in extreme cold or high winds.
I think the top answers already covered most clothing things, but I highly recommend wool and down as they're much warmer for their weight than anything else. My extra cold weather outfit is several layers of wool long underwear (I like merino, which you can find at outdoor stores, as it's much softer than other wool), a thick wool sweater, corduroy or other warm pants, fleece or down vest with a fat zip-up collar so I can skip a scarf (some people prefer neck warmers but I feel like they're choking me), down jacket with a good hood, possible waterproof or at the very least wind proof layers (top and bottom), fleece-lined wool or down hat and mittens, a few layers of wool socks, and fleece-lined waterproof leather boots. I do most of my winter clothes shopping at REI or other outdoor stores, and I think outdoor clothing is generally much warmer than the stuff you'd find in normal clothing shops, but I could be wrong about that.
Also I just wanted to add these... not exactly clothing, but definitely life hacks for staying warm:
A supply of those little chemical hand warmer packets can go a LONG way if your clothing isn't quite keeping up with the cold. The disposable, iron-based/air-activated exothermic ones are nontoxic, environmentally safe, odorless, non-combustible, super cheap (like $1 or less), and last like 10-12 hours, but I think there are also reusable ones on the market if you want to avoid disposable products. I stuff them into my mittens, bra and all my pockets when I need an extra boost of heat. They are AMAZING. They do need oxygen to activate though, so they'll stop working after a few minutes if you stuff them into tight waterproof boots, which is one reason I like the fleece/fur-lined boots as they seem to let a little bit of air in to keep them at least slightly activated.
Eating and physical activity are the two best ways you can get your body to generate heat from the inside out. If you're cold and haven't eaten for a while, sometimes having a snack will be enough to warm you back up. You can also jog in place to hear yourself up (extra points for generating heat through friction if you do this while rubbing your thighs together!), although it's important to find the balance between warming up and breaking a sweat which you definitely want to avoid. And I always keep an insulated thermos of hot tea on me to sip throughout the day when I'm doing field work in the snow (those Swell bottles or similar knockoffs are the best ones I've found for staying hot all day). If you aren't going to be somewhere remote and can refill a bottle of hot tea or water throughout the day, it's actually better to use a less insulated bottle like a Nalgene or one of those old school rubber water bag/bottles if you can find one, and store it in an inner jacket pocket on your torso so it heats you up until you finish it.