Here's how I did it.
'''Starting state:'''
The bike was a steel roadbike with horizontal dropouts.
As other commenters have said, this is essential. Don't start if this is not the case.
'''New Parts:'''
I bought a fixed gear hub, and a rim, and took it to a bikeshop to build.
They put a stainless steel fixed cog on one side and a freewheel on the other.
The fixed cog was built for a thicker chain. So I bought the appropriate one.
A thicker chain can work on smaller cogs, but not vice versa.
'''Removing Old Parts:'''
I removed the shifters, and all the cables, and then I removed one of the front chainrings.
I kept the brakes.
'''Putting On new parts:'''
Fixed gears are easy!
Add wheel.
Add chain.
You'll need to to size the chain properly, so Put the wheel on in the middle-ish of the dropouts, and then remove links until it's snug. Use a magic-link, or whatever it's called to finish it.
Tension it.
With the wheel in the dropouts tug back gently till there is no slack in the chain. Hold it snug while you tighten the bolts. The chain should not droop, but it should not be super super tight either.
Now tighten the bolts more.
And snug those bolts down again.
Check the chain tension, if it's too tight, loosen it.
If it's too loose tighten it.
'''Maintenance:'''
Chain Tension is the only thing you ever have to worry about.
Check it regularly.
Keep it snug.
Too loose and it will slip off
Too tight and it will climb off.