I'm always on the lookout for good starter educational kits for students to learn electronics through their own hacking project, and I hit the mother load this week. First, this month's Wired magazine had a plug for the new Micro-controller kit from Thames and Kosmos, which you can purchase here at Amazon.
The $136 price tag is a bit steep compared to the raw cost of all the parts, but everything is ready to go with almost no effort or overhead whatsoever, including all the necessary interface cables and whatnot.
Secondly, for those more intrepid (or budget conscious), there is the path for the purist. This link over at Instructables will send you on your way to building your first prototype systems from scratch including your own interface cables. On the up-side, total cost to start doing fun stuff can be managed within a $10 budget or so as long as there is an old PC with a parallel port lying around somewhere. The canonical first project it to light up an LED and control it.
This Instructable also include links to Cornell, where the devices are used in a class, complete with links and blogs about the student's final projects, and there is an extensive web reference site called AVR Freaks.
Go forth and Compute!