Useful sites
Enables you to compare a wide range of the available CMS products side by side. Keep in mind that the content is all created by the product developers themselves. Nevertheless it provides you with a useful checklist of the features you should be looking for in a CMS and some of the terminology.

Wikipedia articles

The following wikipedia articles are a good place to start to get your head around things.