Aside from stamps and inks there really aren't that many supplies that you will need for your stamping projects. Notice I said "need" - there are many, many tools available to you for stamping!
What you will need is really quite simple...

Most importantly, you'll need something to clean the ink off of your stamps.
A scrubber pad is a great tool to use for cleaning stamps. Along with the scrubber pad you will need a stamp cleaning liquid. These come in both spray form that you can spray directly on the scrubbing pad, or onto the stamps themselves. Also available is a cleaning liquid with a sponge dauber on the top. These little guys are great for cleaning detailed stamps with those tiny little crevices. They are also good for stamps that might have been a little...uh, neglected and have been left for a while with pigment ink dried up on them. Not that you would ever neglect your stamps...
My favorite, most versatile (and inexpensive!) tool for cleaning stamps is baby wipes.

Scissors
You might be wondering why a pair of scissors would be a tool for stamping. It's obvious why you might need them for card-making... but for stamping?
Yes, these scissors are craft scissors that are especially useful for trimming rubber. Unlike acrylic stamps, rubber stamps sometimes need to be trimmed in order to remove the little edges that can catch ink and leave unwanted marks on your paper.
Blocks

Acrylic blocks are especially handy if you have unmounted stamps in your arsenal of supplies. Traditionally stamps have come as rubber images mounted on wooden blocks. Over the years the images have been made available as flat images stored on a sheet of plastic (or, as an ingenious storage idea, in clear CD cases).

As more and more companies are moving towards unmounted
images, the
need for acrylic blocks becomes
more prevalent. They come in a variety
of shapes and sizes. At the very least, you'll want a small, medium and
large block.
So there's your list of basic stamping supplies. I bet it's not nearly as much as you thought it would be, is it?