Well, if you plan on actually taking and authorizing credit card data, then you'll need two things.
An SSL certification with a minimum of 128 bit encryption, this can be free for your own SSL option, or around 20/year for a decent outsourced option.
Full PCI compliance. Which costs MONEY, BIG MONEY. Even running your own servers still requires that a third party evaluates your services, and this third party must be a registered AVS.
Basically, go with an outsourced solution such as PayPal until you're ready to drop a few hundred a month (worst case is about $250, cheapest I've experienced personally is around $80/mo). And if you do become ready to accept credit cards on your site directly, then PayPal allows for that too using "direct pay". Stay away from express checkout if at all possible, even if it is required for Direct Pay. It sucks to implement, and 1/2 of the captures fail after authorization if the funding source of the customer is a bank account without a business account (according to PayPal's support, this was the issue).
Well good luck! Sorry about the rant... heh.