I'd like to build a freelancer-like website, but much simpler.

Freelancers would be split up in groups of expertise.
Freelancers can register, they have to choose by then one or two of the groups.
Freelancers can create a profile where they can add a CV, previous works, contact details, their price, their specialty and so on.

Employers can register, search for freelancers, maybe rate them.

I already have the database of freelancers.

Basically, this is what I'm looking to start with.
I'd like to add possibility to chat, hire, payment requests, but later.

I've done static and dynamic websites, mostly from scratch and some using Wordpress.

I'd like to go for a CMS like Joomla, Drupal or Concrete5, cause they have a big community with addons and themes.
I've looked into many CMSs together with addons, but most of them offer much more than I'm looking for, like whole social networking and stuff.

Any advice would be great.