bandwith & hardware

[tukangnanya]

I plan to host my own site from my own place, and i have several question,

1. How do big websites like yahoo, google or youtube get their extraordinary bandwith and speed?
Millions of people daily can open the site without any problem at all.

2. Do they register themself to be an ISP to reduce cost?

If money is not the problem (but still, must be spend with frugal, no unecessary spending),

3. Do the best solution to get extraordinary bandwith and speed with low cost is to register ourself to be an ISP? Any suggestions?
I dont believe in "unlimited bandwith" webhost solutions, im not looking for this kind of answer.

4. With server using HP ProLiant SL390s G7, 2 processors + 192GB RAM. and let say in average there are 5 million daily page view on my site, = +200k page view per hour = +3k page view per minute = 58 page view per second. I know its hard to predict, but just approximately, for that kind of load, how many server do i need for optimum performance?

Thanks for the help.

[svidgen]

1. How do big websites like yahoo, google or youtube get their extraordinary bandwith and speed?
Millions of people daily can open the site without any problem at all.

Lots of servers, working very intelligently together, serving very intelligently written apps.


2. Do they register themself to be an ISP to reduce cost?

Well, if you're going to be consuming that much bandwidth, you bring in some incredibly expensive connections or you colocate with an ISP or hosting company. Colocation will probably give you the better bang for your buck -- you'll be put on a rack with a 10mb or 100mb connection, backed by shared redundant T3's or better. I think you can expect to pay around $100/month for this kind of service -- often far better than shelling out for the connections yourself.

3. Do the best solution to get extraordinary bandwith and speed with low cost is to register ourself to be an ISP? Any suggestions?
I dont believe in "unlimited bandwith" webhost solutions, im not looking for this kind of answer.

I'm not sure what your infatuation with "registering to be an ISP" is: regardless of the state of your organization, you need an upstream provider. You need to connect to an internet backbone, and that's going to cost you. The significant difference between an ISP and a not-ISP is that an ISP is reselling the bandwidth it has purchased.


4. With server using HP ProLiant SL390s G7, 2 processors + 192GB RAM. and let say in average there are 5 million daily page view on my site, = +200k page view per hour = +3k page view per minute = 58 page view per second. I know its hard to predict, but just approximately, for that kind of load, how many server do i need for optimum performance?

As you've stated, there's no way to know. If the site is simple enough, you could easily serve up 58 pages/second from a PII with 64mb RAM. On the other hand, if your site is complex enough, you may seriously struggle to serve 58 page/second from the server you're describing. And if your application is designed poorly, you could struggle to serve 2 or 3 pages per second with that beefy server.

Also consider a few things:

5 million page views will like come with a lot of additional HTTP requests. Even the google.com main page, which is a ridiculously simple page, is a 9 request page load -- 10 if you count the redirect most users will incur by typing in google.com instead of www.google.com. So, take your 5 million page views and assume you need to support 50 million HTTP requests per day.

Those requests will probably NOT be distributed evenly throughout the day. The distribution will depend a great deal on target audience, locale, and nature of the site. But, you can expect to have an uneven distribution: you'll have particular hours of particular days that stress your system, and you need to prepare for those. If you peak at 5m pageviews per day, I wouldn't be at all surprised if 50% of those pageviews came within a 4 to 6 hour span, with critical peaks within those hours.

Make some educated guesses about your traffic patterns -- when will people want to and be able to hit your site? Where are your visitors? Do they hit the site right before or after their lunch break? When they get home? etc.

You probably don't have to plan for 5 million pageviews right away. That would be a pretty phenomenal start. And if you think you can hit it within the first month or two, great. But, chances are, even if your site is hot ****, you'll have a month or two (or at least a couple weeks) to see what peak load does to your servers and hire some engineers and developers to accomodate your particular traffic patterns.

[noahqw]

All I have to say is you're not going to have to worry about this issue of "over bandwidth" until you start getting it. Websites that have existed for several years don't even compare to the traffic you're wanting/anticipating. Trust me... if you're planning on getting this amount of traffic your site will be in the top 99.999% percentile. With this said you're going to need serious server power. Probably more then 198GB of RAM and more then 2 processors. I'd suggest at least 500 GB of RAM with at least ten full fledged servers. Divide the RAM between these 10 servers and we get 50 GB of RAM per server.

When it comes to processing all this data you should go with top performance processors. At least a Intel Core i7-980X Extreme Edition Gulftown 3.33GHz 6 x 256KB L2 Cache 12MB L3 Cache LGA 1366 130W Six-Core Desktop Processor BX80613I7980X in each server. This is about $999.99 per processor.

Hope this insight helps man.

[svidgen]

When it comes to processing all this data you should go with top performance processors. At least a Intel Core i7-980X Extreme Edition Gulftown 3.33GHz 6 x 256KB L2 Cache 12MB L3 Cache LGA 1366 130W Six-Core Desktop Processor BX80613I7980X in each server. This is about $999.99 per processor.

Google would probably disagree with you on this, and so would I. It's more cost effective to run a lot of lower-end servers than it is to run on high performance boxes. When Google purchases new hardware, to the best of my knowledge, it targets hardware that yields the most power per dollar -- and that typically means low-end. Hardware performance does NOT scale linearly with cost. My guess would be some logarithmic scaling.

unit performance = C * (log(unit cost) / log(some base))

... whereas, by purchasing more units instead of more powerful units, you can expect something close to ...

overall performance ~= C * over cost

But, this also means your application needs to be designed to run on multiple servers -- a little extra cost up-front. But a good investment if you plan having anywhere near 5 million daily users ...

Also bear in mind, whatever fancy math you're doing to decided that processor A > B is probably wrong. Benchmarks matter. Performance = benchmark scores != CPU speed, cache size, etc.