Unless you have some really great need to have streaming video "to stream video stored on my server", don't. Just upload the file and progressively download the file. True streaming servers can be 20 times+ more expensive than using a progressive download to serve up the same video.
If you need live broadcasts, like a Webinar, that's a little different. For broadcasting live envents I use Windows Media Encoder to encode and "push" to the streaming server. Free from Microsoft at
Your concern with download times/buffering is well founded. Here is where the encoding bit rate will come into play. The higher the bit rate, the better the quality. Depending on your audiance, try to determine the speed of their Internet connection. As a comparison, YouTube video is about 250kbps. So a 1.5MB connection could easily handle the job. However, a 56kbps dial-up modem would be swamped with about 5 time the input it could handle. If your viewers are expecting to view video, they should have at least a 250kbps Internet connection. For that, you would have to encode at 180kbps or lower...not very good quality.
By the time you get to 1.5MB connections you can encode at 400kbps and up. 400 would be considered average quality. If you are showcasing your video creations, you may want 600 or so.
The buffer time is set in the player and can be set to increase the delay, but really does nothing to increase download speed.