The idea is behind the repeat in css is that instead of loading one huge file as a background image, the image is broken down into smaller repeatable patterns.
For example, we want to fill in a background image on a header of the site, that is 960px width and 100px height.
We then create a gradient file, with blue on top and light blue at bottom, we simply create a 1px width image file with, (let us say for example), 100px height. We then use repeat-x to repeat the file horizontally to the right. The CSS and the browser will automatically repeat it 960 times.
Now, the size of the 1 x 100 image file is certainly going to be smaller than 960 x 100 image file.
With a simple math, assuming that 1 px = 1 byte :
1 x 100 = 100 bytes
960 x 100 = 96,000 bytes.
It is a whole lot faster to download 100 bytes and 96,000 bytes.
That is the basic concept.
As for your concern regarding the memory, processing and cache; There are a lot of variables involved in a web application. I am sure the matters of using a 1px vs 960px wide image is very small compared to the logic of the web application.
For my standard, I normally set the web page to be able to load in under 1 seconds on my development server, which can be on the same computer or over LAN. As for the real world, this load will vary, according to the web sever load and the client connection speed, but generally with 1 second on the development server, it will be quite fast to load on the real world.