Every webpage is made up of what are called stacking contexts. You can think of these as, quite literally, a stack of elements. The z-index property determines the order of items in each stack, with higher z-index being placed further up.
All pages begin with a root stacking context, which builds from the root element (as you'd expect). But more stacking contexts can be created in a number of ways. One way is an absolutely positioned div; its children will be in a new stacking context.
The specs lists all of the instances that create a new stacking context. As others have stated, this includes explicitly positioned elements and will soon include elements that aren't completely opaque.
z-index only takes effect if you explicitly set the position of the element. This means setting it to be fixed, absolute, or relative.