As you found, the document.write() command will erase the content of a page, but only when called after the page has finished loading (if it is called before the page has finished loading it will only insert content).
Now, there are a couple of different ways you could do this. If you add a new <div> element to your page and give it an id then you can simply add the text to the <div> element's innerHTML:
<button onclick="countup()">Click</button>
<div id="testID"></div>
<script>
function countup() {
var i = 0;
while(i <= 10) {
document.getElementById("testID").innerHTML += i;
i++;
}
}
</script>
That also places everything on the same line (but absolutely no spacing). The reason your code does not do this is because each number is inside of a <p> element, which is a block element by default (meaning it takes up an entire line, whether the inside content fills it or not).
Another method would be to create the elements in the while() loop dynamically so you end up with something like:
<button onclick="countup()">Click</button>
<script>
function countup() {
var i = 0;
while(i <= 10) {
var $e = document.createElement("span");
$e.innerHTML = i;
document.body.appendChild($e);
i++;
}
}
</script>
The above method creates a new <span> element (an inline element so it will stay on the same line) and adds the number from the while() loop, then places this new element in the <body>.