This thread and it's predecessor have been rather lonely, since I keep answering my own question before anyone has a chance to respond!
I found this directive in php.ini:
short_open_tag = Off
This is what was disabling the use of <? So naturally I turned it on, and finally everything worked.
It's possible I used php.ini-recommended with this installation but used php.ini-dist in the past. php.ini-dist has this turned on by default.
On this topic, what are the other important differences between the two ini files?