[NogDog]

If it makes you feel any better, even the Chicago Manual of Style is kind of wishy-washy about it:

6.3 Punctuation and font: primary system

All punctuation marks should appear in the same font—roman or italic—as the main or surrounding text, except for punctuation that belongs to a title or an exclamation in a different font. This departure from Chicago’s former usage serves both simplicity and logic.

Smith played the title role in Hamlet, Macbeth, and King Lear; after his final performance, during which many in the audience wept, he announced his retirement.
Many editors admire Wired Style: it is both elegant and easy to use.
An Apache Life-way: The Economic, Social, and Religious Institutions of the Chiricahua Indians
Are you saying the wound was self-inflicted?
She is the author of What Next?
For light entertainment he reads King Lear!
The manual Online! is always at my elbow.
We heard his cries of “Help!”

6.5 Punctuation and font: alternative system

According to a more traditional system, periods, commas, colons, and semicolons should appear in the same font as the word, letter, character, or symbol immediately preceding them if different from that of the main or surrounding text. In the first example in 6.3, the first two commas and the semicolon would be italic. Question marks and exclamation points, however, should appear in the same font as the immediately preceding word only if they belong to a title or an exclamation (see examples in 6.3).