Zed: The Demoroniser is gloriously public-domain, and as far as I recall it was really easy to strip out the important bits and embed them into their own little scoop routine, so it ought to be equally easy to hack it into plugin form.

My take (note that I'm biased, as I wrote SmartyPants, a plugin that produces automated smart punctuation for Movable Type and Blosxom):

http://daringfireball.net/2003/02/short_and_curlies.html

How much trouble is it to copy from a web page, paste in a BBEdit window, hit cmd-opt-t, then hit return? BBEdit converts all the high-ascii chars to numeric (or named or hex) entities automatically. Via applescript this would happen in a blink, and the resulting text won't break your RSS.

That said, I think dismissing the practical value of well-stated typographic meaning as misguided attempts at suave text - or hanging it all on Robin Williams and her silly hat - is plainly reductive and quite wrong.

Of course you do, Anil, because you don't know what you're talking about.

Your momma.

Back on topic, I only mentioned Blogger because that's what's generating the RSS in this particular case. And, no question, lots of apps handle curly quotes incorrectly. But I think it's not unreasonable that most tools that generate RSS should either do a simple search and replace for funky characters or should accept input and output in an encoding scheme that can handle them.

Put another way, tools vendors know people are going to paste in funky quotes, why not simply throw in a couple of regular expressions to fix the problem before it happens?

This discussion is pissing me off at computer geeks in a way I haven't been pissed off since the days when they were making fun of people who couldn't code their own programs. What's wrong with you people? Are you so wrapped up in your codes that you've forgotten why they exist? The point is to be able to create on a screen whatever you want to create, and that includes decent-looking text. The same people who delight in creating monster programs to show off the latest Flash technology (that either crash most people's computers or take forever to load) also get impatient with users who simply want their text to look good. What the hell does it mean to say "the designers are theoretically right in the same way that communism is theretically the best possible economic system"? Are you trying to red-bait people who care about type? Jesus, by now we should be able to use any weird symbol ever created, let alone fucking em dashes; it boggles my mind that we're still clinging to ASCII. "I hate non-ASCII characters": do you realize how crazy that is coming from somebody in the 21st century who's supposedly at the cutting edge of technology? You might as well say "I hate anything other than ones and zeros"; that's all you really need, after all.

The thing I find the most interesting is that people are talking about the visual appearance of curly quotes or dashes, but not many are talking about the fact that's it's a grammatical error to put the wrong punctuation.

(Don't flame saying I've made error in that text, its possible, I'm French.)

Cory, let's get some facts straight:

Despite your claims, BBEdit, Mozilla, Movable Type, mutt, Google Groups, AppleWorks, Word, and every modern newsreader I've played with not only doesn't barf and die, it handles smart quotes and other Unicode characters beautifully. I don't know how you got the idea they didn't.

I don't have Mailsmith, Outlook, or Outlook Express but I suspect they work too. That leaves pine, elm, WordPerfect.

Programmers who care about their users know it's not acceptable to snub everyone who wants more than ASCII's 128 basic characters. Users who are content with them are in a very quickly shrinking minority.

You're wrong. Every single one of those apps has failed to correctly render a curly quote generated by one or more other apps for me, personally. The absence of problems for you does not indicate the absence of problems -- my car has never crahsed, so car crashes don't exist.

Well, I'd like to hear about what curly quotes caused the problem.

There are two parties in the conversation: the generating and the receiving app. If all those apps are able to correctly render a correctly generated curly quote then it seems likely your problems were due to an incorrectly-generated one.

(Obviously if you speak Gzorgenplotz, no one will be able to understand you. That doesn't mean everyone's broken, it just means you should speak a different language.)

> Every single one of those apps has failed to correctly
> render a curly quote generated by one or more other apps
> for me, personally.

Then you're doing something wrong, because it Works For
Everyone Else.

Would you mind pointing to the Mutt bug report you raised,
Cory?

Edited by author 02-24-2003 11:11 AM
Accurate and detail-obsessive notes about character rendering problems are unfortunately necessary. I've built systems that have to straddle the edge conditions between various layers, and have, on occasion, spent Weeks resolving problems that never should have been problems in the first place.

The key problem is that what looks like a curly quote - which I assume to mean using correct left and right single and double quotes - is not always that, nor is it always the same. The difficulty is compounded by the fact that ISO-8859-1 (one of the most common character sets) does NOT include the relevant quotes, but Windows-1252 does, largely by including them in the 128 through 159 space that is not specified in ISO code sets.

The relevant common layer for appropriate quotes is Unicode, but this is hampered by the fact that many operating systems do not have good, or standard, or any, method of accurately entering the relevant characters. Anbody know how you actually enter Unicode character 8220 in a Windows app?

The only knowingly safe way to do this for now is with the HTML character entities lsquo rsquo ldquo and rdquo (with ampersands and semicolons left off so you can SEE them). If you're cutting and pasting these, then you can expect to get real quotes from all apps. For now, anything else is luck of the draw.

In HTML browsers, they look like:

lsquo ‘
rsquo ’
ldquo “
rdquo ”

“Put it in ‘quotes’”, he said. So I did.

P.S.#1 Just to prove my point, I accidentally screwed up the entities in the example. Grrr.

P.S.#2 But just to prove my point that this is a luck of the draw thing - QuickTopic converts my entities to quote marks, so that when I edit this, I DON'T see my entities, I see the quotes themselves. BUT - I am no longer sure about what they are. They might now be windows-1252 quote marks, and thus they will break (or at least not display correctly) on any system that does not support that character set.

People on email will have to come and look at the QuickTopic archive pages, because you're not getting the full effect on email. Email doesn't send out a new page after an edit happens. Sorry.

"Anbody know how you actually enter Unicode character 8220 in a Windows app?"

I assume you mean character 201C (Unicode counts in hex, not decimal). It's easy to get. Type 2, 0, 1, C, Alt-x

On OS X, switch to Unicode mode, hold down the option key, and type 2, 0, 1, C.

At least you knew better what I meant than any Windows app I can find. (Although Unicode might be 'offically' in hex, the decimal codes work fine in many situations, such as when generating HTML versions.)

I repeated your instructions in several common windows apps and all I get is "201c" in my text. Was this instruction for a specific application or version of the OS? Or some specific combination thereof?

I'm not a Windows users and I don't know the limitations, but I tried it on WinXP in Wordpad and it works fine.

A little digging shows that this is dependent on having one of WinXP or Win2K - AND having a Unicode version of the application in question. An ANSI build simply can't support the Unicode characters, even if you have an input method for entering them.

For purposes of having the quotes at least come out correctly, there are plugins for Movable Type that coerce your entered material into being quote-correct.

I will still note, however, that nowhere in the experimenting of the last day has anything blown up on me. In most cases, I've had to use validation tools to find the problems, which don't even show up as problems in day-to-day use.