Here’s a datapoint from my Leopard machine and LCD monitor (Dell 1905FP):

Appearance font smoothing setting:
Automatic – best for main display
Landscape: standard antialiasing
Portrait: standard antialiasing

Medium -best for flat panel
- Landscape mode: sub-pixel antialiasing
- Portrait mode: standard antialiasing

It may require relaunching the app in the new orientation to see the effects.

You can test it for yourself. Try the different orientations. Launch TextEdit with sample text. Screenshot. Open in Preview. Zoom in. If you see red and blue color fringing, it’s using sub-pixel antialiasing. If not, it’s using standard.

I’d heard that Automatic would recognize LCDs automatically and use font smoothing when appropriate, but apparently it’s not doing so for me.

The upshot is that I lose sub-pixel antialiasing when in portrait mode, and that for me, it’s best to leave font smoothing at Medium. Also, I suspect that I need to relaunch my app for the appropriate antialiasing routines to kick in and that it’s possible to be sub-pixel rendering incorrectly when I switch to portrait mode without relaunching.

Maybe they lost my report. :-)

http://www.bytecellar.com/archives/000090.php

I, too, had taken close-up photos of “normal” and rotated screen font rendering with subpixeling active:

http://pix.blakespot.com/view/computers/screenshots/osx_subpixel_bug/

Thanks to this bug, I must let the opportunities of subpixeling pass me by on my main 30″ display in order to utilize a secondary, rotated display.

http://pix.blakespot.com/view/computers/macpro/IMG_8796.JPG.html

Sadly, it’s standard anti-aliasing for me.

It’s worth mentioning that I noticed on my old Compaq iPaq PocketPC that it (“it” being Windows CE or Windows Mobile or Windows On The Hoof or whatever they call it now) handled rotation just fine. Yes, the “effect” was more noticeable when the subpixeling was benefiting the horizontal, but it was certainly better than no subpixeling when it was in use on the vertical.

Yes letterforms have distinguishing elements which break the vertical, but the point is that humans read faster when there is a steady horizontal rhythm to the verticals — like stripes. Type is kerned to improve that horizontal rhythm. Bad kerning makes things harder to read, this is yet another reason why sub-pixel text rendering across the horizontal benefits legibility. There is a long and rich history of western typographic evolution which brought us to horizontally syncopated letterforms.

The point here is the confluence of luck that made sub-pixel rendering work. It’s a hack, but it’s a hack that works because of how we see and how we read.

Gruber posted an excellent link explaining the use of subpixels: http://alienryderflex.com/sub_pixel/

The sub-pixel orientation is the reason black-to-white edges go from blue to cyan to white on the left, while the opposite transition goes red to yellow to white. Sub-pixels are being added to fill in horizontal resolution. We don’t perceive it, as you said, because our eyes are more attuned to brightness (contrast) than color.

“Sub-pixel antialiasing algorithms require to know the background color over which the glyphs will be draw. Since Core Animation is a compositing engine, and allows layers to have an arbitrary opacity, and does the final composite on graphics hardware (if possible), Quartz cannot assume anything about the background color and thus disables sub-pixel antialiasing.

This is bang on the answer as to why things work as they do with respect to this.