John Gruber linked to the news that Lucas is giving in to demand (eg. money) and will finally release the Star Wars trilogy on DVD in their original, unaltered theatrical format. Han shoots first. John writes:

Bastards. I broke down and finally bought the current DVD trilogy collection just a few months ago now I’ve got to pay for it yet again just to get the versions of the films that I really want.

Hold your wallet. The release will be in SD, despite the news that HD DVDs will start appearing in stores this summer. Lucas, who is as brilliant a money-maker as he is a horrible dialog-writer, will rake in tons of money selling an obsolete DVD. All the Star Wars films will be available again in HiDef soon enough. And too many people will buy them all for the third, fourth or fifth time.

Lucas isn’t alone. All the studios are flooding the DVD market right now to sell as much as possible before the switch to Blu-Ray or HD-DVD.

Supposedly the porn industry likes likes Blu-Ray best, but you’d have to be delusional to think physical pornography sales are anywhere near as strong as they were before the Internet. Still, porn is generally credited with choosing VHS over BetaMax, at least this time they seem to be going with the better format.

Personally, I think HD-DVD may win this time. Not because it’s better, but because “Blu-Ray” is a dumb name. Ask any non-technical consumer what they want for their new HD TV, HD-DVDs or Blu-Ray disks. Sure a certain percentage of people will have Blu-Ray explained to them, “see, they’re both HD,” but how many will buy on name alone?

Never mind that the name itself looks (and sounds) like blurry. Not what you want to hear when dropping a lot of money to replace a bunch of movies you bought in SD a few years ago.

And so the studios might be slitting their own throats. Movie attendance is down over the past several years. Partly movies are kind of boring now, also suspense and action movies keep dancing around what’s really scary. But the studios now want consumers to re-buy movies they just bought. The great DVD migration wasn’t that long ago, most DVDs were probably purchased within the past 5 years. But now Hollywood wants everyone to buy those movies yet again, and they’re going to spend a fortune on conversion and manufacturing in a bet that consumers will buy the new disks. And if we don’t? Lots of bankruptcies.

I did my annual whirlwind trip to Las Vegas for 40 hours of NAB.

Great to see everyone and meet some new faces. This year though it seemed I missed more people than I saw. However I did get to spend some good time talking with Christoph Vonrhein of CHV. Christoph is scary smart and pushing FXScript way beyond what has been done before.

Buzzless

Not much to get really excited about this year. The whole show seemed to lack a certain energy it’s had in the past. If anything it seems like HD is fully here and now people are figuring out how to work with it.

Lots of people around the show seemed to be complaining about various aspects of Panasonic’s P2 cards. Everyone loves how the video looks, but was down on the workflow. Complaints I overheard included the constant swapping of cards, their still astronomical cost, the lack of third-party cards, 8gb maximum available size, the lack of workable hacks (FireStore excepted) contributed to a general malaise about the format.

Far more hostile were the descriptions of working with HDV. It’s great that the format has a relatively low buy-in cost, but people were spitting blood about working with the files. HDV users seemed angry, frustrated and annoyed. And a lot of them are jealous of the Panasonic image quality, HDV looks mushy by comparison. I heard several people who are using HDV cameras steering others away from HDV cameras.

What that basically tells me is that the prosumer HD space is still very much up for grabs. Maybe that’s why the only perceptable buzz at the show was from RED Digital Cinema.

Except for RED

A lot of people seem to think RED is a joke. I’d be feeling the same way except that I know two people working at the company, Ted Schilowitz and Graeme Nattress. They’re both brilliant and dead serious about this camera. I have total confidence in them, this camera is no joke. Despite the fact that it looks more like a weapon from Unreal Tournament than a camera, and with a $17,500 price tag, the RED camera will completely transform the digital filmmaking marketplace.

In the near-term, HDV may yet become the interim standard. It’s just so much cheaper than anything else. Dealing with HDV files is really a software problem, so maybe Apple will shake a better workflow out of Quicktime and FCP making HDV more fun to use.

Overall it was a great trip. Sometimes, working alone in my little dark office, I forget there are people using the stuff I make. I know how absurd that sounds, but it’s real. Meeting so many Joe’s Filters users and hearing stories about how my filters helped people was revitalizing. Thanks to everyone, there’s more stuff coming soon.

Part of my FXScript compiler works by joining two code chunks with a shell script. Each chunk lives in its own file and contains one “high-ASCII” character, a © symbol in one, and a ’ (typographically correct apostrophe) in the other. Those are processed with sed and joined with a few additional strings via echo and cat.

For several hours I was stumped because one of the two characters would be garbled after passing through the script.

Finally I noticed that one source file was encoded as ASCII and the other was UTF-8. When both were set to UTF-8, everything worked.

The iconv command converts files between encodings. I used the following script to covert a directory of ISO-8859-1 Latin1 text files to UTF-8:

for f in *
    do 
    cp "$f" "$f.TMP"
    iconv -f LATIN1 -t UTF-8 "$f.TMP" > "$f"
done
rm *.TMP

Here’s a one-line version:

for f in *; do cp "$f" "$f.TMP"; iconv -f LATIN1 \
-t UTF-8 "$f.TMP" > "$f";  done; rm *.TMP

Just don’t run that more than once or it will re-convert already converted characters which isn’t pretty. Iconv doesn’t buffer data, so attempting to convert in place results in zero-length files. I moved the files first to keep Subversion from freaking out that the files were all new.

As much as it seems like something that should be detectable on the surface, 8-bit text encoding can’t be sniffed out.

It’s completely impossible to detect which of the 8-bit encodings is used without any further knowledge (for instance, of the language in use). …

If you need a formal proof of “undetectability”, here’s one: – valid ISO-8859-1 string is always completely valid ISO-8859-2 (or -4, -5) string (they occupy exactly the same spots 0xa1-0xff), e.g. you can never determine if some character not present in another set is actually used.

That’s the reason I couldn’t find a counterpart to iconv which would detect and return the encoding of a text file. An alternate solution would be to detect UTF-8 and not reconvert a file that’s already unicode, but I think I’m done with this for now.

For a beginning understanding of Unicode and text encoding, start with Joel Spolsky’s canonical article, The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!).

I got my FXScript Compiler working on the new machine and pulling sources from Subversion without too much trouble. But I decided that my practice of embedding shell scripts in AppleScript kind of sucks. It’s just desperately ugly. Tools like sed are ugly enough on their own, slashing and escaping every other character just makes them completely impossible to dissect after a few months.

For example, I print the following in the header of each file’s source code before compiling:

[tab] // [tab] Version: r145
[tab] // [tab] build200604181617
[tab] // [tab] April 18, 2006

One echo statement looks like this ($d and $b are already set and $b is formatted):

echo -e "\t//\tVersion: $d\n$b\n\t//\t`date '+%B %d, %Y'`\n\n

Not exactly pretty, except in comparison to this:

echo -e \"\\t//\\tVersion: $d\\n$b\\n\\t//\
\\t\`date '+%B %d, %Y'`\\n\\n\"

Echo is using the -e argument because these are being piped through other commands.

The real killer is anything involving regular expressions. Say a matching pattern needs to match a string containng double-quotes inside a double-quoted sed pattern. Then this already slash-infested command:

sed -e "s/\([Ff]ilter[\t ]*\"[^\"]*\)\"/.../"

becomes this:

do shell script "sed -e \"s/\\([Ff]ilter[\\t ]*\
\\\"[^\\\"]*\\)\\\"/.../\""

No part of me wants anything to do with keeping track of that many backslashes. It’s slightly better when using sed with the -E extended regex flag, but still.

In the ongoing pursuit of long term legibility, I’m putting my shell scripts functions into individual files inside the XCode project. More on how that works later.

[I know the main page template doesn’t work right when long strings break the column width, it’s on my long-term to do list]

A few days ago I was struggling to install CS2 on my MacBook Pro. For whatever reason, it just would not install on my system in either user account. Each failed attempt took about 30 minutes before crapping out. Adobe’s CS2 uninstall instructions should be embarrassing, especially for a $22.36B company selling a $1600 product.

So anyway, I start writing a shell script which goes through and rips out all the tidbits of CS2 that Adobe’s installer barfs all over the hard drive. Nothing fancy, just a list of rm -rf statements pointing a dozen or so various spots around the volume.

I’m also feeding Noemi while Lila takes her afternoon milk break and watches Blue’s Clues.

Long story short, I got interrupted typing out the location of the last remove statement. Interruption causes me to forget that I was in the middle of typing something. I come back to the keyboard, try to remember where I was and decide to run the script, to, you know, see what still needed deleting. Where did I leave off? Here:

And then my entire home folder happily deleted itself.

I back up frequently (though not frequently enough) and luckily only lost a few dozen emails I’d filed the day before. Still, I did get to enjoy that moment of tunnel-vision panic where all the blood in my face seemed to rush to the back of my neck.

Could have been a total catastrophe, it wasn’t because I had a very recent backup. I recommend SuperDuper! without hesitation.

Update: The installation problem turned out to be related to a Quicktime update. Adobe and Apple straightened it out and I had no trouble with a recent CS2 install onto a new MacBook Pro at IOP. My original installation succeeded because I chose not to install Version Cue, which we’ve never used anyway.

Apparently Sea World in Queensland are renaming their Fairy Penguins to “Little Penguins” in a pathetically unnecessary grasp at political correctness.

This quote by Gold Coast Breakers chairman Kamahl Fox is the highlight

“I wouldn’t be upset by fairy penguins at all. I don’t think our community is that sensitive about those things. If the penguins were called poofter penguins or something more direct then it might be a problem, but I don’t see the name fairy penguin as a mickey-take at all.”

No word on how New York’s own “fairy” penguins received the news.

Wikipedia, as usual, takes all the fun out of this. Apparently “little penguin” is the more common name for Eudyptula minor, Fairy Penguins is the popular Aussie name for the little birds.

I’m all for sticking it to political correctness, but that seems like a rather significant bit of information that should have been included, it’s not like Wikipedia is some obscure resource. The same story as reported by the Herald Sun does sort of mention it at the end, they also had more fun with the title, Gay old time over a little fairy bird

I turned on comments again for The FXScript Reference, hopefully the spammers won’t show up right away.

24 hours later… It only took four hours for the first drug spams to appear, four in 24 hours and zillions of referrer hits.

This time I’m going to try something different. Almost everything is posted to the k30fps constant, which isn’t particularly interesting, for whatever reason the spammers are pounding that page. So I’m reassigning k30fps which will change its url, I’m also and adding a custom rule to 403 that page. (otherwise they’ll bring down my 404/search page).

The vengeful part of me would love to forward the spammer’s requests to some huge file or link to their Windows Registry or something. But I’m not going to waste someone else’s bandwidth and I don’t want some innocent user to end up borking their registry because of me. Akismet, someday.