Ripping and Burning DVDs with Mac OS X

The simplest way I've found doesn't require installing any additional third-party software at all. Just use Disk Utility.

A number of people told us how much they enjoyed the poems and songs at our wedding in October. We'd like to share them with you here, including the opening remarks by Whit and the amazing toast by Cheng-Hsi's father.

Whit's opening remarks.

Welcome friends and family to this glorious day here in Boulder, Colorado. We all want to acknowledge the parents of the bride, Sobee and James, and those of the groom, Bob and Ginny. And of course, this could not occur without the entire cast of family and friends.

This is a great day. Let the celebration begin.

The great German philosopher Goethe said “Love is an ideal thing, marriage is a real thing”. Today we gather here to witness the marriage vows of Nicholas Andrew Dronen and Cheng-Hsi Ho. This marriage, my friends, is the real thing. It is no accident.

A very smart individual, Dr. Albert Einstein, commented: “Graviation can not be held responsible for people falling in love”. This marriage is far more profound than the passive forces on this earth. Nick and Cheng-Hsi – their relationship – is far more stunning than chance.

Nick met Ms. Cheng-Hsi Ho relatively recently. They had a very comfortable friendship. One evening my wife Mary and I had a dinner with the now bride and groom. We were at our home, and could recognize something very special between these two. Nick, at that time, did not. Yes, he thought Cheng-Hsi was a great woman and a fine friend.

But give Nicholas credit. He dug deep (at times, over the years, we weren’t sure he would or could), and he recognized that Cheng-Hsi was a really special woman. Someone he could love, cherish, and most importantly, share his life with. This is Nick's part on this specialday. He recognized Cheng-Hsi Ho. He saw deeper than just the everyday activities, the meals they were sharing. If this would not have happened – recognition – we would not be gathered here today.

Cheng-Hsi as well had her part to play to bring us all together today. Recognition of Nick of course was essential to walk the path of marriage. But let’s look closely at Ms Cheng-Hsi Ho. Cheng means receiving something old and making it better. Hsi means happiness. I have studied Chinese philosphy and the traditional medicine of China for my entire adult life. However, I don’t have command of the language, so please, give me some leeway here in my interpretations. With Cheng, we are talking about a process of transformation ... of taking the routine daily activities of life, sometimes not all that glamorous, and making them into glorious events. Making better old routines, old patterns of behaviour.

Cheng-- receiving, transforming. Hsi-- Happiness. Who could ask for anything more?

And that is where we, the community of friends and family, come in. We have the job of recognizing the marriage of Nick Dronen and Cheng-Hsi Ho. It is our job, as their friends and family, to witness the commitments and the vows that we are about to hear. And we need to embrace them, in support of our dear friends. Because, God knows, there will be times they may forget how happy they are in this moment Let’s make sure we remind them, and reflect this back to them, to support them, to nourish their marriage.

Interestingly, Nicholas, from the Greek language, comes from two root words – Victory and People. We are the community – The People – and this sure feels like Victory to me.

May Nicholas and Cheng-Hsi succeed beyond our and their wildest dreams. The great Chinese philosopher, Lao Tze, says: “Nature does not hurry, yet everything is accomplished.” We have the entire Universe supporting us. There is nothing to worry about.

Friends, let us continue with the Celebration. I call forth my wife, Mary Saunders, to read several poems.

Poetry read by Mary Saunders.

I cursed the rain for

pounding upon my roof and driving

away sleep.

I cursed the wind

for ravaging my garden.

Then you entered; and I gave thanks

to the rain because you must put off

your wet dress; and I gave thanks to

the wind

that he came and blew out my lamp.

-- Chang Wu-Chien

O Marvel! a garden amidst the flames.

My heart has become capable of every form:

it is a pasture for gazelles and a convent for Christian monks,

and a temple for idols and the pilgrim's Ka’ba,

and the tables of the T

orah and the book of the Quran.

I follow the religion of Love: whatever way Love's camels take,

that is my religion and my faith.

-- Ibn Arabi

The TrueLove

There is a faith in loving fiercely

the one who is rightfully yours,

especially if you have

waited years and especially

if part of you never believed

you could deserve this

loved and beckoning hand

held out to you this way.

I am thinking of faith now

and the testament of loneliness

and what we feel we are

worthy of in this world.

Years ago in the Hebrides

I remember an old man

who walked every morning

on the grey stones

to the shore of baying seals,

who would press his hat

to his chest in the blustering

salt wind and say his prayer

to the turbulent Jesus

hidden in the water,

and I think of the story

of the storm and everyone

waking and seeing

the distant

yet familiar figure

far across the water

calling to them,

and how we are all

preparing for that

abrupt waking,

and that calling,

and that moment

we have to say yes,

except it will not come so grandly,

so Biblically,

but more subtly

and intimately in the face

of the one you know

you have to love,

so that when

we finally step out of the boat

toward them, we find

everything holds

us, and everything confirms

our courage, and if you wanted

to drown you could,

but you don’t

because finally

after all this struggle

and all these years, you don’t want to any more, you’ve

simply had enough

of drowning, and you want to live and you

want to love and you will

walk across any territory

and any darkness,

however fluid and however

dangerous, to take the

one hand you know

belongs in yours.

-- David Whyte

Poety read by Jeff Pontillo

You know, my Friends, how bravely in my House

For a new Marriage I did make Carouse;

Divorced old barren Reason from my Bed

And took the Daughter of the Vine to Spouse.

For "Is" and "Is-NOT" though with Rule and Line,

And "UP-AND-DOWN" by Logic I define,

Of all that one should care to fathom, I

Was never deep in anything but -- Wine.

-- Rubaiyat by Omar Khayyam

The song performed by Wendy Lewis (vocals, guitar), Kitty Jenson (vocals), and Ted Dronen (guitar) was "True Love Will Find You in The End," composed by the gifted songwriter Daniel Johnston:

True love will find you in the end

You'll find out just who was your friend
Don’t be sad, I know you will,
But don’t give up until
True love finds you in the end.

This is a promise with a catch
Only if you're looking will it find you
‘Cause true love is searching too
But how can it recognize you
Unless you step out into the light?
But don’t give up until
True love finds you in the end.

Here's a version performed by Beck (Amazon, Nick's web server). I encourage you to buy the MP3 from your favorite download site.

Finally, the magnificent toast by Cheng-Hsi's father, James Ho.

Ladies and gentlemen, I am Cheng-Hsi's father. Please look at me carefully. Today is a big day. It is also an historical moment. We are so glad to attend this wedding ceremony. Congratulations! Nick, you are so lucky to marry my daughter, Cheng-Hsi. She is very kind, diligent, and humble. Also she is as beautiful as an angel. You are now building a home, a sweet home, and a family. I love her so deep and so much. She is my heart. Please love one another ever and forever. Well, if i were President Obama, I would speak fluent English. If you do not understand what I say, please pretend to understand. Ginny! Bob! I really love my daughter. Please love her as your own. Nick! I trust you and believe you are a good husband. And certainly Cheng-Hsi is a good wife. I promise you, Nick! You have something to do, whatever that is, to contribute to your family, society, nation, and all mankind. That is my wish. God bless you! Thank you.

Who is Don Draper?

Even Don Draper probably doesn't know who Don Draper is, but what we know is that he has two selves -- a private self and a public one, a past self and a present one, one that he is and one that he pretends to be -- and what's withheld and revealed about them constitute much of the show's appeal. You have to give Matthew Weiner credit for creating a great character. I suspect he put a lot of thought into Don, and I'm certain that Don's names -- Dick Whitman and Don Draper -- are deliberately crafted as hints to the answer to the question, who is Don Draper? Certainly, other characters have names that somehow give you a sense of who the character is in their particular world -- Peggy Olson and Pete Campbell come to mind -- but Don is a man with more names than one, and he's the simmering star of the show (except for Peggy, of course), so without further ado, I'll explain them.

Dick Whitman. First name first. This one needs no explanation, just the background, but I really have to wonder whether Don's philandering (or, this season, general man whorishness) is motivated by a desire to connect in some way to his late mother. Whitman. This one's hard to explain, but I've come to think of his advertising genius as a kind of American poetry, sort of like this guy's. It's almost messianic, like he sees his role as the caretaker of the American psyche. Take, for instance, the first line in this important moment between Don and Peggy last season:

Don Draper: No. Because there are people out there who buy things, people like you and me. And something happened. Something terrible. And the way that they saw themselves is gone. And nobody understands that. But you do. And that’s very valuable.

Peggy Olson: Is it?

Don: With you or without you I’m moving on. And I don’t know if I can do it alone. Will you help me?

Peggy: What if I say no? You’ll never speak to me again.

Don: No. I will spend the rest of my life trying to hire you.

And finally, Don Draper. This one's actually pretty easy. From Webster's Ninth New Collegiate Dictionary:

don vt 1: to put on (an article of wear) 2: to envelop oneself in :ASSUME

draper n, chiefly Brit: a dealer in cloth and sometimes also in clothing and dry goods

drapery n 2: a: a decorative piece of material usu. hung in loose folds and arranged in a graceful design b: hangings of heavy fabric used as a curtain

The idea being, of course, that Don Draper -- the cipher, as he's called in the fourth season's first episode -- is a facade.

The Wrongness, The Wrongness!

This is so wrong. NCAR, the National Center for Atmospheric Research, in Boulder is opening a supercomputing facility in Wyoming. Their facility in Boulder isn't suited to house the massive clusters of computers needed to support the center's research, which includes the study of climate change. A mere 10% of the Wyoming facility's power will come from wind. The remaining 90%, presumably from coal. Armageddon reimagined: supercomputers dedicated to research on climate change consume more and more coal-generated electricity, which adds more carbon to the atmosphere, thus accelerating the warming of the planet, resulting in gobs more federal funding for the study of climate change, and more supercomputers, with the concomitant carbon -- an accelerating cycle of supercomputers, carbon, and rising temperatures.

These Made Me Ha-a-appy

I'm putting together a list of flicks for a fellow cinephile. The criteria for the list are (1) it should be something I think he hasn't seen and (2) it should have been made in the last ten years. There will be exceptions to the second criterion. It was originally going to be all Asian films, too, but there are just too many good ones made everywhere. Ultimately, they're in the list because watching them was profoundly pleasurable.

• You, the Living Sweden, 2007
• Silent Light Mexico 2007
• Memories of Murder Korea, 2003
• Mother Korea, 2009
• Yi Yi Taiwan, 2000
• Goodbye Solo USA, 2008
• Cache France, 2005
• Syndromes and a Century Thailand, 2006
• Beau Travail France, 1999
• Zatoichi Japan, 2003
• Let the Right One In Sweden, 2008
• Private Fear in Public Places France, 2006
• Waltz with Bashir Israel, 2008
• Taste of Cherry Iran, 1997

Enjoy!

When 5% is too much

It's just criminal that a single industry generates %5 of man-made carbon:

Cement manufacture contributes greenhouse gases both directly through the production of carbon dioxide when calcium carbonate is heated, producing lime and carbon dioxide, and also indirectly through the use of energy, particularly if the energy is sourced from fossil fuels. The cement industry produces about 5% of global man-made CO2 emissions, of which 50% is from the chemical process, and 40% from burning fuel. The amount of CO2 emitted by the cement industry is nearly 900 kg of CO2 for every 1000 kg of cement produced.

Why do News Corp editorials hate reality?

In an interview in this week's edition of Barron's, Ray Dalio of Bridgewater Associates, a fund with institutional assets of $75 billion, explains in six simple sentences why inflation is not a problem right now. This is from the print edition; the interview is available online to subscribers.

Explain why the printing of money won't cause inflation.

The printing of money will offset the deflation that is coming from the weak demand for goods and services due to weak credit growth. For example, in March of 1933 the U.S. printed a whole lot of money, and that had the effect of converting deflation into modest inflation, but not a high rate of inflation.... My point is, in developed countries there is too much of most things at the moment, ad that's creating a deflationary environment. There is too much manufacturing capacity. There is too much labor. There is too much housing stock.

Six sentences, including actual facts, spoken by someone who is actually responsible for other people's money (and a lot of it). Compare this to last week's inflation hawk editorial by Barron's editor Tom Donlan, in which he seems to desire to make the opposite case -- an argument for contractionary policies -- but instead wanders off on an embarrassing digression about old bonds and barely manages to burp out some you'll-be-sorry warnings about inflation.

This is another case of a financial paper's journalism being sane and the editorial pages being completely unhinged from reality. Where else does this happen? Why, Wall Street Journal, which is owned by News Corp, which owns Fox News and, of course, Barron's.

You would think that an editorial that's subtitled "If the Fed won't tighten up, we'll pay the price" would present a case for why the Fed should abandon the current policy of quantitative easing in favor of something more contractionary. That's what I expected when I turned to the last page of this week's print edition of Barron's, but that's not what found. Instead, Tom Donlan fills most of the page with a story about bonds issued three decades ago, the point of which appears to be that things don't always turn out as you expect. But saying, in short, "shit happens" isn't a compelling argument in favor of any particular policy, much less the one that the editorial piece ostensibly advocates. Does journalism get any lazier?

The article isn't completely free of argument. There is an inkling of one towards the end, where he writes "Wall Street doesn't have a money market; its interest rates reflect Fed policy, as do Main Street consumer prices." Now this isn't really an argument, but it's the closest Donlan gets to the kind of writing one expects in a piece advocating a Fed policy change. The dependent clause -- "as do Main Street consumer prices" -- is an assertion. And it's not false; Fed rates and the state of the economy as a whole are correlated. When the economy overheats, the Fed raises interest rates, and there's a recession of some magnitude. The Fed eventually responds by lowering rates, which lets economic growth resume. That's the standard picture.

From there, though, it just gets childish:

It's easy to imagine that inflation is irrelevant to a U.S. economy that's posting the lowest increases in consumer prices in 44 years. But "imagination is funny; it makes a cloudy day sunny," as Johnny Burke wrote for Frank Sinatra in 1940.

Sunny days of low inflation won't last.

And it would be easier for us to imagine that what he says actually matters if Donlan had bothered to make an argument. Low inflation won't last. So what? When inflation becomes palpable to Fed policy makers, they'll raise interest rates. But if there's no sign of inflation now, what reason is there for Fed policy to become contractionary now? Hello?

UPDATE: The New York Fed chairman disagrees with Donlan too.

I saw Bong Joon-Ho's "Mother" last night at the Lyric Cinema Cafe in Fort Collins. The style, theme, and humor are similar to his "The Host" and "Memories of Murder".

They're all awesome. "Memories of Murder" should have been on at least one major critic's best-of-decade list, but perhaps it was elided because Bong plays with genre too much to be considered alongside the directors of consistently sublime movies like "Syndromes and a Century" or "Yi Yi". In any case, if you like yourself some Asian cinema, watch "The Host", "Memories of Murder", and "Mother" (in order of accessibility).

iPhone, Safari, and tel URI's

I think it's very cool that tapping a tel URI in Safari on an iPhone allows you to actually call the number. (A "tel URI" is basically a phone number link, similar to an "http:" or" mailto:" link, except it starts with "tel:".) Because not all phone numbers start with "tel:" on the Web, Safari tries to detect them so you can just tap and call, but it needs to be improved. For example, document citations often contain text that Safari wrongly believes is a phone number, such as the text after the comma in this fragment of a review: "No. 4, 541-578 (2009)". My phone thinks this is the number (541) 578-2009.

Futher brokenness is evident when I tap and hold the link: a pop-up menu appears with the option to "Call 541-578 (2009" (with no parenthesis at the end).

Rewatching Mad Men (cont.)

I believe this scene is key to understanding why Campbell shows up at Peggy's apartment that night. Here's the pattern. First, Draper thwarts Campbell's desire to engage in a sort of corporate-ladder-climbing intercourse. Clearly, given the way the scene is shot, and given the bile of Campbell's "f*ck you" at the end, sex is a weapon for Campbell. Later, when the other Sterling Cooper lads take him out for his stag party, Campbell brazenly puts the moves on one of the gals who joins the lads at the strip club. Here again he confuses force for flattery, and again he's rejected and humiliated, only this time in front of his peers.

Campbell mixes desire and power to the point that they're indistinguishable. Being humiliated by Draper in the afternoon, and in front of his peers at the strip club, forces his hand. He goes to Peggy's apartment not -- or not strictly -- because he wants some wanton nookie before he gets hitched, but to get something that Draper has (or so he thinks). Sleeping with Draper's secretary is Campbell's way of getting revenge. Again, sex to him is only a weapon. This is, in part, why Peggy's speech at the end of the second season -- the one that left me and probably you breathless -- is so powerful: what she tells Campbell shocks him; he's surprised; but it's not about the baby. It's about what she says and how she says it and the fact that she is a woman. And it's about Campbell's idea of who he is.

Shots from "Syndromes and a Century"

I watched "Syndromes and a Century" again last night after seeing it about a year ago. Something about it is so fresh and vibrant, even after multiple viewings.

Here are two powerful shots -- the first is from the rural vignette, the second from the urban one -- posted just because I think they're beautiful.

Readings on Analogy

I'm thinking about analogy at work. Some

• The UberCruncher: Concept Formation by Analogy Discovery
  
• Human-Level Performance on Word Analogy Questions by Latent Relational Analysis
• A Solution to Plato's Problem: The Latent Semantic Analysis Theory of the Acquisition, Induction, and Representation of Knowledge
  

Rewatching Mad Men

It may be cheap sport, playing spot the symbols of sex and power in an episode of "Mad Men," but rewatching the first episode, it's too obvious to pass up. It's about at the 24-minute mark, right after the distasterous meeting with Rachel Menken.

Campbell: I'm not going to pretend I don't want your job, but you were right, I'm not great with people, and you are, I mean, not counting that meeting we were just in, so I'm kinda counting on you to help me out.... There's plenty of room at the top.

Draper: Look, I'm sorry I was so hard on you before. It's just this damn tobacco thing.

Campbell: You'll think of something. [Emboldened.] A man like you I'd follow into combat blindfolded, and I wouldn't be the first. Am I right buddy? [Presents hand to shake.]

Draper: Let's take it a little slower, I don't want to wake up pregnant. [Walks away.]

Campbell: [Under his breath.] F*** you.

Sometimes a cigar is just a cigar and sometimes the outstretched hand of your weasel of an underling is just a phallic symbol appearing on your TV screen.

Oh, and would it be too much to note that the previous scene features Peggy Olsen being humiliated by the gynecologist when she goes to him to get oral contraceptives? I think not.

Independent Study Reading List, Spring 2010

• The Missing Link - A Probabilistic Model of Document Content and Hypertext Connectivity
• Recursive Attribute Factoring
  
• A Measure of Similarity between Graph Vertices with Applications to Synonym Extraction and Web Searching
• Higher-order web link analysis using multilinear algebra
  
• A Course on the Web Graph
• From paragraph to graph: Latent semantic analysis for information visualization
• Web search via hub synthesis
• Spectral filtering for resource discovery
• Finding authorities and hubs from link structures on the world wide web
• Graph structure in the web
• The structure of broad topics on the web
• Topical locality on the web
• Towards logical hypertext structure - a graph-theoretic perspective
• The connectivity sonar: detecting site functionality by structural patterns
• Knowing a web page by the company it keeps
• Human-Level Performance on Word Analogy Questions by Latent Relational Analysis
• Trawling the web for emerging cyber-communities
  

A Memory of Vic Chesnutt

After I graduated from high school, I got on a bus and went to Athens for two weeks, and had the best possible experience an 18-year-old music junkie could have had at the time. It's unbelievable but true. I met Michael Stipe (how exactly is a story in itself) and ended up spending almost a half day with him. We hung out in the R.E.M. office in downtown Athens while listening to Pylon rehearse in the basement. We went to his house and had a drink. We went to a cafe and later, before he dropped me off at the city park where I was camping, a bar. The next morning I got kicked out of the park and ended up camping uninvited on the roof of the R.E.M. office. The woman who ran the fan club noticed me through a window sleeping in the sun. I woke up with a bag next to me; it had a tube of sunscreen and some fruit. Turns out her boyfriend, Armistead Wellford, was the bass player in Love Tractor. She arranged for me to stay in a big old house where a lot of the band members lived. All of that was more than I imagined would happen when I got on the Greyhound bus in Fargo, North Dakota.

But in retrospect the most significant thing I saw there was when Stipe and I were at the cafe. He introduced me to some people at a table -- two women who I don't remember, and a man in a wheelchair, Vic Chesnutt. A cassette player played recording of one of Vic's live performances. While we sat at the table and talked, Stipe told Chesnutt that he'd like to help him record a record. It was the summer of 1988. Two years later New West Records released Chesnutt's Little, which Stipe produced.

Wiki terminology, like wikipedia, not optimized for search.

In MediaWiki terms, including a page in another page is called transclusion, an accurate word, but not one that automatically comes to mind. I searched for "macros" and "include" for a while and, after a few minutes, found a page that mentions transclusion.

More papers to read

From a comment on the Natural Language Processing blog, classic papers in NLP:

• [Bahl et al., 1983] L.R. Bahl, F. Jelinek and R.L. Mercer. "A Maximum Likelihood Approach to Continuous Speech Recognition." IEEE Journal of Pattern Analysis and Machine Intelligence.
• [Charniak, 1983] Eugene Charniak. Passing Markers: A Theory of Contextual Influence in Language Comprehension, Cognitive Science, 7, pp. 171-190.
• [Charniak, 1973] Jack and Janet in Search of a Theory of Knowledge. In Proceedings of the International Joint Conference on Artificial Intelligence (1973)
• [Charniak, 1977] Eugene Charniak. Ms. Malaprop, A Language Comprehension Program. In Proceedings of the International Joint Conference on Artificial Intelligence (1977).
• [Cohen et al. 1982] Philip R. Cohen, C. Raymond Perrault, and James F. Allen. Beyond Question Answering. Strategies for Natural Language Processing, pp. 245- 274.
• [Grosz, Joshi, and Weinstein, 1995]. Centering: A Framework for Modeling the Local Coherence of Discourse. Computational Linguistics, 21 (2), pp. 203-226.
• [Grosz and Sidner, 1986]. Attention, Intention, and the Structure of Discourse. Computational Linguistics, 12 (3), pp. 175-204, 1986.
• [Hobbs et al., 1993]. Interpretation as Abduction. Artificial Intelligence, vol 63. pp. 69-142.
• [Hobbs, 1979] Jerry Hobbs. Coherence and Coreference, Cognitive Science 3(1), pp. 67-90.
• [Hovy, 1988] Hovy, E.H. 1988. Planning Coherent Multisentential Text. Proceedings of 26th ACL Conference. Buffalo, NY.
• [Karttunen, 1969] Lauri Karttunen. 1969. Pronouns and variables. In CLS 5: Proceedings of the Fifth Regional Meeting, pages 108-116, Chicago, Illinois. Chicago Linguistic Society.
• [Kay, 1986] Martin Kay. Parsing in functional unification grammar.
• [Lakoff & Johnson, 1980] George Lakoff and Mark Johnson. Metaphors We Live By, Chapters 1-4. (short - a total of 21 pages).
• [Lehnert, 1981] Wendy G. Lehnert. Plot units and narrative summarization. Cognitive Science, Volume 5, Issue 4, October-December 1981, Pages 293-331
• [Lehnert, 1977] Wendy Lehnert. Human and Computational Question Answering. Cognitive Science, Vol. 1, No. 1, pp. 47-73.
• [Mann and Thompson, 1988]. Rhetorical Structure Theory: Toward a functional theory of text organization. Text 8 (3), pp. 243-281, 1988.
• [Martin et al., 1986] P. Martin, D. Appelt and F. Pereira. Transportability and generality in a natural-language interface system.
• [McKeown 1986] Kathleen McKeown. Discourse strategies for generating natural-language text.
• [Rosch and Mervis, 1975] Eleanor Rosch and Carolyn B. Mervis. Family Resemblances: Studies in the Internal Structure of Categories, Cognitive Psychology, 7, 573-605.
• [Schank, 1986] Roger Schank. Language and memory.
• [Schubert and Pelletier, 1986] L Schubert and F J Pelletier. From English to logic: context-free computation of "conventional" logical translations.
• [Wilks, 1975] Yorick Wilks. An Intelligent Analyzer and Understander of English, CACM 1975.
• [Woods, 1986] W.A. Woods. Semantics and quantification in natural language question answering.

Independent Study: Concolic testing for web applications

(This is one of a series of posts about papers I'm reading for an independent study with Prof. Evan Chang at the University of Colorado, Boulder. The format is similar to that of a review of a paper submitted to a computer science conference. There are already-published papers, so I'll be writing with the benefit of hindsight, especially when the paper was published at least several years ago.)

This week, two papers about web applications from the International Symposium on Software Testing and Analysis '08:

• Dynamic Test Input Generation for Web Applications, Wasserman, et al
• Finding Bugs in Dynamic Web Applications, Artzi, et al

Dynamic Test Input Generation for Web Applications

Here the authors use the concolic testing method pioneered in the seminal paper on Directed Automated Randomized Testing (PDF) to generate tests automatically for web applications written in PHP.

Use a taint-based (clarify) PHP runtime environment as the test oracle (which determines whether a failure occurs). The purpose of automatically generating tests is to automatically identify bugs in the program and, narrowly, the type of bugs the authors are trying to identify are SQL injection vulnerabilities. To that end, they iteratively construct (what they call) an approximate backward slice of the PHP program by (loosely speaking):

1. identify statements where such vulnerabilities may cause undesired behavior (viz. database library calls where the injected SQL can ultimately do harm),
2. add the functions where such statements occur to a set of functions to be analyzed,
3. execute the program by loading it in a browser
4. resolve control dependencies by recording stack trace at the beginning
5. analyze data dependencies
6. repeating (with some variations) until all data dependencies are resolved

The purpose of the preceding steps is to exclude from the analysis aspects of the program that won't help identify SQL vulnerabilities in the code.

Section 3 of the paper discusses the authors' algorithm for generating constraints for PHP.

Section 4 evaluates the system. Constraint generation is accomplished by a plugin to phc, a PHP compiler front-end. The plugin "wraps each statement in a function call"; the function call logs a trace of the program's execution to a file. They deal with eval by passing the string to be eval-ed through the plugin, so each statement in the eval-ed string is wrapped in a function call which logs a trace to the same file. Constraints are resolved by reading and symbolically executing the trace file. The result is "a list of Boolean control expressions where each subexpression is annotated with a concrete value from the execution."

An interesting aspect of the authors' evaluation is the overhead of the tracing process. When they evaluated an entire PHP program, the trace file for loading a single web page was almost 3 GB and the page load timed out. So the iterative process of limiting the scope of their analysis mentioned above was necessary for obvious practical reasons.

Previous work on leveraging symbolic and runtime values for input test generation falls back on concrete values when the symbolic value being addressed lies outside the theory of the resolution algorithm’s decision procedure. Our constraint resolution algorithm generates constraints only based on one variable instance per value. Therefore it may underapproximate the symbolic values of variables when program predicates depend on multiple variables, and it may miss paths that other resolution algorithms would find. In principle our constraint resolution algorithm could be enhanced to include multivariate constraints in some cases, but we leave that to future work.

An object-oriented web test model for testing web applications, Kung, et al, may be interesting reading:

This paper describes an Object-Oriented test model that captures both structural and behavioral test artifacts of Web applications. The model represents the entities of Web applications as objects and describes their structures, relationships, and dynamic behaviors. Based on the test model, test methods are presented to derive test cases automatically for ensuring the structures and behaviors of Web applications

Finding Bugs in Dynamic Web Applications

The previous paper focused on web application security. This paper focuses on web application reliability. Where the previous paper's goal was to identify vulnerabilities to SQL-injection attacks, this paper's goal is to identify bugs that cause web applications to crash or generate invalid HTML. (Web application crashes that can be triggered by user input become denial-of-service attack vulnerabilities once they become known to bad actors.) Similarly, where the test oracle of the previous paper is a PHP runtime environment that supports checking strings for taintedness (failure being defined as the use of an untained string in an SQL statement), the test oracle of this paper is an HTML validator (failure being defined as the web application generating invalid HTML).

Testing whether a web application generates valid HTML is hard for dynamic web pages. Systems exist for validating dynamically-generated web pages, but the require the tester to create tests manually. Here the authors present a system, Apollo, for automatically generating tests for dynamic pages.

Something I don't understand. Here's a passage from the paper:

The HTML pages generated by a PHP applications may contain buttons that—when pressed by the user—result in the loading and execution of additional PHP source files. We simulate such user input by transforming the source code. Specifically, for each page h that contains N buttons, we add an additional input parameter p to the PHP program, whose values may range from 1 through N. Then, at the place where page p is generated, a switch statement is inserted that includes the appropriate PHP source file, depending on the value supplied for p. The steps of the user input simulator are fully mechanic, and the required modifications are minimal, but for the evaluation we performed the program transformation by hand (due to time constraints).

Normally, submit buttons result in an HTML form being POST'ed to the web application. From the context, it's not clear why the system wouldn't simply POST the form. An additional passage

The stand-alone component of the User Input Simulator performs a transformation of the program that models interactive user input by way of additional parameters.

Still a little confused. :-)

Ah, I get it:

  <?php echo "<h2>WebChess ".$Version." Login"</h2>;?>
  <form method="post" action="mainmenu.php">
      <p>
          Nick: <input name="txtNick" type="text" size="15"/><br/>
          Password: <input name="pwdPassword" type="password" size="15"/>
      </p>
      <p>
          <input name="login" value="login" type="submit"/>
          <input name="newAccount" value="New Account"
                type="button" onClick="window.open('newuser.php', '_self')"/>
      </p>
   </form>

Nothing else very interesting here. They evaluate their system, present the results, related work, etc.

Other reading:

Improving test case generation for web applications using automated interface discovery, Halfond, et al, focuses on JavaScript.