There are two ways to automate web testing. The goal is to test the functionality of a web application. One way is to write automation that drives a browser. The second is to use a library that imitates a browser session and communicate directly with the server.
The other method is to cut the browser out of the equation. Tools such as HTTPUnit, HTMLUnit, Canoo Web Test, WebDriver, and TestMaker use this method. A similar method is used by other tools including JMeter, LibWWW, and curl. The obvious disadvantage to using this method is browser compatibility issues.
Tools that use macros to drive a browser would fall into the first category. Tools that have an “expect” like dialog would fall into the latter.
I usually advocate the first method, because nothing beats having a real browser excercise your application.
The often overlooked advantage of the second method is that you can more easily run browserless tests as integration and smoke tests. Because it doesn’t have the overhead of the full browser, it is lightweight, and client independent. It runs faster and can be more easily run concurrently (which makes it useful for performance testing.) Browser timing issues (and crashes) are less of a problem.
I’d actually recommend type 2 tools for testing links, page flow, content (text/images), and principal functionality. But if user interface testing is needed, I’d use type 1 tools. However, the truth is that UI and ajax timing related issues are much easier to find with manual testing. I’d guess 3/4 of what automation buys you can be done with a browserless tool.
The advantage that browser-driven testing buys you is with recording tools. However, the penalty is with stability and speed of your tests.