SECURITY TESTING WITH WATIJ

Watij is a tool designed for functional web security testing. It’s effectively a Java API which drives an instance of Internet Explorer. You can then use your favourite unit testing framework to structure tests and make assertions of the results. Like similar functional testing tools, watij can be used to script security defects in web applications. This is inline with good security testing service providers and development practice of writing abuse cases and specifically testing for failure conditions rather than only testing for success conditions.I wrote a paper (pdf) on using functional testing tools in this way, some time ago, and since then I’ve been having a look at the various tools to find one most suitable for security testing services. Today, I whipped up some simple tests of the owasp.org website using Watij. It was easy to install and be up and running in Eclipse in a few minutes. Before starting on the tests, I wanted to define common functions, such as login and logout:

public static void login(String username, String password) throws Exception {
ie.navigate("https://www.owasp.org/index.php?title=Special:Userlogin&returnto=Main_Page");
ie.textField(name,"wpName").set(username);
ie.textField(name,"wpPassword").set(password);
ie.button(name, "wpLoginattempt").click();
}

public static void logout() throws Exception {
ie.navigate("https://www.owasp.org/index.php?title=Special:Userlogout&returnto=Special:Userlogin");
}

As you can see the syntax and API is quite intuitive. Now that these are defined, time to write the startup code for the tests – using JUnit 4 this is run only once for the whole test case:

//Run only once to initialise the browser
@BeforeClass
public static void init() throws Exception {
ie = new IE();
ie.start("https://www.owasp.org/index.php/Main_Page");
if (ie.containsText("There is a problem with this website's security certificate")) {
ie.link(name,"overridelink").click();
}
if (!ie.url().equals("https://www.owasp.org/index.php/Main_Page")) throw new RuntimeException("Error getting to the front page!");
webapp = new WebApp(ie);
}

And finally, we can write the tests which should be self explanatory:


@Test
public void testLoginLogout() throws Exception {
webapp.login(username, password);
assertTrue("Logged in.", ie.containsText("You are now logged in"));
webapp.logout();
assertTrue("Logged out.", ie.containsText("You are now logged out"));
}

@Test
public void testUserEnumeration() throws Exception {
webapp.login("notauser","test");
if (ie.containsText("There is no user by the name")) {
fail("Usernames can be enumerated through the login error messages");
}
}

@Test
public void testBruteForce() throws Exception {
int count=5;
for (int i=0;i webapp.login(username,"wrongpassword");
}
webapp.login(username,password);
if (ie.containsText("You are now logged in")) {
fail("After "+count+" incorrect login attempts, the user could still login. This exposes the accounts to brute force attacks.");
webapp.logout();
}
}

Here's a video of the result:

Quite slick. But say I'd like to test whether a user who's not logged in can post articles to the wiki. Well the "Edit" page is only displayed when you are logged in, so there's no way I can click on a non-existent link. And just because the link isn't there doesn't necessarily mean that the server won't accept the request. So what we need to do is perform the POST request to submit an article to the wiki directly and this is where watij stumbles. There's no easy way to create an arbitrary POST request. The recommended approach is to use something like HTTPUnit or HtmlUnit to perform this specific test. Ok, no problem, so I can just use the login function I've already defined in the watij tests, then read the session cookie and pass the cookie onto an HTTPunit test, right? Wrong. You can't read IE cookies from the watij API! So the only option is to re-code the login function in HttpUnit and use that instead. Lots of code duplication and now I'm mixing two different testing frameworks, not ideal.

To summarise the issues with using watij to test security:

• You can't make raw POST requests without there first being a form to post. This is problematic because a lot of security tests are really aimed at testing the server side, not the client. You could augment watir with some Java code, or HttpUnit or HtmlUnit to perform the post request...
• It doesn't appear to be possible to read IE's cookies from watij! This is really annoying, because now I can't reuse the login and logout function and simply pass the session cookie over to httpunit. I'd have to re-write the login function using httpunit - a whole lot of extra and messy work.
• Since it drives a real instance of IE, you have to trick it into ignoring client side validation or client side access control tests. (Of course client side security is no security at all)
• Tests can be quite slow, as watij waits for the entire page to load into the browser before continuing (turning off images and using ad blockers can help).

On the plus side, the JavaScript support is as good as it's gonna get so you'd very very rarely run into a web application which doesn't support IE.

On the whole, I can see the attraction for using watij to positively test functionality - but because of the reasons above, I don't think it's the best choice for performing negative testing (i.e. test what's not there). I hope to write a similar entry about Canoo's WebTest, which provides similar testing functionality, but using XML tests and it's own HTTP and Html engines.