Saturday, 15 August 2009

Real Testing Tales (with Fiddler) – The Case of the Get that became aPOST

The following contains a true life summary of some recent testing to illustrate the use of Fiddler and some test thinking in action.
I know I have mentioned Fiddler 2 before and how I could not test web sites without it (OK, so I could but I’d use something like BurpSuite instead), but I like Fiddler because of its ease of use and flexibility, as I will explain…


I recently had cause to examine a Flash application to provide some feedback to the authors.
I normally start such investigations by using Fiddler to examine the communication between the Flash app and the server so as to understand the communication between the two applications. I tend to use Firefox as my browser for this, and put Fiddler into “Force traffic to Fiddler” otherwise Fiddler doesn’t seem to pick up all the communication and I have to double check some of the AJAX message transfers with HTTPFox).
So I use the application for a while with Fiddler running in the background to see what traffic goes back and forward and see what I find interesting.
I like to watch out for calls to different servers, as this often reveals areas of potential vulnerability, because when swapping between servers, sometimes session or authentication information gets passed in the request, and sometimes you have access to new functionality without authentication.
In this instance I saw calls to a different server so I took the URL from fiddler with the “copy > just URL” function and went directly to this in another browser window to see what I could see.
Since the URL led to a cgi app, I amended the URL to see what the page at the root of the server would tell me.
I did not expect to see a default page with link to the admin functionality, but such things can happen when you build on open-source frameworks – one reason why, as a tester, you need to know and learn about the technologies and frameworks that underpin the application
So I passed this information on to the developers so they could mitigate this risk.
… time passed…
Later I visited the URL to the admin page and found that a password prompt popped up to stop me. So I had a quick look at the application again to see how the traffic between the flash application and the server had changed to see how the application bypassed the security mechanism:
  • perhaps I would see something obvious like a password in the URL,
  • or perhaps the application used a new set of URLs now.
Nope, same URLs, but this time it POSTed information instead of GETting information.
So in Fiddler I constructed a POST to the admin URL and lo and behold I could see the admin pages again.
In order to assess the impact of this I really wanted a way to browse the admin areas using a normal browser in ‘POST mode’, to see if I could still use the site easily, so I had to find a way to surf the site with POSTs instead of GETs.
I first looked for a Firefox extension to convert all GETs into POSTs but I could not find one. (If anyone knows of one then I would love to learn about it.)
And then I realised that I could probably do that with a customised rule in Fiddler.
If you decide to customise the rules in Fiddler then you really must install the Fiddler2ScriptEditor, it provides a context sensitive, syntax highlighting and documented environment for amending the scripts. I didn’t use this (but I have learned my lesson now) so it took me longer to work out how to do what I wanted.
The Fiddler2ScriptEditor has a class explorer on the RHS with documentation about the various Fiddler API sections. I used to find this a little confusing because the top level item HTTPRequestHeaders actually maps onto “oSession.oRequest.headers.” and not “oSession.oRequest.”, but Eric Laurence (the Fiddler author) kindly responded to an email query of mine and helped me traverse the API.
So armed with the class explorer, the code completion and the Fiddler forums you probably have enough information to help you out .
I have not seen the ease or speed with which you can amend the underlying rule set in Fiddler matched by any other proxy tool.
So, should you ever want to surf the web using POSTs instead of GETs, you need to amend your “function OnBeforeRequest(oSession: Session)” rules to include lines such as:
if (oSession.HostnameIs("<hostname here>") && oSession.HTTPMethodIs("GET")) {  oSession.oRequest.headers.HTTPMethod = "POST";  }
I could then ‘surf’ the password protected areas of the site with ease by POSTing my requests instead of GETting them, and I could see all the results in a real browser, instead of just the Fiddler inspector windows.
So, queue another sending of information to the developer to mitigate the risk.
And the reasons for writing this post?
  • to identify some of the thought processes I use when Web Testing:
    • explore the site and build a model of the server side communication
    • watch for other servers in use in the transaction to identify possible risks for exploitation
    • amend the URLs used in the communication and explore other parts of the server
    • learn the underpinning frameworks
    • build theories of possible implementation approaches and then investigate them
    • find or make tools to help me explore risks further
  • to promote Fiddler 2 a little more
    • as it has the easiest customisation I have encountered with the Fiddler2ScriptEditor
    • so you can surf with POSTs instead of GETs (admit it, you always wanted to do that, right?)
  • so I can remember how to do this next time
I always like to read practical explanations of how people approach testing so feel free to leave comments with examples of your “Real Testing Tales” or links to your blog post examples.

Related Links:

7 comments:

  1. Neat trick. I've done similar stuff but this was particularly clever. (and dastardly, not to mention Evil, congratulations!) I'll definitely remember this if I come across it in the future.

    Thanks Chris, hope it helps in the future.

    ReplyDelete
  2. Evil Tester,

    Really nice post. I like it a lot. Always good to see how your evil mind works. Thanks for some cool ideas on how to access forbidden pages. I'll make use of that on my current project.

    Rob.

    Thanks Rob - just make sure you play nice out there :)

    ReplyDelete
  3. [...] Richardson рассказывает о том, как Fiddler может помочь в тестировании [...]

    ReplyDelete
  4. Thanks for this post, I really liked it. You converted me into trying out Fiddler against one of our applications - I like it!
    Keep posting these examples, it really helps!

    ReplyDelete
  5. Nice post, useful info.

    Interesting how the Flash author used POST as a workaround. Wonder if the POST requests contained any body content (data). Perhaps they should have one upped the security by making HTTP PUT requests instead. Not a standard HTTP request (more for HTTP REST web services), so less likely to be hacked and less people understand it. Didn't test, but wonder if HttpFox and other simple sniffers sniff out PUT requests as well. I'm sure Fiddler and Wireshark will.

    ReplyDelete
  6. On a related note, for certain applications, you could also do without Fiddler and just make use of browser traffic sniffers like HttpFox and another good one - livehttpheaders. For IE, there's iehttpheaders.

    Though these tools don't give you the body content, only the response headers and request content (for POSTs), they're useful for reconstructing requests to analyze and automate. I wrote an article a while back on how to automate (as a bot, not UI automation) web activity that you do within a Java applet (or like the Flash app in this article) that communicates with a web server.

    http://www.codeproject.com/Articles/15128/Web-HTTP-Automation-with-Perl

    ReplyDelete
  7. Alan Richardson17 March 2012 at 01:11

    Thanks for the link David.

    I prefer proxies, and drop down to in browser tools for emergency purposes.

    ReplyDelete