“Operating system facilities, such as the kernel and utility programs, are typically assumed to be reliable. In our recent experiments, we have been able to crash 25-33% of the utility programs on any version of UNIX that was tested.” [1]
Those were the original words in one of the first fuzzing studies where Prof. Barton Miller was first to use the term ‘fuzzing’
One can see the importance of fuzzing as one of the techniques used to test software security against malformed input leading to crashes and in some cases exploitable bugs.
Prof. Miller, in the above quote, was using a random generator to generate malformed input. Another approach that was considered at the time was syntax testing by Dr. Boris Beizer [2]. Beizer was able to generate semi-valid test cases based on the syntax “Grammar” of expected input.
The general fuzzing methodology:
Fuzzing in itself, no matter what type, follows a relatively simple methodology:
Identify Target => Identify Inputs => Generate Data => Fuzz Target => Monitor for Memory Corruption Errors
To be able to perform this process at the highest efficiency there are some tools that can help us.
Memory Error Detectors:
Memory error detectors are tools used to rapidly identify memory corruption errors by poisoning memory around allocations and quarantining and poisoning memory areas after they are free-ed and report any attempt to access these memory regions by the target application. and usually they will provide a deeper level of detail in regards to a specific error.
For windows there is PageHeap which is part of the Windows Debugging toolkit and can be enabled on certain processes using “Gflags.exe” [3]
For Linux and Max OSX there is Google’s AddressSanitizer (ASan) which is a clang compiler plug-in that can be enabled on applications at complie time using the “-fsanitize=address” parameter [4].
Fuzzer Harnesses:
A fuzzer harness is a tool the loads the target application, feeds it generated test cases and monitors the target. If a crash occurs, it will save the test case that caused the crash and subsequent crash log containing the details. As is often the case, it will restart the application after a certain period of time or when a crash occurs.
For Windows there is Grinder by Stephen Fewer [5].
For Linux and Mac OSX there is NodeFuzz By Atte Kettunen from OUSPG [6].
Introducing Wadi.
At Defcon 23, Etienne and I introduced a new fuzzer, which we’ve called Wadi.
What is Wadi?
Wadi is web browser grammar-based fuzzer. Grammars are used to describe how browsers should process web content, Wadi turns that around and uses grammars to break browsers.
Wadi is a Fuzzing module to use with NodeFuzz fuzzing Harness and utilizes AddressSanitizer(ASan) for instrumentation on Linux and Mac OSX.
The World Wide Web Consortium (W3C) is an international community that develops open standards to ensure the long-term growth of the Web [7]. It is the W3C that allows us to search for grammar to use in our test cases.
The standards that are relevant to this research are:
– The DOM standards [8].
– The Web API standards [9].
– The Web IDL standards [10].
The Document Object Module (DOM):
The Document Object Model provides a standard set of objects for representing HTML and XML documents, a standard model of how these objects can be combined, and a standard interface for accessing and manipulating them.
The Web API:
When developing for the web, using JavaScript, there are many APIs available, for example speech, webaudio, gamepad, canvas, webgl, animation etc.
The Web IDL:
Web IDL is an interface definition language (IDL) used to describe interfaces that are intended to be implemented in web browsers. The IDL is defined by an LL(1) grammar that is also provided by W3C [11].
This enables a systematic way of parsing the IDL and generating the fuzzer grammar based on it.
Grammar:
In human languages, grammar is used to explain how sentences are constructed. In compilers, grammar is used to explain the programming language syntax and code construction. In fuzzing grammar is used to explain to the fuzzer how test cases are constructed.
Or as saying goes “Grammar is the difference between knowing your shit and knowing you’re shit”.
IDL interface:
According to IDL definitions, an interface is an object with a set of interface members. These can be constants, attributes, or functions. Each interface has a unique identifier and inherits from a parent interface if needed.
A modified example interface is taken straight from the DOM Core IDL
From the example above we can see that the initial information gathered from the interface are:
Identifier: Text Inheritance: CharacterData
IDL Interface Members:
IDL interface members can be a constant, attribute, or function.
LL(1) Grammar Interface Members definition
From the example above
Attributes: isElementContentWhitespace, wholeText Operations: splitText(), replaceWholeText()
IDL Attributes:
An attribute is a declared interface member with an identifier whose value can be retrieved and in some cases changed.
LL(1) Grammar Attribute definition
Attributes of Example Interface:
IDL Functions:
Functions in the IDL specification is referred to as operations. A function is an interface member that defines behaviour that can be invoked on objects implementing the interface.
LL(1) Grammar Operation definition
Functions of Example Interface:
Fuzzing Grammar
Using the gathered information: Identifier, Inheritance, Interface Members, the IDL interface can be mapped into a JavaScript object to be used by Wadi.
For attributes and methods, an array of arrays is created with each child array representing a single attribute or method
A single attribute array will have three members:
A single method array will have three members:
The main attributes and methods arrays are concatenated to the parents interface object attributes and methods arrays to simulate inheritance.
Example Interface as defined in the Fuzzing grammar:
Now that we’ve created, and cleared, the fuzzing grammar, it’s time to see how Wadi works.
Using Wadi
Wadi works as a module for NodeFuzz on Linux and Mac OS X and is used to Fuzz Chrome and Mozilla FireFox ASan builds.
Requirements
– Memory Error Detector: AddressSanitizer (ASan)
– Fuzzing Harness: NodeFuzz By Atte Kettunen of OUSPG
Memory Error Detector: AddressSanitizer (ASan)
AddressSanitizer is a fast memory error detector. It consists of a compiler instrumentation module and a run-time library. The tool can detect the following types of bugs:
– Out-of-bounds accesses to heap, stack and globals
– Use-after-free
– Use-after-return (to some extent)
– Double-free, invalid free
– Memory leaks (experimental)
Typical slowdown introduced by AddressSanitizer is 2x. There are pre-build ASan binaries for both Chrome and FireFox. For building Chrome from source read the following:
– http://dev.chromium.org/developers/how-tos/get-the-code
– http://www.chromium.org/developers/testing/addresssanitizer
For Building and running FireFox read the following:
– https://developer.mozilla.org/en-US/docs/Mozilla/Developer_guide/Build_Instructions
– https://developer.mozilla.org/en-US/docs/Mozilla/Testing/Firefox_and_Address_Sanitizer
Fuzzing Harness: NodeFuzz By Atte Kettunen of OUSPG
NodeFuzz is a fuzzer harness for web browsers and browser-like applications. There are two main ideas behind NodeFuzz: First is to create a simple, and fast, way to fuzz different browsers. Second to have one harness that can be easily expanded with new test case generators and client instrumentations, without modifications for the core.
The mechanics behind NodeFuzz:
– Launch the target web browser.
– Generate the test case from the supplied module.
– Create an object URL for the test case.
– Feed the test case to the browser using web sockets.
– Monitor the web browser process for crashes.
– If a crash occurs, pass the resulting error output to the instrumentation module (currently only supports Linux and Mac OSX ASan instrumentation).
– If the crash was found to be useful, save the last ten test cases served to the web browser before the crash as well as a detailed ASan crash analysis log.
– Restart the web browser process based on time-outs and numbers of test cases served.
For further information on how to install and run NodeFuzz, visit https://github.com/attekett/NodeFuzz
Once NodeFuzz is configured, some modifications must be made to the “config.js” file found in the root folder. NodeFuzz, available at the GitHub Repo, has an example launcher for Chrome, but this is for demo purposes only. As a result, we need to add a launcher for Chrome and FireFox as shown in the images below. You can obtain the modified config file from the SensePost Wadi Repo.
Remember that the “config.asan_symbolize” parameter needs to be changed to point to the path of the local “asan_symbolize.py” file as well as the paths for both the Chrome and FireFox binaries.
Understanding Wadi
Wadi works on created Fuzzer Grammar to generate valid JavaScript statements into an array. It then uses these JavaScript statements to create a valid HTML documents.
Test Case Generation – Spaces
Wadi saves references to the created objects and their respective interfaces in two spaces;
- The Browser Space: accessing DOM elements from the browser using JavaScript.
- The fuzzer space: accessing Interfaces objects members through the fuzzer.
Test Case Generation – Element Creation
The first step is to create DOM elements with the below functions:
| Name | Description |
|---|---|
| createElement() | creates a random element from the list of interfaces and saves a reference to the object both in fuzzer space and browser space |
| createTextNode() | creates random length text nodes and attach them to random elements in the DOM. |
| mangleElements() | Mangle’s DOM element tree |
createElement() output:
Test Case Generation – Fuzzing Interface Function
Wadi calls the fuzz(num) function, supplied with the number of rounds it will execute. The fuzz() function will randomly call one of the functions from this list.
| Function | Description |
|---|---|
| fuzzWindowAttribs() | randomly set the ‘window’ interface object attributes. |
| fuzzWindowMethods() | randomly call the ‘window’ interface object methods. |
| fuzzStyle() | Pick a random element and set a random style property using element.style. |
| fuzzStyle1() | pick a random style sheet with random reference to element and set random style properties using insertRule. |
| fuzzDocumentAttribs() | randomly set the ‘document’ interface object attributes. |
| fuzzDocumentMethods() | randomly call the ‘window’ interface object methods. |
| deleteRandomKey() | deletes a random reference to a created object saved in the ‘createdElements’ object in browser space. |
| fuzzPlayerMethods() | if no animation player found call the createPlayer() function to create a new animation player and add reference to it in the createElements object array. if a player exist call a random method from the animation interface object. |
| fuzzPlayerAttribs() | if no animation player found call the createPlayer() function to create a new animation player and add reference to it in the createElements object array. if a player exist set a random attribute from the animation interface object. |
| MutationObserve() | creates a random mutation observer and add reference to it in the createElements object array. |
| fuzzMutationObserve() | if no mutation observer have been created, call () function if one exists call or set random method or attribute from the Mutation observer interface object. |
| createRangeTraversal() | creates a random TreeWalker or NodeIterator and add reference to them in the createElements object array. |
| fuzzRangeTraversal() | if no range has been created call createRangeTraversal() function. else call or set random method or attribute from the respective NodeIterator Interface or TreeWalker Interface objects. |
| addLoop() | add a random loop function around js block loops are (for, while, setTimeout, setInterval) |
| crossRef() | try to set object references to a random other ex: HTML0 = HTML1.firstChild |
| AddEvent() | attach a random event to one of the created elements. creates an event object using createEvent directive and add reference to the created event object to a list for later use. |
| dispatchEvt | randomly fires one of the created events. |
| intfuzz() | return random function name for use as callbacks for certain operations |
| fuzzElementAttribs() | randomly set the attributes of a randomly selected element from the list of created elements. |
| fuzzElementMethods() | randomly call the methods of a randomly selected element from the list of created elements. |
| GarColl() | force garbage collection. |
Test Case Generation – Helper Functions
Helper functions are used to generate attributes values and method parameters and all through test case generation. Some of these functions can be found in “randoms.js? available in NodeFuzz.
Additional functionality was added to the original file as well as some modifications to the already available functions . The modified “randoms.js” with the additions required for Wadi to work is available on SenePost Github repo.
returnRandomElement()returns a random element from created elements and sometimes will append attributes referencing a near element like ‘firstChild’,’nextSibling’ etc.
| Name | Description |
|---|---|
| randoms() | return random number, float value or hex number |
| rint(num) | return a random number in range of the supplied num. |
| ra(array) | return a random element from an array. |
| arrayWalk(array) | return a random element from array if it is a function execute it and return the return value. if it is a string or an int return the value. |
| string(num | return a random length string with length based on a random number in range of the input num. |
| randbool() | returns random boolean. |
| getRandomColor() | returns random color in hex, rgb or hsl format. |
| distanceValue(num) | returns a random value in range or num with suffix px,cm,%,etc… |
| retURI(num) | returns a URL with random length based on the supplied num. |
Test Case generation – preparing the output script
- Creating Internal callbacks based on the number of function names returned by intfuzz()
- Insert the element creation JavaScript block.
- Insert all other object create JavaScript statements.
- Randomly insert JavaScript statements returned by fuzz() function.
Sample Wadi Output
Running your first fuzz – A simple checklist:
Wadi and NodeFuzz
- Download Wadi Fuzzer files: Wadi
- After installing NodeFuzz and downloading the ASan version of FireFox or Chrome, place the “WADI-Module.js” and “randoms.js” files in the modules directory.
- Replace the “config.js” in the root NodeFuzz Directory with the modified one.
- And run using:
#node nodefuzz.js -m ./modules/Git/WADI-Module.js -c ./config.js chrome
We’ve had some good results using Wadi, including four confirmed bugs in the latest Chromium and not to mention others.
If you use this and find any cool bugs, like we have, we’d love to hear from you. Saif@ and Etienne@ Sensepost.com
References:
[1] An Empirical Study of the Reliability of UNIX Utilities By Prof. Barton P. Miller.
[2] Black Box Testing, Software Testing Techniques, and Software System Testing and Quality Assurance By Dr. Boris Beizer.
[3] Gflags PageHeap.
[4] Google AddressSanitizer.
[5] Grinder By Stephen Fewer.
[6] NodeFuzz by Atte Kettunen of OUSPG.
[7] W3C Organization
[8] W3C Document Object Model Technical Reports.
[9] W3C Web API.
[10] W3C Web IDL.
[11] IDL Grammar
Further Reading:
[1] Fuzzing Brute Force Vulnerability Discovery: Michael Sutton, Adam Greene, Michael Pedram Amini.
[2] Fuzzing for Software Security Testing and Quality Assurance: Ari Takanen, Jared D. Demott, Charlie Miller.
[3] Browser bug hunting – Memoirs of a last man standing: Atte Kettunen 44con talk.