FFPuppet is a Python module that automates browser process related tasks to aid in fuzzing

Overview

FFPuppet

Task Status codecov Matrix PyPI

FFPuppet is a Python module that automates browser process related tasks to aid in fuzzing. Happy bug hunting!

Are you fuzzing the browser? Grizzly can help.

Installation

To install the latest version from PyPI
pip install ffpuppet
Xvfb on Linux

On Linux xvfb can be used in order to run headless (this is not the same as Firefox's -headless mode).

To install xvfb on Ubuntu run:

apt-get install xvfb
Installing minidump_stackwalk

minidump_stackwalk is used to extract a crash report when the browser crashes without a debugger (GDB/Valgrind) or instrumentation (ASan). If desired, minidump_stackwalk should be installed in the users path after obtaining it from FirefoxCI. Choose the appropriate tool for your platform (linux64-minidump-stackwalk, macosx64-minidump-stackwalk, win32-minidump-stackwalk, etc.), choose latest, then extract the executable from the downloaded tar archive.

Browser Builds

If you are looking for builds to use with FFPuppet there are a few options.

Downloading builds

fuzzfetch is the recommended method for obtaining builds and is also very helpful in automation.

Taskcluster has a collection of many different build types for multiple platforms and branches. An index of the latest mozilla-central builds can be found here.

Build your own

If you would like to compile your own, build instructions can be found here.

Usage

Once installed FFPuppet can be run using the following command:

python -m ffpuppet
usage: ffpuppet [-h] [-d] [--log-level LOG_LEVEL] [-e EXTENSION] [-p PREFS]
                [-P PROFILE] [-u URL] [--xvfb] [-a ABORT_TOKEN]
                [--launch-timeout LAUNCH_TIMEOUT] [-l LOGS]
                [--log-limit LOG_LIMIT] [-m MEMORY]
                [--poll-interval POLL_INTERVAL] [--save-all]
                [--gdb | --pernosco | --rr | --valgrind]
                binary

FFPuppet - Firefox process launcher and log collector. Happy bug hunting!

positional arguments:
  binary                Firefox binary to launch

optional arguments:
  -h, --help            show this help message and exit
  -d, --display-logs    Display summary of browser logs on process exit.
  --log-level LOG_LEVEL
                        Configure console logging. Options: DEBUG, INFO, WARN,
                        ERROR (default: INFO)

Browser Configuration:
  -e EXTENSION, --extension EXTENSION
                        Install extensions. Specify the path to the xpi or the
                        directory containing the unpacked extension.
  -p PREFS, --prefs PREFS
                        Custom prefs.js file to use (default: profile default)
  -P PROFILE, --profile PROFILE
                        Profile to use. This is non-destructive. A copy of the
                        target profile will be used. (default: temporary
                        profile)
  -u URL, --url URL     Server URL or path to local file to load.
  --xvfb                Use Xvfb. (Linux only)

Issue Detection & Reporting:
  -a ABORT_TOKEN, --abort-token ABORT_TOKEN
                        Scan the browser logs for the given value and close
                        browser if detected. For example '-a ###!!!
                        ASSERTION:' would be used to detect soft assertions.
  --launch-timeout LAUNCH_TIMEOUT
                        Number of seconds to wait for the browser to become
                        responsive after launching. (default: 300)
  -l LOGS, --logs LOGS  Location to save browser logs. A sub-directory
                        containing the browser logs will be created.
  --log-limit LOG_LIMIT
                        Browser log file size limit in MBs (default: 0, no
                        limit)
  -m MEMORY, --memory MEMORY
                        Browser memory limit in MBs (default: 0, no limit)
  --poll-interval POLL_INTERVAL
                        Delay between checks for results (default: 0.5)
  --save-all            Always save logs. By default logs are saved only when
                        an issue is detected.

Available Debuggers:
  --gdb                 Use GDB. (Linux only)
  --pernosco            Use rr. Trace intended to be submitted to Pernosco. (Linux only)
  --rr                  Use rr. (Linux only)
  --valgrind            Use Valgrind. (Linux only)

Replaying a test case
python -m ffpuppet 
   
     -p 
    
      -d -u 
     

     
    
   

This will open the provided test case file in Firefox using the provided prefs.js file and any log data (stderr, stdout, ASan logs... etc) will be dumped to the console when the browser process terminates. Grizzly Replay is recommended for replaying test cases.

Prefs.js files

prefs.js files that can be used for fuzzing or other automated testing can be generated with PrefPicker.

Comments
  • ffpuppet does not work with older version of Firefox

    ffpuppet does not work with older version of Firefox

    I have build Firefox 57.0 and Firefox 63.0.3 (with ASAN) by myself. Running python -m ffpuppet ../../firefox-57.0/objdir-ff-asan/dist/bin/firefox -p $CODE/browsers/prefs.js -d -u /home/ug16zy2/test.html -v --xvfb --log ./out/ gives:

    I ffpuppet [2020-01-01 19:55:44] Launching Firefox... D ffpuppet [2020-01-01 19:55:44] requested location: '/home/ug16zy2/test.html' D ffpuppet [2020-01-01 19:55:44] launch timeout: 300 D ffpuppet [2020-01-01 19:55:44] profile directory: '/tmp/ffprof_VspYuS' D ffpuppet [2020-01-01 19:55:44] using prefs.js: '/home/ug16zy2/fuzz/browsers/prefs.js' D ffpuppet [2020-01-01 19:55:44] launch command: '/home/ug16zy2/firefox-57.0/objdir-ff-asan/dist/bin/firefox -no-remote -profile /tmp/ffprof_VspYuS http://127.0.0.1:46027' D ffpuppet [2020-01-01 19:55:44] launched firefox with pid: 21507 D ffpuppet [2020-01-01 19:55:48] crash report found I ffpuppet [2020-01-01 19:55:48] Shutting down... D ffpuppet [2020-01-01 19:55:48] close(force_close=False) called D ffpuppet [2020-01-01 19:55:48] browser pid: 21507 D ffpuppet [2020-01-01 19:55:48] 1 crash report(s) are available D ffpuppet [2020-01-01 19:55:53] reviewing 0 check(s) D ffpuppet [2020-01-01 19:55:53] scan_path '/tmp/ffprof_VspYuS/minidumps' does not exist D ffpuppet [2020-01-01 19:55:53] exit reason code 'ALERT' I ffpuppet [2020-01-01 19:55:53] Firefox process is closed. (Reason: 'ALERT') I ffpuppet [2020-01-01 19:55:53] Saving logs to '/home/ug16zy2/fuzz/ffpuppet/out' D ffpuppet [2020-01-01 19:55:53] save_logs() called, dest='./out/', logs_only=False, meta=False I ffpuppet [2020-01-01 19:55:53] Displaying logs...

    and the ASAN log is:

    === Dumping 'log_ffp_asan_21493.log.21507.txt' (1.83KB) ==21507==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x7f9c17e13646 bp 0x7fffd3f6b340 sp 0x7fffd3f6aab8 T0) ==21507==The signal is caused by a READ memory access. ==21507==Hint: address points to the zero page. #0 0x7f9c17e13645 (/lib/x86_64-linux-gnu/libc.so.6+0xb1645) #1 0x7f9c18ef7146 (/usr/lib/x86_64-linux-gnu/libasan.so.4+0x42146) #2 0x7f9c11652660 (/usr/lib/x86_64-linux-gnu/libglib-2.0.so.0+0x39660) #3 0x7f9c11652806 in g_dgettext (/usr/lib/x86_64-linux-gnu/libglib-2.0.so.0+0x39806) #4 0x7f9c144f13ae in gtk_get_option_group (/usr/lib/x86_64-linux-gnu/libgtk-3.so.0+0x22f3ae) #5 0x7f9c144f15fb in gtk_parse_args (/usr/lib/x86_64-linux-gnu/libgtk-3.so.0+0x22f5fb) #6 0x7f9bfb69192f in XREMain::XRE_mainStartup(bool*) /home/ug16zy2/firefox-57.0/toolkit/xre/nsAppRunner.cpp:3911 #7 0x7f9bfb698931 in XREMain::XRE_main(int, char**, mozilla::BootstrapConfig const&) /home/ug16zy2/firefox-57.0/toolkit/xre/nsAppRunner.cpp:4852 #8 0x7f9bfb699030 in XRE_main(int, char**, mozilla::BootstrapConfig const&) /home/ug16zy2/firefox-57.0/toolkit/xre/nsAppRunner.cpp:4962 #9 0x7f9bfb6c3d6d in mozilla::BootstrapImpl::XRE_main(int, char**, mozilla::BootstrapConfig const&) /home/ug16zy2/firefox-57.0/toolkit/xre/Bootstrap.cpp:45 #10 0x55fa5cd415df in do_main /home/ug16zy2/firefox-57.0/browser/app/nsBrowserApp.cpp:236 #11 0x55fa5cd41c87 in main /home/ug16zy2/firefox-57.0/browser/app/nsBrowserApp.cpp:309 #12 0x7f9c17d83b96 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21b96) #13 0x55fa5cd408d9 in _start (/home/ug16zy2/firefox-57.0/objdir-ff-asan/dist/bin/firefox+0x3e8d9) AddressSanitizer can not provide additional info. SUMMARY: AddressSanitizer: SEGV (/lib/x86_64-linux-gnu/libc.so.6+0xb1645) ==21507==ABORTING

    When I run FFPuppet with /usr/bin/firefox, it works fine and Firefox instance pops up with the input HTML file.

    Does FFPuppet support older version of Firefox?

    opened by ZihanYe 6
  • Added the ability to specify a timeout in order to terminate during hangups.

    Added the ability to specify a timeout in order to terminate during hangups.

    This adds a timeout parameter (-w) by implementing a check called CheckRunningTimeout. The timeout lets someone specify a number of seconds before terminating the browser. Pretty basic really.

    enhancement wontfix 
    opened by arizvisa 6
  • Finish typing support.

    Finish typing support.

    This finishes the work by @nth10sd in #115 . Much of the nicer syntax depends on __future__ annotations which has other caveats, so I've backported this to 3.6 for now.

    This also enables mypy in pre-commit, enables Python 3.10 CI, and enables lint across all Python versions in CI.

    Supersedes #115.

    opened by jschwartzentruber 4
  • Run minidump_stackwalk on any minidumps in the profile directory …

    Run minidump_stackwalk on any minidumps in the profile directory …

    …and append to the log.

    • only works if minidump_stackwalk is in path, otherwise a warning is printed if a minidump is found
    • remove obsolete and incomplete breakpad symbolization
    • symbolize argument to save_log() and clone_log() has been removed
    opened by jschwartzentruber 4
  • Add mypy type information and bump minimum Python version to 3.7

    Add mypy type information and bump minimum Python version to 3.7

    This PR adds mypy type information for ffpuppet, which helps the PR for FuzzManager type info as well. I also deleted type information if present in the comments, since some were already inaccurate IIRC.

    It bumps the minimum Python version to 3.7 since we import annotations from __future__.

    GitHub Actions CI is here.

    To-do:

    • [x] Push a py.typed file on completion
    opened by nth10sd 3
  • Fix #18: --url localfile doesn't work on Windows.

    Fix #18: --url localfile doesn't work on Windows.

    After this change yields:

    windows:
    D ffpuppet [2017-09-07 14:55:24] sending response with redirect url: 'file:///C:/Users/user/Desktop/tc.html'
    linux:
    D ffpuppet [2017-09-07 15:01:45] sending response with redirect url: 'file:/home/user/Desktop/tc.html'
    

    Both redirects work properly.

    opened by jschwartzentruber 3
  • Add support for arbitrary/multiple extensions.

    Add support for arbitrary/multiple extensions.

    Extensions are not going away, and we should support installing them for testing even if we stop using them regularly (which is not planned). This supports arbitrary add-ons by looking up the extension ID in the manifest (may be fragile for legacy add-ons because of XML namespaces).

    Fixes #4.

    enhancement 
    opened by jschwartzentruber 2
  • Only set max_allocation_size_mb and soft_rss_limit_mb for ASan and TSan

    Only set max_allocation_size_mb and soft_rss_limit_mb for ASan and TSan

    Setting these values in UBSAN_OPTIONS as well is a foot gun. It can can lead to confusion if set for multiple sanitizers. It can still be set manually in the environment.

    opened by tysmith 1
  • Reduce the shutdown wait after a crash

    Reduce the shutdown wait after a crash

    This will reduce the wait period for the browser processes to close after a crash log is detected.

    The browser should shutdown quickly after a crash report is written.

    opened by tysmith 1
  • Consider randomizing the resolution passed to xvfb

    Consider randomizing the resolution passed to xvfb

    It might be worth considering setting a random resolution to xvfb as this might affect layout. Similar concept to the TTF Fuzzer outlined here:

    https://media.blackhat.com/bh-eu-12/Lee/bh-eu-12-Lee-GDI_Font_Fuzzing-WP.pdf

    opened by pyoor 1
Releases(0.9.2)
Owner
Mozilla Fuzzing Security
Fuzzing projects at the Mozilla Corporation
Mozilla Fuzzing Security
Automated testing tool developed in python for Advanced mathematical operations.

Advanced-Maths-Operations-Validations Automated testing tool developed in python for Advanced mathematical operations. Requirements Python 3.5 or late

Nikhil Repale 1 Nov 16, 2021
A configurable set of panels that display various debug information about the current request/response.

Django Debug Toolbar The Django Debug Toolbar is a configurable set of panels that display various debug information about the current request/respons

Jazzband 7.3k Jan 02, 2023
Sixpack is a language-agnostic a/b-testing framework

Sixpack Sixpack is a framework to enable A/B testing across multiple programming languages. It does this by exposing a simple API for client libraries

1.7k Dec 24, 2022
Getting the most out of your hobby servo

ServoProject by Adam Bäckström Getting the most out of your hobby servo Theory The control system of a regular hobby servo looks something like this:

209 Dec 20, 2022
Python drivers for YeeNet firmware

yeenet-router-driver-python Python drivers for YeeNet firmware This repo is under heavy development. Many or all of these scripts are not likely to wo

Jason Paximadas 1 Dec 26, 2021
A grab-bag of nifty pytest plugins

A goody-bag of nifty plugins for pytest OS Build Coverage Plugin Description Supported OS pytest-server-fixtures Extensible server-running framework w

Man Group 492 Jan 03, 2023
A single module to link Python ecosystem to the Web

A single module to link Python ecosystem to the Web. Have a quick look at the Gallery first to get convinced ! FAQ For any questions, please use Stack

66 Dec 21, 2022
WomboAI Art Generator

WomboAI Art Generator Automate AI art generation using wombot.art. Also integrated into SnailBot for you to try out. Setup Install Python Go to the py

nbee 7 Dec 03, 2022
Active Directory Penetration Testing methods with simulations

AD penetration Testing Project By Ruben Enkaoua - GL4Di4T0R Based on the TCM PEH course (Heath Adams) Index 1 - Setting Up the Lab Intallation of a Wi

GL4DI4T0R 3 Aug 12, 2021
Pytest-rich - Pytest + rich integration (proof of concept)

pytest-rich Leverage rich for richer test session output. This plugin is not pub

Bruno Oliveira 170 Dec 02, 2022
Automated Security Testing For REST API's

Astra REST API penetration testing is complex due to continuous changes in existing APIs and newly added APIs. Astra can be used by security engineers

Flipkart Incubator 2.1k Dec 31, 2022
Generic automation framework for acceptance testing and RPA

Robot Framework Introduction Installation Example Usage Documentation Support and contact Contributing License Introduction Robot Framework is a gener

Robot Framework 7.7k Jan 07, 2023
User-oriented Web UI browser tests in Python

Selene - User-oriented Web UI browser tests in Python (Selenide port) Main features: User-oriented API for Selenium Webdriver (code like speak common

Iakiv Kramarenko 575 Jan 02, 2023
pytest plugin for testing mypy types, stubs, and plugins

pytest plugin for testing mypy types, stubs, and plugins Installation This package is available on PyPI pip install pytest-mypy-plugins and conda-forg

TypedDjango 74 Dec 31, 2022
pywinauto is a set of python modules to automate the Microsoft Windows GUI

pywinauto is a set of python modules to automate the Microsoft Windows GUI. At its simplest it allows you to send mouse and keyboard actions to windows dialogs and controls, but it has support for mo

3.8k Jan 06, 2023
Given some test cases, this program automatically queries the oracle and tests your Cshanty compiler!

The Diviner A complement to The Oracle for compilers class. Given some test cases, this program automatically queries the oracle and tests your compil

Grant Holmes 2 Jan 29, 2022
Pytest modified env

Pytest plugin to fail a test if it leaves modified os.environ afterwards.

wemake.services 7 Sep 11, 2022
The (Python-based) mining software required for the Game Boy mining project.

ntgbtminer - Game Boy edition This is a version of ntgbtminer that works with the Game Boy bitcoin miner. ntgbtminer ntgbtminer is a no thrills getblo

Ghidra Ninja 31 Nov 04, 2022
MongoDB panel for the Flask Debug Toolbar

Flask Debug Toolbar MongoDB Panel Info: An extension panel for Rob Hudson's Django Debug Toolbar that adds MongoDB debugging information Author: Harry

Cenk Altı 4 Dec 11, 2019
Implement unittest, removing all global variable and returning values

Implement unittest, removing all global variable and returning values

Placide 1 Nov 01, 2021