Jungletest is an system for automated testing of OpenOffice.org (OOo) against a large set of documents in complex, binary formats. Currently, tests determine whether OOo will crash or hang when loading or scrolling through the document. The documents can be automatically harvested from the OOo Issue Tracker or through random URLs found on Yahoo.

Jungletest is alpha-quality software.

Practical applications

Practical applications of Jungletest include:

  • Quantifying the stability of OOo. For example, perhaps you could say that OOo can import 99.9999% of Microsoft Word documents without crashing.
  • Detecting regressions of fixed issues in the Issue Tracker.
  • Generally improving the stability of OOo.

Prerequisites

You must have the following software installed:

  • “System Python” (Python installed outside OOo)
  • “OpenOffice.org Python” (Python typically included with OOo)
  • file, a tool for identifying file types (common on UNIX systems)
  • wget (common on UNIX systems)
  • A list of words (usually /usr/share/dict/words)
  • OpenOffice.org (of course)

Jungletest should work on Windows, but only Linux has been tested so far.

Changes

What's new in Jungletest? Read the change log.

Downloads

The jungletest application:

Below is a list of 110,000 URLs to feed into jungletest, so jungletest can download the URLs for testing. The list contains doc, ppt, xls, rtf, pps, dot, docx, pptx, and xlsx.

Installation

To install Jungletest, follow these instructions.

  1. Download Jungletest
  2. tar xjvf jungletest-2005000.tar.bz2
  3. cd jungletest-20050000
  4. Edit config.py

Using Jungletest

Jungletest operation is split into three, discrete operations:

  1. Gathering URLs of documents
  2. Downloading documents
  3. Conducting the tests

Gathering document URLs

Use any combination of the following methods to populate the database with document URLs.

Random document URLs from Yahoo

If you want to harvest document URLs from Yahoo, run:

python jungletest.py --yahoo

NOTE: This process currently runs until you receive your daily limit of Yahoo queries, but when you have enough URLs, killing the process is safe.

Document URLs from OpenOffice.org Issue Tracker

Document URLs from a text file

If you want to import a list of URLs from a text file (such as the large file available from the downloads section), run:

python jungletest.py --import-urls=FILE

The text file simply has one URL per line.

Downloading documents

Now that the database has a list of URLs, this step downloads the files for later testing. Each download is checked to verify it is probably a legitimate file. For example, HTML downloads are rejected.

python jungletest.py --harvest-files

NOTE: This process currently runs until the list of URLs is exhausted, but when you have enough test documents, killing the process is safe.

Conducting the tests

Now that a cache of documents is available, use this command to conduct the test:

python jungletest.py --conduct-test

NOTE: This process currently runs forever, but when you have enough tests, killing the process is safe.

Security warning

Opening random, complex binary documents can be dangerous because of buffer overflows. Therefore, you should take precautions. For example, run Jungletest from its own system account.

Running without X

Once you have tested the basic system, you may want to switch to Xvfb for two benefits:

  • It's convenient for operating a separate user account just for Jungletest.
  • It's convenient because windows don't pop up to mess up your work.

See Can I run OpenOffice.org without having an X server installed,for example, to use only its UNO API? and Mandriva KB.

Headless

OpenOffice.org 2.3+ has support for running headless without Xvfb.

Future plans

  • Fix internal bugs
  • Expand tests
    • Identify buffer overflows
    • Identify memory leaks
    • Look into Mudflap and libc's MALLOC_CHECK_
    • Test export filters
    • Investigate other operations that likely cause a crash
  • Improve basic reporting
    • Identify regressions in test results
  • Develop a system for sharing (import/export) test results between systems
  • Investigate automating reporting
  • Investigate supporting AbiWord, Gnumeric, and KOffice (DCOP)

Run

grep -i "\(todo\|fixme\)" *

for more plans.

Resources

Bugs found

Ways to provoke OpenOffice.org

  • Load a smart tag component (#82346)
 
jungletest.txt · Last modified: 2008/04/17 08:36 by andrewz
 
Except where otherwise noted, content on this wiki is licensed under the following license:CC Attribution-Noncommercial-Share Alike 3.0 Unported
Recent changes RSS feed