Jungletest is an system for automated testing of OpenOffice.org (OOo) against a large set of documents in complex, binary formats. Currently, tests determine whether OOo will crash or hang when loading or scrolling through the document. The documents can be automatically harvested from the OOo Issue Tracker or through random URLs found on Yahoo.

Jungletest is alpha-quality software.

===== Practical applications =====

Practical applications of Jungletest include:

  * Quantifying the stability of OOo. For example, perhaps you could say that OOo can import 99.9999% of Microsoft Word documents without crashing.
  * Detecting regressions of fixed issues in the Issue Tracker.
  * Generally improving the stability of OOo.

===== Prerequisites =====

You must have the following software installed:

  * "System Python" (Python installed outside OOo)
  * "OpenOffice.org Python" (Python typically included with OOo)
  * [[ftp://ftp.astron.com/pub/file/|file]], a tool for identifying file types (common on UNIX systems)
  * wget (common on UNIX systems)
  * [[http://www.info-zip.org/pub/infozip/UnZip.html|unzip]] ([[http://www.info-zip.org/pub/infozip/UnZip.html#Win32|Windows]])
  * A [[http://packetstormsecurity.nl/Crackers/wordlists/|list of words]] (usually /usr/share/dict/words)
  * OpenOffice.org (of course)

Jungletest should work on Windows, but only Linux has been tested so far.

===== Changes =====

What's new in Jungletest?  Read the [[/jungletest/changelog|change log]].




===== Downloads =====

The jungletest application:
   * [[http://katana.oooninja.com/f/jungletest/jungletest-20080412.tar.bz2|jungletest-20080412.tar.bz2]] April 12, 2008
   * [[http://katana.oooninja.com/f/jungletest/jungletest-20060731.tar.bz2|jungletest-20060731.tar.bz2]] July 31, 2006

Below is a list of 110,000 URLs to feed into jungletest, so jungletest can download the URLs for testing.  The list contains doc, ppt, xls, rtf, pps, dot, docx, pptx, and xlsx.
   * [[http://katana.oooninja.com/f/jungletest/urls/urls.txt.bz2|Microsoft Office document URLs]] (2.3M; April 12, 2008)

===== Installation =====

To install Jungletest, follow these instructions.

  - Download Jungletest
  - tar xjvf jungletest-2005000.tar.bz2
  - cd jungletest-20050000
  - Edit config.py

===== Using Jungletest =====

Jungletest operation is split into three, discrete operations:

  - Gathering URLs of documents
  - Downloading documents
  - Conducting the tests

==== Gathering document URLs ====

Use any combination of the following methods to populate the database with document URLs.

=== Random document URLs from Yahoo ===

If you want to harvest document URLs from Yahoo, run:

<code>python jungletest.py --yahoo</code>

NOTE: This process currently runs until you receive your daily limit of Yahoo queries, but when you have enough URLs, killing the process is safe.

=== Document URLs from OpenOffice.org Issue Tracker ===

If you want to harvest document URLs from the OOo Issue Tracker, run:

<code>python jungletest.py --harvest-urls-issuetracker=QUERY_URL</code>

Here are some sample query URLs:

  * [[http://qa.openoffice.org/issues/buglist.cgi?issue_type=DEFECT&issue_status=CLOSED&short_desc=crash&short_desc_type=substring|Closed defects with summary including crash]]
  * [[http://qa.openoffice.org/issues/buglist.cgi?issue_type=DEFECT&issue_status=UNCONFIRMED&short_desc=crash+hang+freeze&short_desc_type=anywords|Unconfirmed defects with summary including crash, hang, or freeze]]
  * [[http://qa.openoffice.org/issues/buglist.cgi?issue_type=DEFECT&issue_status=UNCONFIRMED&issue_status=NEW&issue_status=STARTED&issue_status=REOPENED&issue_status=RESOLVED&issue_status=VERIFIED&email1=&emailtype1=exact&emailassigned_to1=1&email2=&emailtype2=exact&emailreporter2=1&issueidtype=include&issue_id=&changedin=&votes=&chfieldfrom=&chfieldto=Now&chfieldvalue=&short_desc=crash+freeze+hang+stack&short_desc_type=anywords&long_desc=&long_desc_type=substring&issue_file_loc=&issue_file_loc_type=substring&status_whiteboard=&status_whiteboard_type=substring&keywords=&keywords_type=anytokens&field0-0-0=attachments.description&type0-0-0=anywords&value0-0-0=doc+xls+ppt+rtf+wpd&field0-0-1=attachments.mimetype&type0-0-1=casesubstring&value0-0-1=application&cmdtype=doit&order=Reuse+same+sort+as+last+time&Submit+query=Submit+query|Non-closed defects with summary contains {crash, freeze, hang, stack} and has an attachment]]
  * [[http://qa.openoffice.org/issues/buglist.cgi?issue_type=DEFECT&issue_status=CLOSED&resolution=FIXED&email1=&emailtype1=exact&emailassigned_to1=1&email2=&emailtype2=exact&emailreporter2=1&issueidtype=include&issue_id=&changedin=&votes=&chfieldfrom=&chfieldto=Now&chfieldvalue=&short_desc=&short_desc_type=substring&long_desc=&long_desc_type=substring&issue_file_loc=&issue_file_loc_type=substring&status_whiteboard=&status_whiteboard_type=substring&keywords=crash&keywords_type=anytokens&field0-0-0=attachments.mimetype&type0-0-0=regexp&value0-0-0=.&cmdtype=doit&order=Reuse+same+sort+as+last+time&Submit+query=Submit+query|Closed, fixed defects with keyword crash and any attachment]] (good for OOo regression testing)
  * [[http://qa.openoffice.org/issues/buglist.cgi?issue_type=DEFECT&email1=&emailtype1=exact&emailassigned_to1=1&email2=&emailtype2=exact&emailreporter2=1&issueidtype=include&issue_id=&changedin=&votes=&chfieldfrom=&chfieldto=Now&chfieldvalue=&short_desc=&short_desc_type=anywords&long_desc=&long_desc_type=allwords&issue_file_loc=&issue_file_loc_type=substring&status_whiteboard=&status_whiteboard_type=substring&keywords=&keywords_type=anytokens&field0-0-0=short_desc&type0-0-0=anywords&value0-0-0=crash+freeze+hang&field0-0-1=keywords&type0-0-1=substring&value0-0-1=crash&field0-1-0=attachments.description&type0-1-0=regexp&value0-1-0=.&field0-2-0=short_desc&type0-2-0=notsubstring&value0-2-0=change&cmdtype=doit&newqueryname=&order=%27Importance%27&Submit+query=Submit+query|(Summary contains crash, freeze or hang) OR (keywords include crash) AND has attachment AND summary does not include change]]

=== Document URLs from a text file ===

If you want to import a list of URLs from a text file (such as the large file available from the downloads section), run:

<code>python jungletest.py --import-urls=FILE</code>

The text file simply has one URL per line.

==== Downloading documents ====

Now that the database has a list of URLs, this step downloads the files for later testing.  Each download is checked to verify it is probably a legitimate file.  For example, HTML downloads are rejected.

<code>python jungletest.py --harvest-files</code>

NOTE: This process currently runs until the list of URLs is exhausted, but when you have enough test documents, killing the process is safe.

==== Conducting the tests ====

Now that a cache of documents is available, use this command to conduct the test:

<code>python jungletest.py --conduct-test</code>

NOTE: This process currently runs forever, but when you have enough tests, killing the process is safe.

===== Security warning =====

Opening random, complex binary documents can be dangerous because of buffer overflows. Therefore, you should take precautions. For example, run Jungletest from its own system account.

===== Running without X =====

Once you have tested the basic system, you may want to switch to Xvfb for two benefits:

  * It's convenient for operating a separate user account just for Jungletest.
  * It's convenient because windows don't pop up to mess up your work.

See [[http://www.openoffice.org/FAQs/faq-source.html#13|Can I run OpenOffice.org without having an X server installed,for example, to use only its UNO API?]] and [[http://club.mandriva.com/xwiki/bin/KB/OpenOfficeCommandline
|Mandriva KB]].

==== Headless ====

OpenOffice.org 2.3+ has support for running headless without Xvfb.

===== Future plans =====

  * Fix internal bugs
  * Expand tests
    * Identify buffer overflows
    * Identify memory leaks
    * Look into Mudflap and libc's MALLOC_CHECK_
    * Test export filters
    * Investigate other operations that likely cause a crash
  * Improve basic reporting
    * Identify regressions in test results
  * Develop a system for sharing (import/export) test results between systems
  * Investigate automating reporting
  * Investigate supporting AbiWord, Gnumeric, and KOffice ([[http://developer.kde.org/documentation/tutorials/index.html|DCOP]])

Run <code>grep -i "\(todo\|fixme\)" *</code> for more plans.

===== Resources =====

  * [[http://packetstormsecurity.nl/Crackers/wordlists/|Word lists]] from packet storm

===== Bugs found =====

The following bugs were found using Jungletest.

  * Bug 58353 [[http://qa.openoffice.org/issues/show_bug.cgi?id=58353|XViewCursor::goDown() gets stuck in certain situation]]
  * Bug 58904 [[http://qa.openoffice.org/issues/show_bug.cgi?id=58904|WW8: loop on formatting frames and redlines in RTL context]] FIXED
  * Bug 59026 [[http://qa.openoffice.org/issues/show_bug.cgi?id=59026|crash on desktop terminate() through PyUNO]]
  * Bug 67571 [[http://www.openoffice.org/issues/show_bug.cgi?id=67571|RTF: crash opening file having many footnotes and graphics]] FIXED
  * Bug 67711 [[http://www.openoffice.org/issues/show_bug.cgi?id=67711|crash exporting to old MS Word formats]]
  * Bug 67907 [[http://www.openoffice.org/issues/show_bug.cgi?id=67907|WW8: opening .doc containing loads of redlines consumes memory / loops]] FIXED
  * Bug 71487 [[http://qa.openoffice.org/issues/show_bug.cgi?id=71487|WordPrefect .wpd instantly crashes on import]] (and [[http://fridrich.blogspot.com/2006/12/benefits-of-getting-lost.html|Fridrich Strba's related blog post]]) FIXED

===== Ways to provoke OpenOffice.org =====

  * Load a smart tag component ([[http://www.openoffice.org/issues/show_bug.cgi?id=82346|#82346]])