root/trunk/apps/pdfdiff/README

= pdfdiff, A semi-generic pdf diffing and merging utility =

== Rationale ==

Most people who send us are either clueless, nefarious, or both.  Despite
a world of FLOSS document collaboration tools, people who don't "get"
FLOSS insist on doing document collaboration and/or distribution by
emailing PDFs around.

If they've sent you previous versions, the new PDF cannot be easily
compared to the new version.  If you want to make changes, you have to
import it into some other software and attempt to get it formatted
properly.  This is such a pointless hassle, when of course, 99.999% of the
time, *they* didn't edit as a PDF, they used some other more sane software
(except, perhaps in the case where the software used to generated the PDF
was Microsoft Word).

Sometimes, they don't realize this is absolutely the worst way to share a
document.  Other times, they probably just don't want you to easily figure
out what changed, and merely acquiesce to their control of the document.
Either way, we shouldn't be beholden to the clueless nor the nefarious.
pdfdiff tries to get around this problem.

== What It Does ==

pdfdiff attempts to do anything it can to extract the text from two PDFs,
show you the differences, and try to help you merge the versions if you
want to.  The basic process for detection of the text content works like
this:

  * See if the PDF actually wraps some reasonable embedded text and extract it.
  * If no embedded text is found:
      * Attempt various OCR mechanisms to extract text from the images
      * If running interactively, ask the user for help orienting the pages
  * Use various heuristic mechanisms to try to format the text in a reasonable manner.

Once text is available for both PDFs, it does a standard diff, a wdiff, a meld or other diff/merging'ing operation based on user preferences.

== What It Uses ==

The embedded text extraction is done by poppler-utils' pdftotext.  (I prefer poppler-utils to xpdf because it has a more community-oriented development process from what I've read.)

The OCR systems used are:
   * tesseract
   * gocr
   * ocrad

Once OCR data is extracted, various heuristic techniques are used to guess at which one gave the best output.  This is primarily done by the Lingua::Ident and Lingua::Identify Perl modules.

Various Perl modules are used to try to format the text in a reasonable manner after all this.  That's primarily done by Damian Conway's Text::Autoformat.

== Development ==

pdfdiff is part of the [wiki:WikiStart Loblaw] project, so development is done via its development resources for now.

== Other Ideas ==

These are merely ideas for where this program might be taken.  Please don't assume these are commitments by any developer of the project to take on or complete this work.

=== univdiff ===

It would be interesting to expand the utility from pdfdiff to something called univdiff, which would take any two file formats and attempt to normalize them down to the lowest common denominator and merge them.

On the backend, this could put out any number of formats for final editing.  For example, it could simply output a change-tracked ODF file, for a LaTeX file with markup.

[http://johnmacfarlane.net/pandoc/ pandoc] might be a helpful utility to make this happen.

== Other PDF Programs Worth Mentioning ==

 * There is [http://people.inf.ethz.ch/cremersc/misc/pdfdiff.html another program called pdfdiff], although it is a pretty simple wrapper to poppler-utils
 * [http://www.accesspdf.com/pdftk/ pdftk], a program to do various PDF manipulations and form-fill-ins.
 * [http://www.ecademix.com/JohannesHofmann/flpsed.html flpsed], a more generic Postscript file editor for inserting text.
 * [http://poppler.freedesktop.org/ Poppler], the PDF manipulation library and associated CLI programs (a fork of XPDF).
 * [http://www.foolabs.com/xpdf/ XPDF]