11.8. Standard Module `htmllib`

This module defines a number of classes which can serve as a basis for parsing text files formatted in HTML (HyperText Mark-up Language). The classes are not directly concerned with I/O --- the have to be fed their input in string form, and will make calls to methods of a ``formatter'' object in order to produce output. The classes are designed to be used as base classes for other classes in order to add functionality, and allow most of their methods to be extended or overridden. In turn, the classes are derived from and extend the class SGMLParser defined in module sgmllib. The following is a summary of the interface defined by sgmllib.SGMLParser:

The interface to feed data to an instance is through the feed() method, which takes a string argument. This can be called with as little or as much text at a time as desired; p.feed(a); p.feed(b) has the same effect as p.feed(a+b). When the data contains complete HTML elements, these are processed immediately; incomplete elements are saved in a buffer. To force processing of all unprocessed data, call the close() method.
Example: to parse the entire contents of a file, do* parser.feed(open(file).read()); parser.close().
The interface to define semantics for HTML tags is very simple: derive a class and define methods called start_tag(), end_tag(), or do_tag(). The parser will call these at appropriate moments: start_tag or do_tag is called when an opening tag of the form <tag ...> is encountered; end_tag is called when a closing tag of the form <tag> is encountered. If an opening tag requires a corresponding closing tag, like <H1> ... </H1>, the class should define the start_tag method; if a tag requires no closing tag, like <P>, the class should define the do_tag method.

The module defines the following classes:

HTMLParser () -- function of module htmllib: This is the most basic HTML parser class. It defines one additional entity name over the names defined by the SGMLParser base class, &bullet;. It also defines handlers for the following tags: <LISTING>...</LISTING>, <XMP>...</XMP>, and <PLAINTEXT> (the latter is terminated only by end of file).

CollectingParser () -- function of module htmllib: This class, derived from HTMLParser, collects various useful bits of information from the HTML text. To this end it defines additional handlers for the following tags: <A>...</A>, <HEAD>...</HEAD>, <BODY>...</BODY>, <TITLE>...</TITLE>, <NEXTID>, and <ISINDEX>.

FormattingParser (formatter, stylesheet) -- function of module htmllib: This class, derived from CollectingParser, interprets a wide selection of HTML tags so it can produce formatted output from the parsed data. It is initialized with two objects, a formatter which should define a number of methods to format text into paragraphs, and a stylesheet which defines a number of static parameters for the formatting process. Formatters and style sheets are documented later in this section.

AnchoringParser (formatter, stylesheet) -- function of module htmllib: This class, derived from FormattingParser, extends the handling of the <A>...</A> tag pair to call the formatter's bgn_anchor() and end_anchor() methods. This allows the formatter to display the anchor in a different font or color, etc.

Instances of CollectingParser (and thus also instances of FormattingParser and AnchoringParser) have the following instance variables:

anchornames -- data of module htmllib: A list of the values of the NAME attributes of the <A> tags encountered.

anchors -- data of module htmllib: A list of the values of HREF attributes of the <A> tags encountered.

anchortypes -- data of module htmllib: A list of the values of the TYPE attributes of the <A> tags encountered.

inanchor -- data of module htmllib: Outside an <A>...</A> tag pair, this is zero. Inside such a pair, it is a unique integer, which is positive if the anchor has a HREF attribute, negative if it hasn't. Its absolute value is one more than the index of the anchor in the anchors, anchornames and anchortypes lists.

isindex -- data of module htmllib: True if the <ISINDEX> tag has been encountered.

nextid -- data of module htmllib: The attribute list of the last <NEXTID> tag encountered, or an empty list if none.

title -- data of module htmllib: The text inside the last <TITLE>...</TITLE> tag pair, or '' if no title has been encountered yet.

The anchors, anchornames and anchortypes lists are ``parallel arrays'': items in these lists with the same index pertain to the same anchor. Missing attributes default to the empty string. Anchors with neither a HREF nor a NAME attribute are not entered in these lists at all.

The module also defines a number of style sheet classes. These should never be instantiated --- their class variables are the only behavior required. Note that style sheets are specifically designed for a particular formatter implementation. The currently defined style sheets are:

NullStylesheet -- data of module htmllib: A style sheet for use on a dumb output device such as an ASCII terminal.

X11Stylesheet -- data of module htmllib: A style sheet for use with an X11 server.

MacStylesheet -- data of module htmllib: A style sheet for use on Apple Macintosh computers.

StdwinStylesheet -- data of module htmllib: A style sheet for use with the stdwin module; it is an alias for either X11Stylesheet or MacStylesheet.

GLStylesheet -- data of module htmllib: A style sheet for use with the SGI Graphics Library and its font manager (the SGI-specific built-in modules gl and fm).

Style sheets have the following class variables:

stdfontset -- data of module htmllib: A list of up to four font definititions, respectively for the roman, italic, bold and constant-width variant of a font for normal text. If the list contains less than four font definitions, the last item is used as the default for missing items. The type of a font definition depends on the formatter in use; its only use is as a parameter to the formatter's setfont() method.

h1fontset -- data of module htmllib

h2fontset -- data of module htmllib

h3fontset -- data of module htmllib: The font set used for various headers (text inside <H1>...</H1> tag pairs etc.).

stdindent -- data of module htmllib: The indentation of normal text. This is measured in the ``native'' units of the formatter in use; for some formatters these are characters, for others (especially those that actually support variable-spacing fonts) in pixels or printer points.

ddindent -- data of module htmllib: The indentation used for the first level of <DD> tags.

ulindent -- data of module htmllib: The indentation used for the first level of <UL> tags.

h1indent -- data of module htmllib: The indentation used for level 1 headers.

h2indent -- data of module htmllib: The indentation used for level 2 headers.

literalindent -- data of module htmllib: The indentation used for literal text (text inside <PRE>...</PRE> and similar tag pairs).

Although no documented implementation of a formatter exists, the FormattingParser class assumes that formatters have a certain interface. This interface requires the following methods:

setfont (fontspec) -- function of module htmllib: Set the font to be used subsequently. The fontspec argument is an item in a style sheet's font set.

flush () -- function of module htmllib: Finish the current line, if not empty, and begin a new one.

setleftindent (n) -- function of module htmllib: Set the left indentation of the following lines to n units.

needvspace (n) -- function of module htmllib: Require at least n blank lines before the next line. Implies flush().

addword (word, space) -- function of module htmllib: Add a word to the current paragraph, followed by space spaces.

nospace -- data of module htmllib: If this instance variable is true, empty words should be ignored by addword. It should be set to false after a non-empty word has been added.

setjust (justification) -- function of module htmllib: Set the justification of the current paragraph. The justification can be 'c' (center), 'l' (left justified), 'r' (right justified) or 'lr' (left and right justified).

bgn_anchor (id) -- function of module htmllib: Begin an anchor. The id parameter is the value of the parser's inanchor attribute.

end_anchor (id) -- function of module htmllib: End an anchor. The id parameter is the value of the parser's inanchor attribute.

A sample formatter implementation can be found in the module fmt, which in turn uses the module Para. These modules are not intended as standard library modules; they are available as an example of how to write a formatter.

Next: sgmllib Prev: urlparse Up: Internet and WWW Top: Top

11.8. Standard Module htmllib

11.8. Standard Module `htmllib`