Tool Testing Guidelines

This is a draft set of guidelines for testing the function of the TAPoR set of tools.

• Have three texts available for testing, each in plain text/XML/HTML format. One should be short (about 1 page, or 500 words), one short-fiction or essay length (about 10 pages, or 5000 words), and one long-form or novel length (about 300 pages, or 150,000 words).
• The short text should be manually counted and have all formatting documented. An additional short text with unusual elements, such as poetry, may be desirable to expand the testing parameters.
• Each text should be run through the tool in a methodical fashion, beginning with the shortest text, recording any anomalies that become apparent. Begin with the plain-text version, then proceed to HTML, then XML.
• Check, with each format, how the tool handles various common textual elements:
  • How does it react to bulleted lists? Sentences ending versus not ending in punctuation? Space-delimiters such as tabs, returns and multiple spaces? Duplicated punctuation? (!!!!! Or ???) Ellipses, semicolons, colons, slashes, brackets, parenthesis and so forth? Accented characters? Non-punctuation special characters (monetary symbols, @, &, mathematical symbols and so forth)?
  • How does the tool count a sentence? Paragraph? Word? Are contractions like "don't" one word or two? Are abbreviations such as etc. treated as just a word, or as the end of a sentence? Do the results match the manual count of the text?
• Are stop list words removed correctly (for example, strings of characters such as 'and' which can also appear within other words)? Is the stop list sufficiently inclusive for each text tested?
• Are the specific-purpose tools behaving as described? For example, are the HTML and XML tag extractors accurate and pulling out both halves of each tag pair? Are conversion results from XML to HTML well formed? Does the date indicator pick up on synonyms and variant forms such as Halloween/Hallowe'en/All Hallow's Eve/Samhain or autumn/fall? Do "surrounding text" functions correctly display the context for all instances of a given word (choose one or two to check manually)? Cycle through these in the order they appear on each tool for each text format.
• Do the background features for each tool function as intended? For example, TAPoR should automatically carry over the user's chosen source URL from tool to tool.
• Does each tool display and behave as expected under the most common browsers (Firefox, Chrome, Internet Explorer, Safari)? How about the visualizations?

See Tool Testing Texts for the list of texts pre-prepared for this testing.

-- AmyDyrbye - 21 Sep 2011