• Select output format: HTML or Java applet graphics

Use PCA Tool in Your Web Page

You can add a a button in your web page to get the PCA results of the current page by call TAPoRware cgi script.

Here is the code for the PCA button above:

<form method="post" name="textForm" enctype="multipart/form-data" target="_blank" action="http://taporware.mcmaster.ca/~taporware/cgi-bin/prototype/tpcacluster.cgi" onsubmit="document.textForm.texturl.value=document.location.href">

<input type="hidden" name="source" value="url" />

<input type="hidden" name="texturl" />

<input type="hidden" name="freetext" value="Y"/>

<input type="hidden" name="pcatext" value="2" />>

<input type="hidden" name="percent" value="10" />

<input type="hidden" name="numword" value="30" />

<input type="hidden" name="glasgow" value="Y" />

<input type="hidden" name="disp" value="2" />

<input type="submit" value="Get PCA" />

</form>

• Service Method: plainPCA

• • textSource -- any text string. If the text format is html or xml, all the tags will be stripped
  • suboption -- subtext unit selection, the values 1/2/3 are corresponding to paragraph/percent of characters/chunk of text in words
  • percent -- this selection is related to the choice of "percent of characters"
  • chunk -- this text field is related to the choice of "chunk of text in words" in the suboption parameter
  • topWords -- number of top frequency words (may or may not exclude stop words, see below) to be investigated
  • glasgow -- a boolean value (NO) to exclude the glasgow stop words in the top frequency word list
  • userwords -- a text field for user enter his/hers stop words (separated by comma). This list will combined with the glosgow stop list if you select it
  • outFormat -- values are 1/2 which are corresponding to HTML and java applet respectively

• Obtain source text by URL or form user's local disk. If the text format is XML or HTML, strip off all the tags
• Chop the text into subtext based on user's choice, generate the word list in each subtext unit.
• Get the top frequency words user specified in the entire text, which may exclude stop words, and generate the top words counting matrix of each subtext unit
• Calculate the mean matrix of the word count matrix
• Generate the co-variance and the correlation coefficient matrix based on the mean matrix
• Generate the eigenvector and eigenvalue of the co-variance matrix
• Generate m x 2 and m x 3 matrix from the eigenvector above, where m is the number of top frequency words
• Obtain final matrix by perform the matrix (the two above separately) transpose and then time and mean matrix
• Generate result based on user selection -- a java applet or table format
• Note: for java applet, only the 3-dimensional graph is generated

• Enter a URL pointing to the source text or browse (enter) the local text file path in the corresponding field
• Select subtext unit, enter or select the related field or using default value
• Select the number of top frequency you want to investigate
• Check to include or exclude Glasgow stop words as you wish. You can also enter your own stop word list in the specified field
• Select output format: HTML or java applet graphics
• Click the submit button

• URL: http://taporware.mcmaster.ca/cgi-bin/prototype/tpcacluster.cgi

Here are the parameters:

• Endpoint URL: http://taporware.mcmaster.ca:9982
• Service URI: http://taporware.mcmaster.ca/~taporware/webservice
• Service Method: find_Collocation_Plain
• parameters:
  • textInput -- any text string. If the text format is html or xml, all the tags will be stripped
  • pattern -- unix styled pattern or regular expression
  • context -- value can be 1/2/3/4 which corresponding to Words/Lines/Sentences/Paragraphs respectively
  • contextLength -- length of context
  • sorting -- values can be 1/2/3 corresponding to co-occurrence words by frequency/alphabetically/z-score
  • outFormat -- values are html/xml/others where others format will "generate XML text in HTML"

Principal Components Analysis

Description

Generally, one corpus could be divided into many text units based on subtitles, paragraphs, number of words etc. The word distribition of each text unit can be generated easily, but the usage (or similarility) of group (clustter) of word in all the text units can not be easily measured.

The Principal Components Analysis tool applies the corresponding mathematics to reduce the dimension of the text units into 2 or 3, the the results can be displayed. The words with similar usage will be displayed close to each other to form a cluster, otherwiae, they are diaplayed apart away.

Note 1: This tool works better with large text units (i.e. at least 500 words).

Note 2: The total number of units should be less than 150. You can use the List Words tool to determine the total number of words.

Note 3: Paragraphs are determined by multiple line breaks, so some paragraphs may be very short (e.g. titles). In this case, it is better to use a subtext other than paragraphs

History

Pseudocode

Ways of Using

CGI Interface

Web Service Interface

Known Bugs

To Do

-- LianYan - 19 Jun 2006