Digital Humanities and High Performance Computing

This page is intended to assemble some information about efforts by the digital humanities computing to use high performance computing. There is a also a technically oriented listserv for those interested in exchanging ideas and strategies for on-demand, burst computing: digital humanists seem eager to push the boundaries of traditional, batch-oriented HPC and focus more on just-in-time, high intensity computation (often web-based). Please contact Stéfan Sinclair for more information about the listserv.

Projects

Matt Jockers is working with several colleagues here to set up an xGrid and do some parallel processing. For example, they will be sentence tokenizing and POS tagging 250 novels (1.7 million sentences). This exact job on a G4 took 6.5 days.

Geoffrey Rockwell and others are developing and testing a viable Burst Channel between a research server and a HPC cluster. The idea is that researchers in the humanities do mostly web based research and use "always-on" servers that run web services. Such services don't need HPC, but there are some cases where a humanities research service needs to run a compute intensive process (a "burst" to use Stéfan's felicitous term). This is where Snet (a HPC consortium) comes in. We want to demonstrate the viability of this sort of research service collaboration where a web research server can pass a process to a cluster and get results over a secure channel. The research application in the first instance would be JiTR. JiTR is designed to gather and process large numbers of texts. It might run a spider and gather hundreds of web pages on a subject. We want to use WEKA to do datamining on texts from JiTR.

Stephen Downie and colleagues (from IMERSEL and MIREX) have conducted various experiments with HPC in service of music retrieval.

Organisations

McMaster hosted a Digital Humanities and High Performance Computing workshop in April 2008. Among other useful products of this event is a set of links on DH/HPC

The NEH has a useful resource site on Humanities High Performance Computing

-- StefanSinclair - 18 May 2008