Assessing Multimedia

This is a (biased and incomplete) compilation of relevant resources for assessing non-traditional digital humanities scholarship. It may be considered more of a lightly annotated bibliography of resources that might be selectively re-bundled for local administrators. My bias in particular is on recognizing the scholarly value of collaborative design and development of digital humanities tools.

Our Common Commonwealth, The report of the American Council of Learned Societies Commission on Cyberinfrastructure for the Humanities and Social Sciences (2006)

The Commission believes that digital scholarship is the inevitable future of the humanities and social sciences, and that digital literacy is a matter of national competitiveness and a mission that needs to be embraced by universities, libraries, museums, and archives. In order to foster digital research, teaching, and publishing, we recommend specifically that ... policies for tenure and promotion that recognize and reward digital scholarship and scholarly communication; recognition should be given not only to scholarship that uses the humanities and social science cyberinfrastructure but also to scholarship that contributes to its design, construction, and growth ... [W]ork online or in new media—especially work involving collaboration—is not encouraged. Senior scholars now have both the opportunity and the responsibility to take certain risks, first among which is to condone risk taking in their junior colleagues and their graduate students, making sure that such endeavors are appropriately rewarded.

Report of the MLA Task Force on Evaluating Scholarship for Tenure and Promotion (2006)

A broad-ranging report on assessing scholarship, including these recommendations:

• Departments and institutions should recognize the legitimacy of scholarship produced in new media, whether by individuals or in collaboration, and create procedures for evaluating these forms of scholarship
• Departments and institutions should facilitate collaboration among scholars and evaluate it fairly

Digital scholarship is becoming pervasive in the humanities and must be recognized as a legitimate scholarly endeavor to which appropriate stan-­ dards, practices, and modes of evaluation are already being applied. The rapid expansion of digital technology has been fundamentally transform-­ ing the production and distribution of humanities scholarship, generating not only new forms of publication and dissemination—ranging from Web sites and e-­journals to print-­on-­demand books—but also significant new modes of scholarship, including digital archives and humanities databases.

Collaboration, however, offers significant opportunities for enterprising, untenured scholars to tackle problems or interdisciplinary topics too formidable in scale or scope for an individual ... In fact, recent technological advances have made collaboration with distant colleagues easier, faster, and more efficient. And the special challenges involved in creating digital scholarship have led to new forms of collaboration in that arena as well. Such opportunities to collaborate should be welcomed rather than treated with suspicion because of traditional prejudices or the difficulty of assigning credit.

Guidelines for Evaluating Work with Digital Media in the Modern Languages (2002)

While the use of computers in the modern languages is not a new phenomenon, the popular success of information networks like the World Wide Web, coupled with the proliferation of advanced multimedia tools, has resulted in an outpouring of critical publications, applied scholarship, and curricular innovation. Humanists are not only adopting new technologies but are also actively collaborating with technical experts in fields such as image processing, document encoding, and information science. Academic work in digital media should be evaluated in the light of these rapidly changing institutional and professional contexts, and departments should recognize that some traditional notions of scholarship, teaching, and service are being redefined.

For committees:

• Delineate and Communicate Responsibilities
• Engage Qualified Reviewers
• Review Work in the Medium in Which It Was Produced
• Seek Interdisciplinary Advice
• Stay Informed about Accessibility Issues

For researchers:

• Ask about Evaluation and Support
• Negotiate and Document Your Role
• Document and Explain Your Work

New Criteria for New Media, Jon Ippolito, Joline Blais, Owen F. Smith, Steve Evans and Nathan Stormer (2009)

Includes "alternative recognition measures":

• Invited / edited publications
• Live conferences
• Citations
• Download / visitor counts
• Impact in online discussions
• Impact in the real world
• Net-native recognition metrics
• Reference letters

Things to Do While Waiting for the Future to Happen: Building Cyberinfrastructure for the Liberal Arts, David Green and Michael Roy (2008)

A recent survey of the digital humanities landscape. Advocating consortial arrangements.

Types of Digital Work by Geoffrey Rockwell (2009)

• Online Peer Reviewed Publication
• Scholarly Electronic Editions
• Specifications
• Research Tools
• Hypermedia and Hyperfiction
• Instructional Technology, Computer Assisted Instruction (CAI), and Computer Assisted Language Learning (CALL)
• Research Blogs

Tools therefore instantiate hemeneutical positions about what questions are important and what interpretations should look like with a concreteness that aims for transparency. Tools can be designed naively, without attention to their hermeneutical presuppositions, but they can also be theorized well.

Other resources to mine:

http://www.mla.org/resources/documents/rep_it/dig_eval

http://lists.digitalhumanities.org/pipermail/humanist/2009-February/000274.html

-- StefanSinclair - 12 Aug 2009