From pipe dreams and waste to functional accretion: building a capable infrastructure for the Digital Humanities

<p>This presentation is about the development and trajectory of Heurist (HeuristNetwork.org), a shared, integrated, extensible data infrastructure (model, build, manage, analyse, visualise, share, publish via integrated CMS) for Humanities research capable of handling the needs of many heterogeneous projects on a single standalone service*, with optional integration across multiple servers by a coordinating index (itself based on Heurist).</p> <p>Humanities data are interesting (both technically and to the public). They are rich in text, images, objects, people and events, heterogeneous, eminently linkable and sparse-matrix. Personal computers, the internet and other accessible technologies have spawned an exploding field (or fad?) known as Digital Humanities (DH), and opened exciting new horizons for research and public engagement.</p> <p>However, this technological turn has created many problems for a poorly funded research culture with 1-3 year grant funding cycles - choice of appropriate technology, finding and retaining technical staff, initial and ongoing costs, sustainability ... The outcome is often least-effort and inadequate technology (eg. spreadsheets) or ad hoc development, incomplete functionality, maintenance nightmares, data silos and rapid end-of-funding decay; only rich or statutory organisations can maintain a multi-component system for long. Heurist aims to overcome these problems by mutualised Open Source development, schemas stored as editable data rather than fixed structures, demand-driven priority development, and free centralised services and maintenance.</p> <p>In this presentation I will outline the evolution of our development process, from haphazard experimentation and many costly unused features (2005 - 2009) to a coherent, stable but evolving structure and Extreme Programming (aka living dangerously!), driven by immediate user requirements and incremental daily interface refinement. I will outline some of the fundamental principles we use to maintain backwards compatibility, stability, rapid development and low cost of maintenance for such a complex beast and for so many projects, on a self-funding staff of just 3 FTE. I also hope to attract some technical collaborators, as most of our users are (by design) non-technical.</p>