Working as a key member in a team of researchers, I provided technical insights for an inquiry into the use of online humanities archives, such as the Sunsite Jack London Collection and SPIRO, the UC Berkeley architecture slide library. A key part of this work was creating an online survey instrument that would collect data from hundreds of academics across California.
- researched online survey techniques, existing survey projects
- architected web survey system: design, recruit, collect, and export
- programmed survey: Linux, Apache, Postgresql, Perl
- deployed on Sun Solaris (plus Apache, Postgresql, Perl)
- managed IT staff and remote colocated server
- provided tech support for network and lab of 10 computers
transaction log analysis
Then, using data collected from several surveys, combined with web server transaction logs, the challenge was to identify patterns and report exploratory results. A secondary challenge was to explore the feasibility of merging survey data with transaction log data, to see if usage patterns could predict survey responses.
- managed large data sets in the 5GB range
- created IP address anonymization procedure
- created Postgresql interface for SAS
- implemented web analysis algorithms in SAS
- prepared summary document for stakeholders
- hosted review meeting
- wrote relevant sections of final publication
This article was originally written June 1, 2013.