About Our Research
In the Human Computation and Visualization (HCV) Lab, we investigate the complementary nature of human and machine computation in visual analytics. Visual analytics is a branch of the computational sciences focused on understanding how interactive visual interfaces can be used to support analytical reasoning. These systems leverage the acuity of the human visual system, as well as our capacity to understand and reason about complex data. By pairing the human analyst with a machine collaborator for computational support, we hope to overcome some of the limitations imposed by the human brain such as limited working memory, bias, and fatigue. Similarly, we rely on the intuition that the lived experience, perceptual advantage, and adaptability of the human analyst may prove crucial in areas where purely computational analyses fail.
This strategy has led to substantial advances in many historically challenging domains. For example, over the past eight years my colleagues and I have made dramatic improvements in the tools available for modeling and predicting complex social behavior in political systems, built tools to support historians in understanding the evolution of handwriting in ancient Syriac manuscripts, and implemented interactive data-driven systems that are helping shift geospatial intelligence analysis from a black art to a science.
With these and many other promising examples of human-machine collaboration in the literature and everyday life, how do we tell if a new problem would benefit from similar strategies – and if so, how do we decompose a high-level analytic process into tractable computational tasks? In this vein, our scholarly work has focused on four complementary areas:
Foundations of Visual Analytics: challenging assumptions about the primacy of visualization in facilitating effective reasoning, and exploring the role of external factors in moderating that efficacy,
Theoretical Models of Visualization / Visual Analytics: the development of theoretical models for understanding the use of human effort in computational processes,
Applications of Visual Analytics: the practical application of visual analytics technologies to novel areas, and
Digital Humanities and Computer Science Education: exploring the role of visualization and visual analytics in enhancing humanistic scholarship as well as pedagogical intervention in education.
These studies highlight the importance of understanding human cognition as part of a larger computational system. Robust models of how humans fit into the grand scheme of computational tools is essential to promoting wider understanding of human effort as a legitimate and measurable computational resource. Moreover, our ongoing work seeks to understand how this resource might be leveraged within the larger context of particular application domains, with a particular focus on those traditionally underserved by computational tools.
Student Participation
By its nature, this kind of scholarship is well-suited to engaging undergraduate students across all class years and majors. Since its inception in 2015, the HCV Lab has hosted more than 40 undergraduate research assistants and has supported six honors theses at three of the Five Colleges. Our ranks have included three STRIDE scholars and seven additional first years, as well as many students majoring in fields outside CS (including SDS, PHI, ECO, BIO, EGR, and HST). Seven students have accompanied me to conferences where they have presented their work, and others have presented at Collaborations or to other stakeholders on campus. Most importantly, our research provides meaningful opportunities for engagement from students traditionally underrepresented in CS: two thirds of our research students have self-identified as students of color, and 1 in 4 identify as transgender or outside the gender binary.
Interdisciplinary Collaborations
In addition to the primary research objectives outlined above, a critical component of my work is leveraging the unique perspectives and methodological approaches from disciplines outside the computational sciences, with an emphasis on the humanities and social sciences. In this section, I will briefly describe three of my ongoing interdisciplinary collaborations as vignettes into the ways in which a liberal arts environment enhances my scholarly pursuits. Though at first glance they may seem orthogonal, each provides me with an opportunity to better observe the ways in which interactive visualization is both understood and used in real world contexts, as well as explore domain-specific metrics for success that go beyond speed and accuracy.
Syriac Paleography Project
Paleography is the study and description of ancient and medieval manuscripts, documents, and systems of writing. As part of an ongoing project on computer-assisted paleography entitled the Digital Analysis of Syriac Handwriting (DASH) Project, I am collaborating with Prof. Michael Penn at Stanford University to extract a database of over 50,000 individual letters from the majority of ancient Syriac manuscripts in existence. In addition to providing more fine-grained information about individual manuscripts, this project has led to new questions and new methods of inquiry when combined with traditional tools of historical and literary analysis. This project received a $25,000 award from Stanford’s Roberta Bowman Denning Fund, and was presented in a plenary session at the Eighth North American Syriac Symposium in June 2019. Since that time, we have received requests to port this tool into several additional languages including Latin and Greek, and our software has been shared with fellow digital humanities researchers at other institutions internationally.
Computing for Mental Health
A large amount of data visualization and human-computer interaction (HCI) research leverages studies from psychology to try to understand how humans work. Unfortunately, there is often a disconnect between the tightly controlled laboratory studies being referenced and the application of this knowledge in practice. At the same time, many mental health practitioners are beginning to turn toward computational tools to help stretch limited resources and support equitable access to mental healthcare. These efforts could be dramatically enhanced by leveraging what the HCI community has learned about promoting active engagement and designing unobtrusive interfaces. Working closely with practitioners, my lab is exploring the subtle ways in which technological intervention can support less biased clinical practice, as well as the ways in which it can enhance (or disrupt) therapeutic rapport. One system stemming from this collaboration was recognized with an Honorable Mention for Best Poster at the 2017 IEEE Visual Analytics Science and Technology Conference, and a second resulted in a successful CS honors thesis by Ji Young Yun in 2018.
Tailoring of Messaging Interventions for Promotion of Environmental Health Literacy
Exposure to particulate air pollution from combustion emissions contributes to disparate health risk for residents of Boston Chinatown, which is located at the intersection of two major highways. Health risks from exposure to air pollution are increased by linguistic and cultural isolation because they create health information disparities. A recently completed R21 project (5R21LM011919-02, PIs Wong, Brugge, Grinstein) demonstrated that conveying information on air pollution data in computer- and web-based interactive visualizations coupled with narratives familiar to the Boston Chinatown immigrant population increased understanding of air pollution risks. In our ongoing collaboration, we are working to develop interventions that enhance these tools to culturally tailor health communication messages about air pollution for heterogeneous multi-generational immigrant neighborhoods.