Project Overview
Research Institute: University of Washington, University of California Berkeley
Duration: 1/2021- 09/2021
Contributions: Contributing author for academic papers, primary and secondary source researcher, written and sketch-based ideation, video and photo production
Team: Dr. James Pierce, Dr. Richmond Wong, Dr. Neilly Than, Nathaniel Gray (MHCI+D), Ashten Alexander, Tsai-Shan Chiang, Lavi Tang, Olivia Quesada, Sandra Yao
Project Summary
I was invited to join a research group led by Dr. James Pierce, a speculative critical designer, and researcher. The increasing use of smart devices (IoT) surfaced issues of consent, surveillance, power dynamics, misuse of technology, and bad actors. We explored these issues using emerging co-design methods.
The team generated a video that was submitted and accepted to the Society for Social Studies of Science (4S) conference. We built off these findings by creating a booklet with various speculative use cases of IoT we anchored to reality, crafting a study guide to explore expert and non-expert responses to these speculative uses, and submitted a paper to ACM Conference on Human Factors in Computing Systems (CHI) which was later accepted and published. I also reviewed and provided feedback for this accepted ACM CHI publication.
This video is a compilation of self-probes. We reflected on our findings in a submission to the Society for Social Studies of Science.
This is one of the domains for our designed workbook
Research
Cameras, Cameras, Everywhere!
The research group had one weekly check-in. We reported findings collected from online forums. These were delivered as presentations using video, pictures, app mock-ups, written form, etc. During this exploratory phase, we analyzed artifacts while taking notes of themes. This later helped us define parameters as we generated our research methods and materials.
This image is pulled directly from the SG Home website. Arrows outline hidden cameras and microphones.
Phase 1.1: Literature Review
While gaining context, I learned about "foot-in-door, leakage, and hole and corner" issues. These frameworks are subjective. This type of research intentionally uses subjective frameworks to generate and explore ethics. These could be simultaneously heartwarming and creepy.
Phase 1.2: Netnographics
We started by looking at primary use cases from homeowners, landlords and rental hosts, tenants, and guests, and interviews from the news. These were collected from various forums online: Reddit, Yelp, reviews on Airbnb, Seniorsonly, Bing, YouTube, Lawnforum, blogs, comments spurred from original postings, etc. We compiled quotes, stories, impressions, uses, understanding, consent or lack thereof, policies around technology surfaced from these scenarios, and other information.
Advertised Use Cases
After researching primary use cases, we looked at non-primary use cases in advertisements. We collected functions of various IoT, who the targeted consumers might be, and pulled quotes.
Compiled advertised use cases
Non-Primary Advertised Use Cases
We shared posted reactions and themes to specific prompts on this content. This surfaced ethical concerns, cultural reflection, and questions that would serve to drive deeper understandings of functionality, and benefits.
Compiled advertised use cases
Phase 1.3: Quick and Dirty Interviews
We used our internet findings to probe further into real use cases. In these interviews, we were looking for information specific to where interview participants live, how they use IoT within the context of their home, how these devices might record or transcribe information from primary and non-primary users, and what delight or problems they found in their usage.
Quick and dirty interviews
Phase 1.4: Retrospective Analysis
Next, we took a step back and looked at historical predictions about technology. We compiled evocative images, summarized advertised use cases, and pulled quotes from these use cases. These captured themes, interfaces, and their respective inputs, outputs, uses, placement, etc.
Screenshots taken from (top to bottom): home computing in "The Challenge of Change," 1961, 2-way door answering in "Westinghouse All-Electric House," mid-1950s, Internet shopping in "60s/70s retro-futurism, Predicting Internet Shopping and Skype- Archive Footage," year unknown.
Phase 1.5: Self-Probes
At this stage, we wanted to generate new questions, uses, and re-examine use through placing specific IoT within our households. For this, we did self-probes. We placed two Wyze cameras around our homes and responded to various prompts.
Self probe exploring audio obfuscation- an ultrasonic sensor detects the proximity of people to the camera. "Bleeps and bloops" sound when someone triggers the camera.
As we conducted our self-probes we explored who and what might be captured. We experimented with audio and visual obfuscation methods obscuring automatically triggered recording functionality.
In the above video, the "found footage," is from this configuration of camera placement.
In summation of this leg of research, our critical making project repurposed commercial “smart home” cameras to interrogate the cameras’ workings and the social meaning they produce as information technology and media artifacts. By repurposing and deploying commercially available smart cameras in our own personal environments, we intentionally and playfully created disruptions, misuses, and breakdowns to help us learn about the current capabilities of the cameras, and highlight new junctures that emerge from using them.
We were prompted to generate an image for "where the data lives."
After compiling and refining our most intriguing footage, we wrote descriptions, and submitted Material Experiments and Playful Disruptions: Interrogating Boundaries, Meanings, and Aesthetics of the Smart Camera to the Society for Social Studies of Science (4S). This was accepted to the 4s conference for 2021.
Personal Findings
1. There is a spectrum of privacy that exists both online and in-person. There is a range of considerations around what constitutes "eavesdropping" or voyeuristic behavior for research.
2. Ubiquitous computation is obstructed by form. It's hard to know what sensing technology might be embedded in various products.
3. Repurposing and playing with technology can be a generative form of research. It can uncover a range of behaviors from resistance, distortion, modality, etc. to better understandings of form and data collection, the dynamics of placement, etc.
Beyond Privacy and Surveillance:
Eliciting Interconnected Values with a Scenarios Workbook on Smart Home Cameras
Phase 2.1: Artifacts from Real Use Cases
Ideation
For the first phase of this second leg of research, we took our findings from the 4S submission and ideated on potential technologies. I worked strategically to articulate information and sensing functionalities of IoT with visual designers to illustrate potential use-cases. I came up with over 35 out of 70 ideas.
With a rich contextual understanding of people's attitudes toward IoT, we wanted to find actual artifacts of use.
Visual artifacts collected from real IOT use-cases devices and include domestic work scenarios.
After collecting these findings, we produced our own mockups of the Neighbor's app (Amazon) with visual artifacts from our self-probes and experimentation. These were analyzed and re-worked as the booklet evolved towards its final form.
A Neighbors app mockup I produced as a generative research method.
Phase 2.2: Iterations of the Workbook
This shows iterations of scenarios used within the finalized pamphlet. I was responsible for written content.
As we finalized our workbook and research design, I piloted our materials with a couple of participants. I collected feedback and logistical information that would help us refine the overall timing and flow of our study.
Response
Phase 2.3: Final Workbook and Primary Research
As we refined the written and visual content for the booklet, I also gave feedback on the research guide. We came to the following design for our final iterations for the workbook:
For our research, we conducted preliminary semi-structured interviews to learn about what types of IoT our participants use, job and income status, and where they were located. We used this data to help the flow of the activities when needed.
At a later date we interviewed 14 participants in total.
Each interview lasted between 50 minutes to 2 hours and covered 3 domains:
2. Social relationships
3. Dynamics reflected by access and use of technology
Research Findings
1. Fictional scenarios present the opportunity to learn more about people's life experiences with technology and their understandings of trends and issues. An unintended consequence of presenting possible scenarios was participants relating the scenarios to their own experiences with different IoT.
2. The social relationships between primary and non-primary users may make it difficult for non-primary users to “opt out” of surveillance. Social arrangements lead to different privacy considerations in the smart home as compared to device owners and technology manufacturers.
3. Low-tech or no-tech alternatives could remove surveillance technologies and instead rely on other forms of communication to address underlying problems. This indicates possible unforeseen consequences to be weighed against potential uses of technology.
Looking Back and Looking Forward
Reflection
“psychologists who write sharp and uncrossable distinctions between man’s emotions and those of other living organisms and the responses of the modern type of automatic mechanisms, should be just as careful in their denials as I am in my assertions.” -Norbert Weiner
This research used a variety of methods and demonstrates how IoTs intersect ethical issues. Algorithms, data and uses shape the experiences of individuals who use, or are subjected to, IoT. The underlying relationships and information architecture amplify patterns that have historical and contemporary concerns from national to interpersonal scale: attention economy, data mining, consent, race, profiling, abuse, foreign manipulation and intelligence, social fallout, etc. My dive into this space has equipped me to think critically, methodically, and creatively in the craft of research.
