NYU ITP Thesis: Let’s Go For a Walk
My Role
NYU ITP MPS thesis - solo project
Timeline
August 2019 - May 2020 (10 months)
CONCEPT
Combining new and old ways of storytelling
This exploration was inspired by Shannon Mattern’s essay titled “Maintenance and Care (2018)”. In the essay she reflects on Steven Jackson’s idea of broken world thinking -- the proposition “that we take erosion, breakdown, and decay rather than novelty, growth and progress as our starting points in considering relations between society + technology.”
While reading this essay, I remembered my brother’s worn childhood quilt. I became curious to see what might develop if I made room to find out more about it, to mend or repurpose it, using it as a foundational point for something new. This exploration led to the question of how could I combine the tactile, object-based storytelling of quilts with my nephews’ love for all things mobile and AR. Essentially, how to combine new and old ways of storytelling?
SOLUTION
An AR app experience encouraging outdoor exploration together
Embroidered leaves of neighborhood tree species trigger the AR experience. When activated, leaves begin to fall, each representing a tagged memory of a matching physical tree from a walk. The app lets the user tag photos to the matching tree species while out on walks or later from camera roll.
RESEARCH
Designing tools for a closer look
Research included generational interviews with family about the history of the quilts, an interview with a NYC quiltmaker, and a conversation with my brother. It also included research on HCI papers focusing on interactive outdoor exploration like “ABBOT: a smart toy motivating children to become outdoor explorers” (2018), and also looked to trending apps like Seek and Wonderscope.
I also read essays on the importance of a closer look. One example is Tega Brain’s “The Environment is Not a System”(2018) where she touches on technology and the rise of digital naturalism, “In observing a local bird, and being assisted to identify it as a magpie, I’m learning and tuning my perception to the lifeworlds I inhabit : I’m subject to transformation.”
USER INTERVIEW - MAIN INSIGHT
Have the experience point to off screen exploration
When talking with my brother I was inspired by the walks him and my nephews would go on, often IDing trees along the way. Knowing that parents usually are the stewards of tech in young children’s lives, I asked him how he would feel having an interactive like this in their life. He replied as long as it led to off screen exploration he would feel great.
DESIGN
Tactile image targets and protoyping
The embroidered quilt leaves are fiducial markers, image targets that launch the AR experience. Leaves are based on tree species my brother gave me that are found in their area.
TESTING
Paper prototyping + Reality Composer mockups
The paper prototype tested app onboarding as well as leaves as image targets using Eyejack Creator. Due to the pandemic was unable to test again in person before end of program, but was able to start building out mockups of AR interaction in Reality Composer and test some IOS code examples to build on for saving pictures to objects in a scene.
NEXT STEPS
Continuing to build
Continue working on a prototype for testing
How to aesthetically incorporate visual aspects of quilt into AR loading animation
Ability to create sound or video memories in addition to photo
CONCLUSION + LESSONS LEARNED
What I’d do differently next time
Begin with multimodality
Record video of user testing sessions
Develop a prototype test that works well remotely
Be mindful of tutorial fatigue
Further expand on the affordances of quilts / the materiality of cloth & code