hyperSense: Augmenting human experience in environments

Prof Dina El Zanfaly | Carnegie Mellon University, School of Design | Fall ‘20

9.3 // Intro

Hi! I’m Amrita. I’m currently a second-year MDes student in the School of Design.

path to design: bay area, fitbit, cmu

9.8 // Engaging the Body

light + movement

9.15 // Arduino + Iterations



Case Study Inspiration

9.22 // Refining Interaction + Arduino + Reading Tweets

Refining Interactions

This week, I continued to expand on the idea of connecting two spaces, and continue developing a richer, sensory-based interaction experience.


button + ultrasonic sensor

Reading Tweets

Being Outside The Dominion of Time explores the nature of temporality as a human constraint. Temporality is an important concept to study because “it is perhaps the most fundamental constituent of human cognition” (48).

9.29 // Refining Interactions + Phenomonology Reading Tweet

I’ll be working with Rachel, Christianne, and Isabel for the semester project. We met up this weekend to discuss potential How Might We research questions + possible concepts rooted around them.

HMW brainstorm // concept ideation

Reading Tweet

Phenomenology: Is the study of consciousness and experience, and the interplay between the body experience/sensations from being in the world. As I read the article, I could not help but think about the idea of “design designs.” That encompasses the loops between humans and the tools humans create and who this relationship that unfolds through time.

Concept Progress

We’re looking to develop a perceptual portal that connects remote spaces. We’re playing around with concepts of temporality and memory. How can we leave traces of what came before?

form + interaction

10.14 // Progress + Reading Tweet

It’s been a minute since I’ve updated this process blog. Over the past few weeks, my teammates and I have been refining our interactive installation.

storyboard credit: rachel!

Body to LED Mapping

Right now, we’re thinking through the tech-side of this mirror and starting to prototype various, smaller interactions. I’ve been looking into ways to map body position to LEDs. There is a number of ways it can be done:

touchdesigner-arduino connection // ultrasonic sensor controlled LEDs based on distance

November Updates // Deep diving into the technical

We’ve been experimenting with smaller prototypes / prototyping in stages to get our primary interaction of body movements displaying on a matrix to work. Because we’re all designers, this has also helped scaffold the learning process of technical implementation.

the hardware setup: pi, rgb bonet, matrix, kinect, peripherals

Prototyping stages

I’ve been working pretty closely with my brother, who’s an engineer, to get these prototypes working (huge special superstar thanks!🌟). Dina’s been helping us with the tech implementation of this project also (giant massive superstar thanks!🌟). It’s 100% been a learning process!

Kinect to MacBook / Raspberry Pi via OpenKinect + OpenCV

  • Getting Kinect depth data to display on MacBook, then the Raspberry Pi
  • Installations took some time but eventually worked with these two tutorials and some refinements: Installing OpenCV + Libfreenect Installation
kinect depth data to pi + snacks

Kinect to Pi to Matrix: 4 prototypes

Because the Pi runs on Python code, I worked with my brother to get these prototypes working:

  • Getting the matrix up and running required splicing together/refining code from here (demo threshold file) and here (rgb matrix as a display).
im waving at the kinect here, can see handprint on matrix
matrix: shows superimposition of past + present — can see two sets of my hand // monitor: shows present video + trackbar which controls video rewind on matrix


Christianne put together the form of the installation. Some highlights:

Putting it together

Interaction Designer | Carnegie Mellon University, School of Design | MDes ’21