Introduction and Mission Statement
Our mission is to provide a useful, elegant solution for young professionals that enables them to spend time on the tasks they find worthwhile while multitasking in the kitchen. At its base, this is about providing a useful time management tool for busy people. Another goal is to get people to cook more - once time efficiency is met with this app, the value of the cost-efficiency of cooking will be more apparent, and the act of cooking might even become an enjoyable pastime.
Our prototype has four main screens: Home, Settings, Alerts, and Analytics. The Home screen (fig. 1) consists mostly of a large “Listen” button that starts listening for sounds when pressed.To stop listening, the interface will tell the user to press the same button again (fig. 2). From the Home screen, the user can also navigate to the Settings screen (fig. 3). This screen provides two main functions: one is to choose which kitchen tasks to monitor for (water boiling, microwave done, microwave exploding, tap dripping), and the other is to choose methods of notification (phone alarm, email, text a friend). If the user chooses to text a friend, a secondary screen (fig. 4) will pop up with a list of contacts to choose from. Once the user is done with the settings, he can click “Ready” and be redirected to the Home page. If the user has already pressed “Listen,” then the app is already listening for kitchen sounds and set to notify users at this point. To visually display a list of all ongoing kitchen tasks the app is specifically listening for, the user can go to the Alerts screen (fig. 5). Here the user will just be able to scroll through all the tasks that are currently active (and also remove them if desired). The final screen is Analytics. Here the user can access information about the average time to specific kitchen events and information about events over time in both list (fig. 6) and graphical form (fig. 6, 7, 8).
1) Easy - Boiling water
2) Moderate - Alerting roommate through the app that a kitchen task is done
3) Difficult - Multitasking (ex: microwaving a potato and heating up sauce)
Bonus: Accessing boiling water metrics (also potato exploding metrics)
Busy professional, Dean, is driving home from work whilst talking to roommate Joe on the phone. Joe gets Dean to help with cooking. Joe boils water and goes to work in his room. Dean comes home, and while Dean and Joe are talking, Joe gets an alert on his phone that’s still in the kitchen. Then Joe and Dean decide to microwave a potato and heat up sauce; the app texts Dean when the tasks are done. After they eat, Bob calls asking how long it takes to boil water. Although Joe does not know how long it’d take on Bob’s particular stove and his pots, Joe shares his water boiling analytics with Bob.
Discussion of Video Prototyping
Our video was made with video recordings and pictures taken with an iPhone, and Balsamiq mockup images laid on top of pictures. We simultaneously edited different video scenes, inserted stills of the mockup, and overlaid parts of the result with voice-over narration. All video editing and voice-overs was done with iMovie and Windows Movie Maker. Mockups were cropped and put on top of a generic phone-holding picture. We also inserted some sounds (toilet flushing, music) we found on the Internet with the help of the aforementioned movie maker tools. We initially filmed everything (and often through many angles) just so we could have extra material to work with. When editing, we found that the timing of certain scenes and transitions sometimes took a few tries to get right. Some trivial scenes (ex: walking through a long hallway) were too long relative to the scenes with actual content and had to be sped up or cut. Mostly, it took a bit of effort to find the balance between providing enough context and too much context. Dividing up the scenes and working on them simultaneously was fairly effective. Also, having worked out the Balsamiq mockups ahead of filming, we had a good idea of how the live video-mockup transitions in the video would flow. We also found that improv acting appeared more natural and often created more ideas.
Audio Implementation Strategy
Our app needs to use audio processing in order to detect when the ambient noise in the kitchen corresponds to one of a number of events (water boiling, microwave beeping, etc.). This corresponds to the task of audio fingerprinting, not dissimilar to the popular app Shazam, which compares audio to a database of existing songs. We plan on following a strategy similar to that described here: http://www.redcode.nl/blog/2010/06/creating-shazam-in-java/. That is, we will capture time domain data with the device’s microphone, convert it into frequency domain data using the discrete fourier transformation (or fast fourier transformation) and sliding windows, and fingerprint the keypoints of each window. We’ll compare the data being received to hashed data that we have containing various stock sounds of these kitchen events (pre-recorded and analyzed by us). At the most basic level, we will be using TarsosDSP. If further analysis becomes necessary, we could look at FFTW (for FFT for spectral analysis) and EchoNest (automatic sound measurements). We might run into our biggest issues with consistency among kitchen sounds throughout the various environments our users will be cooking in. Accounting for variations in pot size, water level, type of liquid and heat source used will all have to be accounted for. With some early analysis of kitchen sounds (located below), the sounds appear to be distinct enough to be recognized through various environments.
Here are some spectrum analysis graphs that show the relative distinctness of a sound we will be measuring, boiling water, in various states and environments: