Paper Reading #24: Gesture Avatar: A Technique for Operating Mobile User Interfaces Using Gestures

Reference Information
Gesture Avatar: A Technique for Operating Mobile User Interfaces Using Gestures
Hao Lu, Yang Li
Presented at CHI 2011, May 7-12, 2011, Vancouver, British Columbia, Canada

Author Bios

• Hao Lu is a graduate student in the University of Washington's Computer Science and Engineering Department and DUB Group. He is interested in new interaction methods.
• Yang Li is a senior research scientist at Google and was a research associate at the University of Washington. He holds a PhD in Computer Science from the Chinese Academy of Sciences. He is primarily interested in gesture-based interaction, with many of his projects implemented on Android.

Summary
Hypothesis
Does Gesture Avatar outperform Shift? Is it slower on large objects and faster on small ones? Is it less error-prone? Will mobile situations affect it less than Shift?

Methods
The walking tasks were performed on a treadmill. All participants used touchscreen phones extensively. A within-subjects factorial user study had half the participants use Shift then Gesture Avatar and then vice versa for the other half. For each technique, the task was performed while sitting and walking. Users were asked to locate a single highlighted target out of 24 small letter boxes. Ambiguity was simluated by controlling the distance between objects and the number of letters used. Finger positions were stabilized.

Results
Gesture Avatar is faster than Shift for small targets and slower for large ones. Medium sizes did not significantly different times. Shift while sitting was faster than Shift while standing, but Gesture Avatar found no difference. Gesture Avatar was far more consistent than Shift across the varying tests and also had lower error rates. Most participants preferred Gesture Avatar.

Contents
Touchscreen mobile devices struggle with low precision due to the size of fingers and covering up the target. Increasing the size of widgets is not always feasible. The authors developed Gesture Avatar, which combines the visibility of traditional GUIs and the casual interaction of gestures. The user's gesture is drawn to the screen and can be manipulated to affect the underlying widget. The translucent bounding box of a drawn gesture is associated with an element and can be tapped to use the widget. Gestures can be arbitrary and have multiple types of interaction. An incorrect association can be corrected with directional gestures or by dismissing it. The previous work in this area does not focus on gestures or requires significant adjustment of the UI.

The system first distinguishes taps from gestures and then displays the gesture with its bounding box and target. Gestures are first classified as characters or shapes and then identified, with shapes being compared to on-screen objects. A 2D Gaussian distribution is used to determine the target. Possible uses include a mobile browser, media player, moving the caret in a text box, and navigating a map program. In all cases, the program was wrapped into the UI as an additional interaction layer.

Gesture Avatar can be accelerated in cases of low ambiguity and can track moving targets. The stroked gesture is shown to be consistent with the avatar metaphor. It struggles with mode switching from gestures to actions.

Discussion
The authors of this paper tested Gesture Avatar against a preexisting method and supposed that theirs would perform better in many respects. Their user test was comprehensive in the range of variables, so I have no doubt that Gesture Search performs better than Shift in a number of ways.

When I was reading this paper, I couldn't help but think of Li's Gesture Search (to the point where I would accidentally type "Search" instead of "Avatar"). The upcoming release of Android 4 strives for a more consistent interface, and I can't think of what would be more consistent than combining these two systems on a mobile device. I would have tested this myself, but the interface is not yet available.

This system seems like it could help considerably with navigating webpages, much less the other examples provided. I frequently need to visit the cached version of a website in Google's search results, but usually my fingers, which are very far from huge, trigger another action instead. A targeted gesture avatar could dramatically reduce that frustration, which for me would make the software definitely worth the download for that alone.

Paper Reading #20: The Aligned Rank Transform for Nonparametric Factorial Analyses Using Only ANOVA Procedures

Reference Information
The Aligned Rank Transform for Nonparametric Factorial Analyses Using Only ANOVA Procedures
Jacob O. Wobbrock, Leah Findlater, Darren Gergle, James J. Higgins
Presented at CHI 2011, May 7-12, 2011, Vancouver, British Columbia, Canada

Author Bios

• Jacob O. Wobbrock is an Associate Professor in the Information School and an Adjunct Associate Professor in the University of Washington's Department of Computer Science and Engineering. He focuses on novel interaction techniques.
• Leah Findlater will be affiliated with University of Maryland's College of Information Studies, but is currently in the Information School at the University of Washington.
• Darren Gergle is an Associate Professor in Northwestern University's Department of Communication Studies and Department of Electrical Engineering. His work is in HCI related to visual information.
• James J. Higgins is a Professor in Kansas State University's Department of Statistics.

Summary
Hypothesis
How well do aligned rank transform (ART) analysis on existing data sets correspond with the authors' results performed through other measures?

Methods
The authors used their software with data sets from published HCI work and compared the results with the original authors' findings. One case evaluated the use of ART to provide interaciton effects. The second showed how ART  is not bound to the distributional assumptions of ANOVA. The third is nonparametric testing of repeated measures data.

Results
The first case study revealed a possible interaction that the Friedman test used in the original could not have found. It also found interactions that even an inappropriate test revealed.

The second study originally found minimal interactions because the data was lognormal. The ART test revealed that the interactions were far more significant.

The third study found that ART reduced the skew in the data and revealed all of the significant interactions, which could not be found in the original study.

Contents
Nonparametric data appears frequently in multi-factor HCI experiments, but current methods are likely to violate ANOVA assumptions or do not allow for the examine of interaction effects. Methods exist to solve this problem but are not widely available or easy to use. The authors developed a generalizable system that relies on ART to align and rank data before performing F-tests. ARTool is the desktop version and ARTweb runs online.

ART is usable in similar situations as parametric ANOVA but it does not require a continuous or ordinal response variable and does not need to be normally distributed. Rank tests apply ranks to data sets, and alignment aligns data depending on the effect to remove all effects except one. The prodcedure follows five steps. The first is to compute residuals, which is the response minus the average of all response who have a factor level that matches the response in question. Then, the estimated effects are calculated for all main and interaction effects. The authors presented a generalized version for an n-way interaction for the response. The aligned response is then found by adding the results from the previous two steps. The averaged ranks are assigned with the smallest rank from the previous step receiving a rank of 1 and so on. Then a full-factorial ANOVA is performed on the result of the previous step. The two opportunities to asess correctness are that the results of the third step should have a column that sums to 0. An ANOVA on the results of the third step should show all of the effects stripped out except the effect for which the data was aligned.

ARTool parses long-format data tables and produces aligned and ranked response for all main and interaction effects. It produces descriptive error messages in cause of a problem and has an output that contains (2+N)+2(2^N-1) columns. The system does not work in case of extreme skew of data and is best in randomized designs.

Discussion
The authors developed a system that applies a vetted statistical method and then validated it against pre-existing papers. Their method confirmed the results but also produced interesting venues of future work that went unfound. Because of this, I found their results to be believable.

I held concerns about ART because such a useful method should be standard in the average statistical package. On the other hand, it may just be a matter of the method's having gone undiscovered in CHI.

Possible future work could include comparisons of the various statistical methods the authors discussed with ART on the same data set. This could help to validate it as a means of evaluating CHI results.

Book Reading #4: Obedience to Authority

Milgram's Obedience to Authority was a fascinating, if not disturbing, read. One of my biggest complaints about conference papers is that the page limit puts a harsh constraint on the extent of detail that authors can provide about their experiment. With an entire book to fill, Milgram more than filled the pages and probably could have gone on for much longer. Like Norman, he clearly had a lot to say; however, Milgram actually avoided repeating details wherever possible. That meant that each page had a new detail about the failings of humanity when authority gets involved.

I was greatly concerned that humanity as a whole is so easily manipulated by a man in a lab coat stating that the experiment must continue at all costs. I started the book knowing that I was going to be alarmed by the results, but that didn't make the crushing knowledge that Milgram initially expected results like I did—with people stopping sooner than they did—any less painful. The whole time I kept thinking to myself that I wouldn't be like the people who were obedient, but the more I read, the more I considered that my morality might just fail as theirs did.

I was surprised that no one at all had realized that the shocking experiment was a sham. Surely someone had enough electrical cognizance to think that the hand plate would shock the person holding down the actor too. There was even an electrician who took part in the experiment and was taken in by the ruse. I don't even know how the huge numbers of people—from all backgrounds too—couldn't figure out that they weren't actually going to hurt someone.

The women who deluded herself into thinking that she was more reluctant to continue the experiment was fascinating. I didn't realize just how much memories could be manipulated to make it seem as though we are more benign that we actually act when presented with authority. I suspect that was the case with many of the people involved with the experiment. We are largely good people and something that lets us believe that we acted better than we really did gets considered as a possibility.

I appreciated that Milgram thoroughly considered possible factors that could affect our willingness to comply with authority. The study on conflicting authorities was a relief to read, since it suggested that people make the right decision when presented with an apparent moral quandary. On the other hand, the fact that we readily follow in with other people bothered me. The nature tendency is to do what everyone else wants us to do, even if that is utterly wrong.

The discussion on the importance of obedience in the creation of society provided some explanation of why it is our nature to obey. While I do have to agree that obedience and hierarchy form a good society, these can be corrupted to serve any extremely detrimental cause. Perhaps we as a whole need to rebel against our natural tendencies whenever we feel that something is wrong, since internal morality so frequently tends to be in line with what it should be.

I had expected the book to be considerably more dull than it actually was. This is due at least in part to Milgram, though a large portion of it was because of the interesting nature of the material. Milgram's writing is almost understated; he allows the content to be the predominant focus, not flowery prose. His matter-of-a-fact tone makes it seem as though he means to discuss the most boring topic on earth, which simply makes the material that much more intriguing since he is talking about something that cannot be considered mundane in any sense.

Paper Reading #19: Reflexivity in Digital Anthropology

Reference Information
Reflexivity in Digital Anthropology
Jennifer A. Rode
Presented at CHI 2011, May 7-12, 2011, Vancouver, British Columbia, Canada

Author Bios
Jennifer A. Rode is an Assistant Professor at Drexel's School of Information and a fellow in Digital Anthropology at University College London.Her dissertation research involved an ethnography to examine gender and domestic end-user programming for computer security.

Summary
Hypothesis
What are the contributions of anthropological ethnographies to the study of HCI? How can reflexive ehthnographies contribute?

Methods
The author referred to a wide body of other works to define the various forms of ethnography, especially with respect to HCI. She also determined which factors of modern anthropological ethnographies are missing form HCI works. She then discussed the different ways of producing an ethnography and design together.

Results
The voice of ethnographers is critical to make a complete ethnography. Without it, the understanding of the reasons behind a certain design are untold. Discussing rapport, participant-observation, and use of theory all aid in producing a reflexive ethnography, which can be more useful than positivistic works in design. Iterative ethnographies feature grounded theories and increase the users' interest in a design. However, all forms of ethnography have their individual merits.

Contents
HCI is slow to adopt changing practices in anthropology, especially digital anthropology, which is the comparative ethnography of the effects of techonology on how humans experience life. While there is work in HCI that is in that area, it is not reflexive. Reflexivity is one of two anthropological approaches and embraces intervention as a means of gathering data. The social-technological gap can be studied through reflexive works. The other approach is positivism, which celebrates data above all else. Ethnographies are useful to aid in the design process.

There are three main forms of writing ethnographies: realist, confessional, and impressionistic. The realist form is the only form that is commonly accepted in HCI papers. It bears several common factors with positivism and focuses the need for experimental authority, typical forms, the native's point of view, and interpretive omnipotence. The first indicates that the researcher minimizes reactivity to observed events. Typical forms include data and precision. The native's point of view implies that the ethnography is precisely in line with the views of those being observed. Interpretive omnipotence does not allow the observed to participate in the writing and does not allow for uncertainty of the information. Confessional ethnographies reveal the author's biases to address the inherent subjectivity of ethnographical work. Impressionistic ethnographies create a narrative of the observed's daily life. Confessional and impressionist ethnographies are norammly only found as a portion of a realist work. The author emphasized that none of the presented styles are lacking in rigor.

There are several elements of modern anthropological works that tend to be absent from CHI: discussing rapport, participant-observation, and use of theory. Rapport enables access to vaild data and the discussion of it is an explanation of experimental method. It helps to explain data, but is frequently considered to be understood implicitly, which detracts from the quality of the ethnography because the unknown is not enumerated. Participant-observation is being a part of the participant's daily life. It tests hypotheses through experiences rather than external validation and is not discussed first-hand in realist works. Use of theory entails using previous theories as a basis for newer ones that are only formed after working with a participant.

HCI ethnographies are usually formative, summative, or iterative. Formative ones focus on current technology to improve or produce new ones and are the most common in HCI. Summative ones evaluate technology after its design is finished. Iterative ones produce a design in stages, with participants actively aiding in its design both directly and indirectly. It requires a particularly lengthy period of time to produce.

Discussion
The author sought to analyze the contributions of anthropological ethnographies to the study of HCI. Her paper studied several HCI papers with various qualities and cited prominent papers in ethnographical work. Her work was sufficiently explanatory to let me believe in the validity of her findings that the three forms of ethnography are valid for CHI papers.

With as much as Rode frowned upon positivistic papers, I was a little surprised to see her conclusion at first. On second reading, though, I realized that she was merely suggesting that other forms of ethnographies should have merit too, not that she was advocating reflexivity over positivism. The tone of the paper as a whole led me to that first conclusion, so perhaps it would have been worth it to more clearly explain her stance before the final page.

As far as future work, iterative ethnographies could be useful in the design process, but it might be worth looking into to see if the extra time expended in iterative design results that much more of a better product. Since the type of paper the author proposed is very rare, more research should be done to see if it is a viable alterative.

Book Reading #3: Emotional Design: Why We Love (Or Hate) Everyday Things

I feel there is no better way to describe Donald Norman's Emotional Design that to compare with Design of Everyday Things. It was, simply put, a radical departure from that previous work of his that we read. It seemed to me that the author had some sort of enlightenment, since he went from disapproving of things that probably “won an award” to celebrating that visual aesthetic is also an important factor in the likability of a device. Honestly, I wonder what could have caused such a radical change in viewpoint. I would expect him to have grown more disillusioned with the visceral as time passed, not less so. It could be argued that he had an enlightenment that brought him from relative ignorance (though he hardly believed himself ignorant in the past) to the realization that people like things for more than just their uses. After all, someone had to design the thing in the first place; who would design what they did not like in someone short of being forced to do so? The worship of image that he claimed was detrimental in Design was a perfectly valid reason to produce an object in Emotional. If I had been told when reading Design that Norman was going to transform his mentality to nearly its polar opposite, I would have recommended that the person who informed me should stop lying. It is, after all, quite rare to completely alter one's viewpoint rather than simply changing it slightly. Fifteen years is a long time, but I am not certain that it is sufficiently long of a period.

I liked how he divided the dominant draws of an object into the visceral, behavioral, and reflective factors. His previous book left me wondering why he was so adamantly against beautiful objects that are just that and nothing more. In this book, though, he revealed the hypocrisy of his works: he too likes useless objects. Some of them are more than just knick-knacks, like his impossible tea kettle; they occupy a none-too-ignorable amount of space that he could have filled with more functional objects, like those he praised in Design. Considering he was so concerned with the operability of everyday objects in Design, I have to wonder if he was at all thinking about things that he didn't technically manipulate on a daily basis, but simply observed. Perhaps he thought that curiosities were not everyday enough to be considered. Regardless, I was pleased to see that he extended his definition of everyday objects to include those that are simply observed, not manipulated.

If I had a choice to read either this book or Norman's previous book and this one, I definitely would have selected just this one. Norman apparently is particularly fond of reusing material extensively. When I saw some of the examples (and images!) in Design reappear here, I was a little astonished. (I am, however, grateful that he spared us the wish for a portable computer organizer in this book.) I would understand the reuse of content if he was refuting his previous ideas, but only did this with some of the concepts. Without any background in his prior works, I feel I would have understood his ideas thoroughly. Of course, for all I know, he might have written two more books extolling the visceral and reflective attributes of objects. What I do know is that I hardly needed a further explanation of the same, with the thoroughness with which Norman writes. Perhaps that is why I am so frustrated with Norman. I don't mind his writing style too much, besides the fact that he tends to overexplain a concept. With his previous book, the last chapter taught me all I needed to know. Similarly, this one taught me all that I needed to know about the other book. His writing was actually less dense and more accessible, but still just as informative.

Paper Reading #18: Biofeedback Game Design: Using Direct and Indirect Physiological Control to Enhance Game Interaction

Reference Information
Biofeedback Game Design: Using Direct and Indirect Physiological Control to Enhance Game Interaction
Lennart E. Nacke, Michael Kalyn, Calvin Lough, Regan L. Mandryk
Presented at CHI 2011, May 7-12, 2011, Vancouver, British Columbia, Canada

Author Bios

• Lennart E. Nacke is an Assistant Professor of HCI and Game Science at the University of Ontario Institute of Technology. As a postdoctoral researcher, he worked in the Interaction Lab of the University of Saskatchewan and studied affective computing.

• Michael Kalyn is an undergraduate student in the Interaction Lab of the University of Saskatchewan's Department of Computer Science. His focuses are in affective feedback and interfacing sensors.

• Calvin Lough is affiliated with the Interaction Lab of the University of Saskatchewan's Department of Computer Science. His research is in affective computing.

• Regan L. Mandryk is an Assistant Professor in the Interaction Lab of the University of Saskatchewan's Department of Computer Science. Her research focuses on affective computing and ubiquitous and mobile gaming.

Summary
Hypothesis
How do users respond to physiological sensors that work with game controllers? Which types of physiological sensors work for certain game tasks?

Methods
The authors developed a side-scrolling shooter that uses a traditional control as the normal form of input. Physiological sensors augment the controller through indirectly or directly controlled input. There were thus three versions of a game that participants played. Two game conditions used  direct sensors and four used indirect. The direct measures were respiration and EMG on the leg, and the indirect included GSR and EKG. Both physiological games had the eye gaze power-up, though the control lacked it. All participants played the three games, presented in random order, after playing a training level. The players completed questionnaires that asked about their experience after each game and again after completing all the levels. The players were not very experienced with side-scrolling shooters and mostly used Nintendo's Wii and DS as their sole form of novel input.

Results
Players found physiological controls to be more fun than just playing with the controller, with 90% preferring some sort of physiological control. The pleased users liked the increased level of involvement and variations. The participants agreed that the physiological control was novel, involving a little learning at first, but then feeling quite natural. Users preferred eye gaze the most, but only 1/20 of the votes were for indirect sensing. Overall, direct input was preferred to indirect in each category tested. The GSR and EKG sensors were difficult to use, EMG responses were split fairly evenly, and the respiration sensor was liked due to its ease of use. The temperature sensor was easy at first, but users found it tedious over time. Users preferred multiple forms of input. Direct controls were more real-time and suited for controlling the player icon, but indirect controls were slower to respond and thus were better to control the environment. Direct controls were also thought to increase the player's sense of accomplishment. Natural mappings were preferred. The players were comfortable with wearing sensors if they contribute to gameplay.

Contents
The authors developed a classification of direct and indirect sensor input to work with traditional game control. Current physiological game design paradigms revolve around indirectly controlled signals, like heart rate. Eye gaze and muscle flexion are directly controlled. Computer games allow for a low-risk way of testing physiological HCI. This sort of interaction is called affective gaming and relies on the player's emotional state. Replacing traditional controls with biofeedback has not worked well in the past. Adapting affective games use biofeedback to alter the technical parameters or user preferences based on controller movement or button pressure, for example. Indirect controls allow players to learn to control their brainwaves, though a previous study showed that people liked explicit biofeedback in first-person shooters. Various physiological measures include eye gaze, electromyography (EMG), galvanic skin response (GSR), electrocardiography (EKG), respiration sensors, and temperature sensors. The premise of biofeedback training is to turn indirect physiological measures into increasingly directly controlled ones. Indirect sensors are available to consumers, but direct ones are not available at the moment.

The authors' game used controller mappings common in Xbox 360 shooter games with physiological input controlled separately. The game features bosses and checkpoints, which can only be activated when a player kills all the enemies. The size of enemy targets' shadows increased based on physiological control and provided a larger hit box. The backup flamethrower's range is variable. Speed and jump height also varied, though these factors were tied together. The rate of snowfall during the final boss fight also changed. An eye gaze power-up was included, but only lasted for 20 seconds to reduce eye strain and maintain game balance. The sensors were integrated through a custom C# library.

Discussion
The authors tested user preference for biofeedback controls as part of a game and which types of controls are ideal for certain tasks. Their user test was small and focused on a single genre, but they created enough of a basic framework that a large body of future work will be able to rely upon.

I was initially concerned that the sensors that were applied to users would be cumbersome. However, the players were okay with most sensors so long as they contributed to gameplay and were not excessively taxing. That brought up an important limitation: the types of sensors used must not tire the user.

I hope that future work is performed with regard to additional genres. The next foreseeable step is in first-person shooters, which are closely related to their side-scrolling brethren. I would be very interested in seeing how well "God games" like Civilization could be played with biofeedback, since that seems like the least likely effective genre imaginable for the technology.

Paper Reading #17: Privacy Risks Emerging from the Adoption of Innocuous Wearable Sensors in the Mobile Environment

Reference Information
Privacy Risks Emerging from the Adoption of Innocuous Wearable Sensors in the Mobile Environment
Andrew Raij, Animikh Ghosh, Santosh Kumar, Mani Srivastava
Presented at CHI 2011, May 7-12, 2011, Vancouver, British Columbia, Canada

Author Bios

• Andrew Raij is a Post Doctoral Fellow in University of Memphis's Computer Science Department as a member of Dr. Kumar's lab. He is interested in persuasive interfaces.

• Animikh Ghosh is a junior research associate at SETLabs and was a research assistant to Dr. Kumar at the University of Memphis. He is interested in privacy risks from participatory sensing.

• Santosh Kumar is an Associate Professor in the University of Memphis's Computer Science Department. He leads the Wireless Sensors and Mobile Ad Hoc Networks Lab.

• Mani Srivastava is a professor in UCLA's Electrical Engineering Department. He worked at Bell Labs, which he considers to be the driving force behind his interest in mobile and wireless systems.

Summary
Hypothesis
How comfortable are individuals with the possibility that their data may be made public? How can we reduce the risk of sensitive data leakage?

Methods
The authors administered a survey to users who had their data stored, people who did not have data stored, and people with stored data who were informed of the extent of the stored data to determine their comfort levels with the possibility of data compromises. The people with stored data thus took the survey twice, after having participated in the cooperating study of AutoSense that stored their data. The participants were college students. The survey measured their level of concern about data disclosure both with and without data restrictions and abstractions. Participants were informed of the informed data through the Aha visualization system.

Results
Positively-associated activities, like exercise, were acceptable to share, as was location. The group with no data stake and the group who had not yet learned of the extent of the data collected about them had similar levels of unconcern for data storage. After learning about their data, the second group had higher concern ratings. Some participants referenced that they expect physiological states to remain private. Adding a temporal context increased concern, with increasing abstractions reducing the concern. Duration was less worrisome than a timestamp. Increasing the publicity of data sets concerned users, especially when the identity was included in the data. Participants seemed initially naive to the danger of shared data, with the exception of location. The differing concerns about certain activities suggested that privacy should be handled to different extents depending on the study.

Contents
Wearable sensors record sensitive physical and physiological data, to which machine learning algorithms can be applied. This algorithms reveal a wealth of private information about behavioral states and activities, including stress levels and addictions. These inferences can be shared without the user's permission, potentially revealing private data or identifying the individual. Data sets produced from tests of wearable sensors cannot be released for that reason. Most notably, seemingly innocuous data can be combined to produce informed inferences about a person. Sensor data is hard to anonymize because it is inherently sensitive and quasi-identifying.

The authors produced a framework that focuses on how to displace the boundary where privacy and publicity are in tension. It covers measurements, behaviors, contexts, restrictions, abstractions, and privacy threats. Behaviors and contexts derive from measurements. Contexts can be further subdivided into temporal, physical, physiological, and social contexts. Restrictions and abstractions safeguard data. The former removes data from the set, and the latter tries to reduce the extent of exposure in the set.

The authors developed the Aha visualization system to provide four visualizations of individual behavior, including daily life and stress.

Discussion
The authors wanted to find out how concerned users were about sensor data being used to determine things about them and how to prevent identifiable information from being released. Their survey was well-founded and their framework seems reasonable, so I am convinced that this paper is sound.

I was very interested to see just how much could be determined about a person through seemingly unrelated data points. It was actually extremely disturbing to think that so much information could be inferred through accelerometers and stress meters.

I would be very interested in seeing this survey being expanded to cover a variety of demographics. While I would think that college students would be the most knowledgeable about the extent of information they are revealing, I am curious to see what a child or senior citizen might think. Perhaps generation gaps would emerge--or everyone would be equally ignorant of the dangers. Either way, I would love to see those results.