Tactile rendering of digital buttons and shapes on touchscreens using novel surface haptics technologies

Touchscreens are integrated in every aspect of our daily life as mobile phones, ATMs, tablets, vending machines, and car navigation systems. These screens provide an intuitive interface for touch interactions with no or limited haptic feedback. As a result, eye-free interaction is nearly impossible and users have to rely on visual and audio feedback to perform a task, which reduces the users' performance and experience. However, with the recent advances in surface haptics technologies, it is now becoming possible to generate complex tactile effects on touchscreens and enhance the user interactions by displaying more sophisticated haptic feedback. The overall goal of this thesis is to understand how to render realistic tactile buttons and shapes on touchscreens using novel surface haptics technologies. In this regard, the first part of the thesis focuses on creating vibrotactile feedback on a touchscreen that simulates the feeling of physical buttons using piezo actuators attached to the screen. For that purpose, we first recorded and analyzed the force, acceleration, and voltage data from twelve participants interacting with three different physical buttons: latch, toggle, and push buttons. Then, a button-specific vibrotactile stimulus was generated for each button based on the recorded data. Our results showed that participants were able to match the three digital buttons with their physical counterparts with a success rate of 83%. In addition, participants rated the degree of their subjective feelings using seven adjective pairs for all the physical and digital buttons investigated in this study. Our results showed that there exist at least three adjective pairs for which participants have rated two out of three digital buttons similar to their physical counterparts. In the second part, we investigated the recognition rate and time of five tactile shapes (i.e., triangle, square, pentagon, hexagon, and octagon) rendered by electrovibration on a touchscreen using three different methods and displayed in prototypical orientations and non-prototypical orientations (i.e., 15 degrees CW and CCW to the prototypical orientation). The results showed that the correct recognition rate of the shapes was higher when the haptically active area (area where electrovibration was on) was larger. However, as the number of edges increased, the recognition time increased and the recognition rate dropped significantly, arriving to a value slightly higher than the chance rate of 20% for non-prototypical octagon. Moreover, the recognition time for inside rendering condition was significantly shorter as compared to edge and outside rendering conditions and edge rendering condition led to the longest recognition time. Our analyses of exploration strategies revealed that participants first used global scanning to extract the coarse features of the displayed shapes, and then they applied local scanning to identify finer details, but needed another global scan for final confirmation in the case of non-prototypical shapes. We also observed that it was highly difficult to follow the edges of shapes and recognize shapes with more than five edges under electrovibration when a single finger was used for exploration. The results of these studies show that richer haptic stimuli is necessary for realistic rendering of digital buttons and shapes on touchscreens. For example, lack of kinesthetic feedback in rendering buttons and displaying haptic feedback in tangential direction only in rendering shapes adversely affected the perception of the participants in our study. These results are considered as a starting point for the development of tactile stimuli, haptically improved user interfaces, and touchscreen applications. These findings can also provide some guidance to haptic interface designers in developing techniques for efficient and effective interaction with graphical elements on touchscreens.