research-article
AUTHORs: Brian Jay Tang, Kang G. Shin
SEC '23: Proceedings of the 32nd USENIX Conference on Security Symposium
Article No.: 305, Pages 5449 - 5466
Published: 09 August 2023 Publication History
Metrics
Total Citations0Total Downloads0Last 12 Months0
Last 6 weeks0
New Citation Alert added!
This alert has been successfully added and will be sent to:
You will be notified whenever a record that you have chosen has been cited.
To manage your alert preferences, click on the button below.
Manage my Alerts
New Citation Alert!
Please log in to your account
- View Options
- References
- Media
- Tables
- Share
Abstract
People use mobile devices ubiquitously for computing, communication, storage, web browsing, and more. As a result, the information accessed and stored within mobile devices, such as financial and health information, text messages, and emails, can often be sensitive. Despite this, people frequently use their mobile devices in public areas, becoming susceptible to a simple yet effective attack - shoulder surfing. Shoulder surfing occurs when a person near a mobile user peeks at the user's mobile device, potentially acquiring passcodes, PINs, browsing behavior, or other personal information. We propose, Eye-Shield, a solution to prevent shoulder surfers from accessing/stealing sensitive on-screen information. Eye-Shield is designed to protect all types of on-screen information in real time, without any serious impediment to users' interactions with their mobile devices. Eye-Shield generates images that appear readable at close distances, but appear blurry or pixelated at farther distances and wider angles. It is capable of protecting on-screen information from shoulder surfers, operating in real time, and being minimally intrusive to the intended users. Eye-Shield protects images and text from shoulder surfers by reducing recognition rates to 24.24% and 15.91%. Our implementations of Eye-Shield achieved high frame rates for 1440×3088 screen resolutions (24 FPS for Android and 43 FPS for iOS). Eye-Shield also incurs acceptable memory usage, CPU utilization, and energy overhead. Finally, our MTurk and in-person user studies indicate that Eye-Shield protects on-screen information without a large usability cost for privacy-conscious users.
References
[1]
P. R. Center, "Mobile fact sheet," Pew Research Center, 2019.
[2]
"Social Engineering: Manipulating the Source | SANS Institute," Oct 2008, [Online; accessed 2. Mar. 2022]. [Online]. Available: https://www.sans.org/white-papers/32914
[3]
P. I. LLC, "Global visual hacking experimental study: Analysis," Ponemon Institute Research Report, 2016. [Online]. Available: https://multimedia.3m.com/mws/media/1254232O/global-visual-hacking-experiment-study-summary.pdf
[4]
B. Honan, "Visual data security white paper," Secure, 2012. [Online]. Available: https://multimedia.3m.com/mws/media/950026O/secure-white-paper.pdf
[5]
"SnapHack: Watch out for those who can hack into anyone's Snapchat! | WeLiveSecurity," Dec 2021, [Online; accessed 2. Mar. 2022]. [Online]. Available: https://www.welivesecurity.com/2021/12/09/snaphack-watch-out-those-who-can-hack-anyones-snapchat
[6]
"How I hacked my friend's PayPal account | WeLiveSecurity," Feb 2022, [Online; accessed 2. Mar. 2022]. [Online]. Available: https://www.welivesecurity.com/2022/01/24/how-i-hacked-my-friends-paypal-account
[7]
M. Eiband, M. Khamis, E. Von Zezschwitz, H. Hussmann, and F. Alt, "Understanding shoulder surfing in the wild: Stories from users and observers," in Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, 2017, pp. 4254-4265.
[8]
"Aguirre furious at photo leak of private text message |," Sep 2017, [Online; accessed 2. Mar. 2022]. [Online]. Available: https://politics.com.ph/aguirre-furious-photo-leak-private-text-message
[9]
C. Gartenberg, "Kanye West's iPhone passcode is 000000," Verge, Oct. 2018. [Online]. Available: https://www.theverge.com/tldr/2018/10/11/17964848/kanye-west-iphone-passcode-trump-iplane-apple-meeting
[10]
GearBrain Editorial Team, "5 top privacy screen protectors for iPhone and Android phones," Gearbrain, Aug 2019. [Online]. Available: https://www.gearbrain.com/privacy-screen-protector-iphone-android-2639955681. html
[11]
"Privacy & Screen Protectors | Privacy & Protection | 3M US," Jun. 2022, [Online; accessed 6. Jun. 2022]. [Online]. Available: https://www.3m.com/3M/en_US/privacy-screen-protectors-us
[12]
"What you need to know about privacy screen protectors," Jun 2021, [Online; accessed 2. Mar. 2022]. [Online]. Available: https://www.androidauthority.com/privacy-screen-protectors-explained-970541
[13]
Linshang, "Angle of Mobile Phone Privacy Film and VLT Test," Linshang, Jul. 2021. [Online]. Available: https://www.linshangtech.com/tech/privacy-film-angle-and-transmittance-tech1437.html
[14]
"Airline Seat Pitch Guide | SKYTRAX," Apr 2018, [Online; accessed 2. Mar. 2022]. [Online]. Available: https://www.airlinequality.com/info/seat-pitch-guide
[15]
"Global smartphone shipments by screen size 2018-2022 | Statista," Mar 2022, [Online; accessed 2. Mar. 2022]. [Online]. Available: https://www.statista.com/statistics/684294/global-smartphone-shipments-by-screen-size
[16]
"Most used smartphone screen resolutions in 2019," Oct 2019, [Online; accessed 2. Mar. 2022]. [Online]. Available: https://deviceatlas.com/blog/most-used-smartphone-screen-resolutions#us
[17]
M. Yoshimura, M. Kitazawa, Y. Maeda, M. Mimura, K. Tsubota, and T. Kishimoto, "Smartphone viewing distance and sleep: an experimental study utilizing motion capture technology," Nature and science of sleep, vol. 9, p. 59, 2017.
[18]
A. Papadopoulos, T. Nguyen, E. Durmus, and N. Memon, "Illusionpin: Shoulder-surfing resistant authentication using hybrid images," IEEE Transactions on Information Forensics and Security, vol. 12, no. 12, pp. 2875-2889, 2017.
Digital Library
[19]
N. H. Zakaria, D. Griffiths, S. Brostoff, and J. Yan, "Shoulder surfing defence for recall-based graphical passwords," in Proceedings of the seventh symposium on usable privacy and security, 2011, pp. 1-12.
[20]
M. Kumar, T. Garfinkel, D. Boneh, and T. Winograd, "Reducing shoulder-surfing by using gaze-based password entry," in Proceedings of the 3rd symposium on Usable privacy and security, 2007, pp. 13-19.
[21]
Y. Abdrabou, M. Khamis, R. M. Eisa, S. Ismail, and A. Elmougy, "Just gaze and wave: Exploring the use of gaze and gestures for shoulder-surfing resilient authentication," in Proceedings of the 11th acm symposium on eye tracking research & applications, 2019, pp. 1-10.
[22]
K. Bradshaw, "Future Chromebooks may alert you when someone is snooping over your shoulder," Nov 2021. [Online]. Available: https://9to5google.com/2021/11/24/future-chromebooks-snooping-protection
[23]
S. Lian, W. Hu, X. Song, and Z. Liu, "Smart privacy-preserving screen based on multiple sensor fusion," IEEE Transactions on Consumer Electronics, vol. 59, no. 1, pp. 136-143, 2013.
[24]
H. Zhou, V. Ferreira, T. Alves, K. Hawkey, and D. Reilly, "Somebody is peeking! a proximity and privacy aware tablet interface," in Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems, 2015, pp. 1971-1976.
[25]
M. Khamis, M. Eiband, M. Zurn, and H. Hussmann, "Eyespot: Leveraging gaze to protect private text content on mobile devices from shoulder surfing," Multimodal Technologies and Interaction, vol. 2, no. 3, p. 45, 2018.
[26]
"BlackBerry Privacy Shade - Apps on Google Play," Apr 2022, [Online; accessed 11. Apr. 2022]. [Online]. Available: https://play.google.com/store/apps/details?id=com. blackberry.privacyfilter&hl=en_US%E2%89%B7=US
[27]
M. Eiband, E. von Zezschwitz, D. Buschek, and H. Husmann, "My scrawl hides it all: protecting text messages against shoulder surfing with handwritten fonts," in Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems, 2016, pp. 2041-2048.
[28]
E. von Zezschwitz, S. Ebbinghaus, H. Hussmann, and A. De Luca, "You can't watch this! privacy-respectful photo browsing on smartphones," in Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, 2016, pp. 4320-4324.
[29]
C.-Y. Chen, B.-Y. Lin, J. Wang, and K. G. Shin, "Keep others from peeking at your mobile device screen!" in The 25th Annual International Conference on Mobile Computing and Networking, 2019, pp. 1-16.
[30]
S. C. Lee, "Systems and methods for switching vision correction graphical outputs on a display of an electronic device," Nov 2021, united States Patent Application: 0210350769. [Online]. Available: https://appft1.uspto.gov/netacgi/nph-Parser?Sect1=PTO1&Sect2=HITOFF&d=PG01&p=1&u=%2Fnetahtml%2FPTO%2Fsrchnum.html&r=1&f=G&l=50&s1=%2220210350769%22.PGNR.&OS=DN/20210350769&RS=DN/20210350769
[31]
D. Singh, Fundamentals of optics. PHI Learning Pvt. Ltd., 2015.
[32]
N. Moroney, M. D. Fairchild, R. W. Hunt, C. Li, M. R. Luo, and T. Newman, "The ciecam02 color appearance model," in Color and Imaging Conference, vol. 2002, no. 1. Society for Imaging Science and Technology, 2002, pp. 23-27.
[33]
X. Zhang, B. A. Wandell et al., "A spatial extension of cielab for digital color image reproduction," in SID international symposium digest of technical papers, vol. 27. Citeseer, 1996, pp. 731-734.
[34]
S. Tomar, "Converting video formats with ffmpeg," Linux journal, vol. 2006, no. 146, p. 10, 2006.
Digital Library
[35]
B. Deka, Z. Huang, C. Franzen, J. Hibschman, D. Afergan, Y. Li, J. Nichols, and R. Kumar, "Rico: A mobile app dataset for building data-driven design applications," in Proceedings of the 30th Annual Symposium on User Interface Software and Technology, ser. UIST '17, 2017.
Digital Library
[36]
E. Agustsson and R. Timofte, "Ntire 2017 challenge on single image super-resolution: Dataset and study," in The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, July 2017.
[37]
S. Caelles, J. Pont-Tuset, F. Perazzi, A. Montes, K.-K. Maninis, and L. Van Gool, "The 2019 davis challenge on vos: Unsupervised multi-object segmentation," arXiv preprint arXiv:1905.00737, 2019.
[38]
Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, "Image quality assessment: from error visibility to structural similarity," IEEE transactions on image processing, vol. 13, no. 4, pp. 600-612, 2004.
[39]
"Inspect energy use with Energy Profiler | Android Developers," Aug. 2020, [Online; accessed 24. May 2022]. [Online]. Available: https://developer.android.com/studio/profile/energy-profiler
[40]
"Energy Efficiency Guide for iOS Apps: Measure Energy Impact with Xcode," Sep. 2016, [Online; accessed 24. May 2022]. [Online]. Available: https://developer.apple.com/library/archive/documentation/Performance/Conceptual/EnergyGuide-iOS/MonitorEnergyWithXcode.html
[41]
J. Brooke, "Sus: a "quick and dirty'usability," Usability evaluation in industry, vol. 189, no. 3, 1996.
[42]
"What's next for the smartphone industry - Global site - Kantar Worldpanel," Aug. 2022, [Online; accessed 29. Aug. 2022]. [Online]. Available: https://www.kantarworldpanel.com/global/News/2017-smartphone-industry-insight-report
Recommendations
- Eye-Shield: protecting bystanders from being blinded by mobile projectors
ITS '11: Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces
This paper introduces Eye-Shield, a mobile projector-camera prototype designed for the purpose of protecting people from being accidently blinded with a handheld projector. Since they might be used regularly in public space, mobile projectors can be ...
Read More
- A Framework of Privacy Shield in Organizational Information Systems
ICMB '05: Proceedings of the International Conference on Mobile Business
Preserving Privacy and the protection of personally identifiable information (PII) have been of increasing interest over the last few years. Many privacy advocates, and a significant portion of the general public, feared that the new initiatives used in ...
Read More
- Eye-gaze interaction for mobile phones
Mobility '07: Proceedings of the 4th international conference on mobile technology, applications, and systems and the 1st international symposium on Computer human interaction in mobile technology
In this paper, we discuss the use of eye-gaze tracking technology for mobile phones. In particular we investigate how gaze interaction can be used to control applications on handheld devices. In contrast to eye-tracking systems for desktop computers, ...
Read More
Comments
Information & Contributors
Information
Published In
SEC '23: Proceedings of the 32nd USENIX Conference on Security Symposium
August 2023
7552 pages
ISBN:978-1-939133-37-3
- Editors:
- Joe Calandrino
Federal Trade Commission
, - Carmela Troncoso
EPFL
Copyright © 2023 The USENIX Association.
Sponsors
- Meta
- Google Inc.
- NSF
- IBM
- Futurewei Technologies
Publisher
USENIX Association
United States
Publication History
Published: 09 August 2023
Qualifiers
- Research-article
- Research
- Refereed limited
Acceptance Rates
Overall Acceptance Rate 40 of 100 submissions, 40%
Contributors
Other Metrics
View Article Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
Total Citations
Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 14 Jan 2025
Other Metrics
View Author Metrics
Citations
View Options
View options
Media
Figures
Other
Tables