Hello! I'm Joshua, a PhD Candidate at the Microsoft Research Centre for Social NUI at The University of Melbourne under the supervision of Professor Frank Vetere and Dr Eduardo Velloso.
My current research aims to advance the use of eye tracking in social contexts to enrich the interaction between humans and intelligent user interfaces. During face-to-face interaction, humans naturally monitor the gaze of one another as they offer a powerful 'window into the mind'. This human ability should exist in Human-AI interaction, allowing systems to make social inferences through our gaze behaviours. I am currently in the third phase of this research, more information can be found here.
I am currently hired as the Usability Lab Manager for the Interaction Design Lab at the university where I support a wide range of HCI research projects and UX evaluations internally and externally.
In 2015, I completed a Master of Information Systems (Research Specialisation) at the university that converges on multimodal interaction, particulary with eye tracking and touch(+less) gestures, on large interactive surfaces under the supervision of Professor Frank Vetere and Dr Marcus Carter. My current research interests are in the role of gaze cues in human-computer interaction, the implicit/explicit use of eye movements for interaction in games and the use of gaze as a complementary input in multimodal interaction.
Google PhD Travel Scholarship
Computing and Information Systems,
The University of Melbourne
Melbourne Abroad Travelling Scholarship
Computing and Information Systems,
The University of Melbourne Australian Postgraduate Award
Computing and Information Systems,
The University of Melbourne Dean's Honours List 2015
Computing and Information Systems,
The University of Melbourne SocialNUI Postgraduate Top-Up Scholarship
Microsoft Research Centre for Social Natural User Interfaces,
Computing and Information Systems,
The University of Melbourne
Doctor of Philosophy (Engineering)
Microsoft Research Centre for Social Natural User Interfaces,
Computing and Information Systems,
The University of Melbourne
2016 - Present Master of Information Systems (Research Specialisation)
Microsoft Research Centre for Social Natural User Interfaces,
Computing and Information Systems,
The University of Melbourne
2014 - 2015 Bachelor of Information Technology and Systems
Faculty of Information Technology,
Monash University
2012 - 2012
Srivastava, N., Newn, J. & Velloso, E. (2018). Combining Low and Mid-Level Gaze Features for Desktop Activity Recognition.. In Proceedings of ACM Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT). 2, 4, Article 189 (December 2018).
Velloso, E., Carter, M., Newn, J., Esteves, A., Clarke, C. & Gellersen, H. (2017). Motion Correlation: Selecting Objects by Matching Their Movement. In ACM Transactions on Computer-Human Interaction (TOCHI). 24, 3, Article 22 (April 2017). [DOI]
Singh, R., Miller, T., Newn, J., Sonenberg, L., Velloso, E., & Vetere, F. (2018). Combining Planning with Gaze for Online Human Intention Recognition. In Proceedings of the 17th Conference on Autonomous Agents and MultiAgent Systems (AAMAS '18). Richland, SC.
Newn, J., Allison, F., Velloso, E., & Vetere, F. (2018). Looks Can Be Deceiving: Using Gaze Visualisation to Predict and Mislead Opponents in Strategic Gameplay. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). ACM, New York, NY, USA. [DOI]
Newn, J. (2018). Enabling Intent Recognition Through Gaze Awareness. In Proceedings of the 2018 CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA '18). ACM, New York, NY, USA. [DOI]
Newn, J., Velloso, E., Allison, F., Abdelrahman, Y. & Vetere, F. (2017). Evaluating Real-Time Gaze Representations to Infer Intention in Competitive Turn-Based Strategy Games. In Proceedings of the 2017 Annual Symposium on Computer-Human Interaction in Play (CHI PLAY '17). ACM, New York, NY, USA, 541–552. [DOI]
Newn, J., Velloso, E., Carter, M. & Vetere, F. (2016). Multimodal Segmentation of a Large Interactive Tabletop: Extending Interaction on Horizontal Surfaces with Gaze. In Proceedings of the 2016 ACM on Interactive Surfaces and Spaces (ISS '16). ACM, New York, NY, USA, 251–260. [DOI] [PDF]
Newn, J., Velloso, E., Carter, M. & Vetere, F. (2016). Exploring the Effects of Gaze Awareness on Multiplayer Gameplay . In Proceedings of the 2016 Annual Symposium on Computer-Human Interaction in Play Companion Extended Abstracts (CHI PLAY Companion '16). ACM, New York, NY, USA, 239–244. [DOI] [PDF] [Poster]
Carter, M., Newn, J., Velloso, E., & Vetere, F. (2015). Remote Gaze and Gesture Tracking on the Microsoft Kinect: Investigating the Role of Feedback. In Proceedings of the Annual Meeting of the Australian Special Interest Group for Computer Human Interaction (OzCHI '15). ACM, New York, NY, USA, 167–176. [DOI]
Matthews, O., Sarsenbayeva, Z., Jiang, W., Newn, J., Velloso, E., Clinch, S. & Goncalves, J. (2018). Inferring the Mood of a Community From Their Walking Speed: A Preliminary Study. In Adjunct Proceedings of the 2018 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp’18 Adjunct). Singapore, Singapore. [DOI]
Newn, J., Velloso, E., Carter, M. & Vetere, F. (2016). Dynamically Exposing Gaze to Foster Playful Experiences in Multiplayer Gameplay. In CHI PLAY 2016 Workshop on Designing for Emotional Complexity in Games: The Interplay of Positive and Negative Affect. Austin, TX, USA. [PDF]
Madumal, P., Singh, R., Newn, J., & Vetere, F. (2016). Interaction Design for Explainable AI. In Proceedings of the 30th Australian Conference on Computer-Human Interaction (OzCHI '18). Melbourne, VIC, Australia.
Lankes, M., Newn, J., Maurer, B., Velloso, E., Dechant, M. & Gellersen, H. (2018). EyePlay Revisited: Past, Present and Future Challenges for Eye-Based Interaction in Games. In Proceedings of the 2018 Annual Symposium on Computer-Human Interaction in Play Companion Extended Abstracts (CHI PLAY Extended Abstracts '18'). Melbourne, VIC, Australia.
Looks Can Be Deceiving: Using Gaze Visualisation to Predict and Mislead Opponents in Strategic Gameplay
2018 CHI Conference on Human Factors in Computing Systems (CHI '18),
Palais des Congrès de Montréal, Montréal, QC, Canada.
Evaluating Real-Time Gaze Representations to Infer Intentions in Competitive Turn-Based Strategy Games
2017 Annual Symposium on Computer-Human Interaction in Play (CHI PLAY '17),
Pakhuis de Zwijger, Amsterdam, North Holland, Netherlands.
Multimodal Segmentation of a Large Interactive Tabletop: Extending Interaction on Horizontal Surfaces with Gaze
2016 ACM International Conference on Interactive Surfaces and Spaces (ISS '16),
Crowne Plaza Niagara Falls, Niagara Falls, ON, Canada.
The Effect of Gaze on Gameplay in Co-located Multiplayer Gaming Environments
2016 Computing and Information Systems Doctoral Colloquium,
Melbourne School of Engineering, Parkville, VIC, Australia.
Remote Gaze and Gesture Tracking on the Microsoft Kinect: Investigating the Role of Feedback
2015 Annual Meeting of the Australian SIG for Computer Human Interaction (OzCHI '15),
Melbourne School of Design, Parkville, VIC, Australia.
Investigating Intent Recognition Through Gaze Awareness in User Interfaces
Best Poster Award
2017 Computing and Information Systems Doctoral Colloquium,
Sidney Myer Asia Centre, The University of Melbourne, Parkville, VIC, Australia.
Exploring Gaze Input in Social User Interfaces
hciLab Seminar (25 October 2017),
SimTech Building, The University of Stuttgart, Stuttgart, Germany.
Disentangling Gaze Cues in Intelligent User Interfaces: Opportunities & Challenges
Interaction Design Group Seminar (24 February 2017),
Interaction Design Lab, The University of Melbourne, Parkville, VIC, Australia.
Gaze Patterns in Social Play: Insights into the Mind of Players
Interaction Design Group Seminar (19 August 2016),
Interaction Design Lab, The University of Melbourne, Parkville, VIC, Australia.
Exploring the Combination of Eye Gaze and Gesture on Large Displays
Interaction Design Group Seminar (18 September 2015),
Interaction Design Lab, The University of Melbourne, Parkville, VIC, Australia.