Human-Robot Interaction

UI/UX Design • Desktop Application Design




Performing contamination inspection tasks in nuclear environments is dangerous for human beings. Therefore, UT Nuclear Robotics Group wants to improve worker safety by reducing their presence in the high-risk radiation environments while allowing them teleoperate mobile robots to perform contamination inspection tasks. Because the intended users don't have any robot operation experience, and may not have higher education, the requirement is to design an easy-to-use user interface.


“ How to design an easy-to-use robotics interface for people without robot operation experience?”




Project Info
  • Client: UT Nuclear Robotics Group
  • Date: June - September, 2018
My Role
  • Sole Researcher
  • Sole UI/UX Designer
Methods
  • Desktop research
  • Information architecture
  • Interaction design
  • Rapid Prototype
  • Usability test
Tools
  • Sketch
  • InVision
  • Photoshop
  • PowerPoint

pic






1. Research Insights

avatar
Situational awareness

Operators may lose situational awareness when performing inspection tasks. Remote perception and manipulation are affected by various factors such as limited field of view, orientation, depth perception, etc.1

[1] "Human Performance Issues and User Interface Design for Teleoperated Robots."
avatar
Data overload

Large amount of data is gathered by sensors and cameras. It "is vastly larger than what can fit on the screen and the needed subset of data can change rapidly and unpredictably depending on events in the robot’s environment.”2 The data can make operators feel overwhelmed.

[2] "Interface Evaluation for Mobile Robot Teleoperation."
avatar
Cognitive overload

Operators have to navigate the mobile robot in nuclear bunkers as well as control camera views when performing inspection tasks. Multitasking makes operators cognitively overload3, thus affects task performance and costs extra time.

[3] "Analysis of Human-Robot Interaction for Urban Search and Rescue."



2. Reframing the Challenge

Based on research findings from literature review, robotics professionals' feedback, and robotics companies's websites, I reframed the challenge into a more specific design question to guide me through the complex interaction design process.

“ How to improve operators' situational awareness, reduce data overload, and cut down cognitive load when they are teleoperating the mobile robots and performing inspection tasks?”






3. Task Flow

pic

In order to understand users' workflows and how the application should response in different scenarios, I made task analysis to visualize the whole process. This application has two kinds of users: operators and supervisors. Operators' primary tasks are performing contamination inspection in nuclear environments while supervisors' tasks are managing operators and inspection tasks. Therefore, in the application design I streamlined operators' inspection workflow with supervisors' task management system to make emergency response faster.






4. Information Architecture

pic




5. Sketch

pic




6. Interaction Design

Assign task

Since only supervisors are allowed to assign tasks, I set password to restrict permission.

To simplify operators' work, task templates are made. The templates contain inspection zone information, sensors and cameras' configuration and survey resolution parameters. Therefore, when operators are performing assigned tasks, the mobile robot is in autonomous mode and will follow routine routes to do the inspection. Operators can teleoperate the mobile robot when they notice suspicious events, but at most of the time, they don't have to navigate the robot. In this way, operators' working cognitive load is reduced.





Perform inspection task

In this project, one of my design principle is to borrow from outside experiences and interfaces that the operators have already encountered to help them learn this new interface faster. Because operators have to see the distant environments through the robots' cameras, I borrowed car driving experience to design the layout of camera views. The main camera view is the front view, left view is on the left and right view is on the right. The rear view is on the upper center, like the rear mirror in the car. I believe this layout will help operators feel more comfortable monitoring multiple camera views.

Additionally, the camera controls and navigation controls are put in left and right panels respectively, so that operators will not mix up the operations in any circumstance.





Review task

In task review, it shows a full task list of past tasks, onging tasks and new tasks. Suspicious tasks are highlighted so that supervisors can easily spot them. When reviewing a specific task, the main view is the replay of inspection data visualization. Supervisors can easily go to specific zones by sliding task progress bar. They can speed up or slow down reply at certain speed. If supervisors want to watch multiple replays at the same time, they can click upper right buttons on the replay to switch to multi-window view.

You can see the breadcrumb trail behind the robot in the data visualization. This design is to tell operators where they have explored, thus improve their situational awareness.










7. Usability Testing

I conducted usability testing to evaluate the design of the application. An usability testing script was made for the tests. Six tasks were designed to test the main features of the application that I had developed: task planning, inspection, and task review. UT Nuclear Robotics Group helped to recruite participants from campus. All of the participants are male, aged 18-34, representing the general gender and age of the operators in the nuclear facilities. None of them has robot operation experience.

Overall, all 5 participants thought the application is easy to use, but there was something not clear for them when first time using it. One participant (P2) said, “I think it’s like you login Google Drive or something for the first time. Like - what are these buttons? oh my god. Then like - you just know where you look for stuff...” Each participant evaluated the difficulty of the task after completing one task. The overall average score is 1.8, with 5 being the hardest.

Task 1 Task 2 Task 3 Task 4 Task 5 Task 6 Average score
Participant 1 2 2 1 2 3 1 1.8
Participant 2 4 2 2 1 4 1 2.3
Participant 3 2 1 2 1 4 1 1.8
Participant 4 1 2 3 1 1 1 1.5
Participant 5 2 1 1 2 2 1 1.5
Average score 2.25 1.5 2 1.25 2.75 1 1.8

After the usability testing, I iterated the design and corrected several issues mentioned by the participants. Many small changes were also iterated to give users more feedback about their operation.






8. Future Work

Since UT Nuclear Robotics Group is very happy with my job, they asked me to do subsequent work in this fall semester and next spring semester. After I sent the specs to UT Nuclear Robotics Group at the end of the 2018 summer vacation, they started the development of the application. In the following steps, I would:

• spend more time tweaking the icons and buttons to make the interface intuitive and appealing.

• meet the end users, know more about their needs, and their comments on the InVision prototype when they visit UT this fall. Based on their feedback, I will iterate the design and make it better.

• conduct usability testing when the programmed prototype is done. We may use eye-tracking to capture participants' gaze patterns. Additionally, We will compare users' preference of different input devices such as gamepad, touchpad, and mouse/keyboard.




Next Project

Opioid Overdose Prevention App Design