Simplifying Web Tasks with Interactive Assistance

Navigating a website or completing a task online (i.e., turning off the email notification in Facebook) can be difficult to accomplish for the population with low digital literacy. Remote help is used when a helper cannot be present in person to assist a user completing a task. Many times calling people over the phone, sending an email with multiple screenshots, recording the computer screen, or a combination of the three is how remote help occurs. At times, these mechanisms can lead to a success; however, it is not always easy to communicate certain steps in this manner, can lead to failures, is time consuming, and overall not the most efficient method. Helpers may not have full access to or be aware of user’s current state. It is also time consuming for helpers to create separate materials such as a video and corresponding instructions (usually with a different application). Furthermore, helpers may not be available at the time a user needs help. On the other side of the equation, users may also struggle with these mechanisms of remote help because they have to switch between materials, associating their current state with the helper’s instructions. The difference between the state of a web page and the assistance due to varying devices, software, or access restrictions can also cause confusion if there are gaps or steps missing. Miscommunication and misinterpretation are also potential mishappenings users can experience during this endeavor. Having an interactive method that is closer to in-person assistance for remote and asynchronous help can save both a user and the helper’s time and lead to higher success rates of tasks being completed.

This project proposes making a system that can generate personalized interactive tutorial by a helper’s demonstration, helping a user walk-through an online task step-by-step with the generated tutorial on their own computer. A record of this demonstration would ensue, and this will be used in making the interactive experience for the user that the helper is trying to assist. Furthermore, the demonstrated tutorial will provide visual signifiers to indicate where the users should focus their attention on, where to click, and/or fill out a form. Personalizing the demonstration is one challenge such as making sure a user types in their own username or password during steps where the remote helper did so. In addition, creating an interactive tutorial that can run across multiple web pages pose the challenge of maintaining the working context. The remote helper will also have the option to narrate over the steps they went executed, providing further explanation and assistance to the user. This will also increase the accessibility of the system, keeping people with visual impairment in mind. Additional benefits of this system are that the created interactive tutorial is asynchronous and scalable; it can help general online-users without just a specific user in mind. Overall, this interactive tutorial by demonstration will enable helpers to provide a customized assistance a remote user quickly, and allow users to confidently achieve tasks in a more efficient manner than other remote help methods. A user study will be conducted to compare speeds and success rates for various tasks with our proposed method versus the others aforementioned for both helpers and recipients.

The current system is implemented via a Browser extension. Overall, the goal is to record the demonstration from a remote helper, and let the helper customize their demonstration and share with users, allowing them to have an interactive experience as they attempt to accomplish their task