Modeling Driver Interaction in Semi-Autonomous Vehicles

Project Motivation:

There are currently 6 levels of autonomy in vehicles as defined by the Society of Automotive Engineers, ranging from no autonomy to complete autonomy. 

Level 2 refers to partial automation. It's more advanced features that allows the driver to (at times) take their hands of the wheel and their feet off the pedals. The driver must however remain attentive at all times and ready to resume control if needed. The best publicly available technology, such as Tesla's autopilot, lies between here and level 3. 

Level 3 autonomous vehicles are capable of driving themselves, under ideal conditions and with limitations.  Although hands are off the wheel, drivers are still required behind the wheel. A human driver is still required to take over should road conditions fall below ideal. 

Level 4 autonomous vehicles can drive themselves without human interactions (besides entering your destination) but will be restricted to known use cases. This means that while the car can primarily do all of the driving by itself, there are conditions and situations where humans must take over the wheel. We're not too far from seeing driverless vehicles out on public roads. Though regulations constrict its availability, Waymo has developed and is in the process of testing Level 4 vehicles capable of driving themselves in most environments and road conditions.

Level 5 capable vehicles should be able to monitor and maneuver through all road conditions and require no human interventions whatsoever, eliminating the need for a steering wheel and pedals. 

Level 5 cars are, unfortunately, still closer to science fiction than reality.

At level 2, 3, 4, we encounter a unique problem and a question our project is centered around. It is known as the Handoff Problem, which asks the question "can a person take control of a car at a moments notice and act safely?"

We know from previous studies that its easy to get distracted behind the wheel normally and even easier to get distracted behind the wheel of a vehicle that is semiautonomous. In fact, 

So, how can we deal with this? Can we bring a drivers attention back in time to ensure a safe outcome? And what's the probability of this happening?

It's extremely unethical to take this question straight to the road. So, we had to be a little creative and find a way to test this.

It would be ideal to test this hypothesis in a virtual reality simulation. However, given the high start-up cost of VR, testing it with a lo-fi prototype with a probabilistic model simply made more sense. So, we made a game and are testing it virtually. Here's the tutorial for the game, which thoroughly explains gameplay to users.

We are measuring two things in this game: speed of answer and accuracy of answer. Both of these variables help us to build a probabilistic model of attention.

This was our basic project flow.

For the experimental design, we settled on a within-subjects design, where subjects experience both experimental treatments: the game with the distractor task and the game without the distractor task.

We chose this to account for individual differences in attention and reaction  times. 

We also implemented a counterbalancing technique to counteract learning effects.

Next, I had to debug the game. The game was adapted from another driving game so we had to reprogram it to suit our needs. Our needs included obtaining specific timing needs and application of experimental treatments. Finally, there were some aesthetics changes.

Next, we had to make a tutorial video. This, of course, was an essential step. Users must thoroughly understand the game in order to obtain data that reflected attention rather than learning the game. A trial period was also included into the final game to counteract this.

Next, we had to setup the experiment itself. Given that we could not hold this study in a lab (due to COVID), we put the game and study onto mechanical turk, an online crowdsourcing platform. We also used PsiTurk automate data collection and experimental treatment application.

Next, was the data collection phase where the experiment is hosted on mechanical turk. 

Last, Data Analysis. By far the most important part of the project. Using python data science tools to parse and analyze, we plan to use markov chain analysis for the probabilistic model and linear regression for prediction.

Thank you!

To the Jacobs Institute for Design Innovation and my mentors: Professor Bjoern Hartmann (top left), Yash Pant (top right), and Bala Kumaravel (bottom).