Real-Time Idling Vehicles Detection

Using Combined Audio-Visual Deep Learning

Xiwen Li 1 , Tristalee Mangin 2 , Surojit Saha 1 , Mohammed Rehman 1 , Evan Blanchard 2 __, __

Dillon Tang 2 , Henry Poppe 2 , Ouk Choi 3 , Kerry Kelly 2 , Ross Whitaker 1

1 Scientific Computing and Imaging Institute, University of Utah

2 Department of Chemical Engineering, University of Utah

3 Incheon National University

Motivation: Why it is a problem?

Health Concern: vulnerable populations

Commercial Waste: fleet idling

__Global Warming: greenhouse gas __

Shorten sentences Roadways such as schools and hospital drop-off zones, where idling vehicles congregate, produce increased air pollution to such as long-haul idling trucks at depot/delivery centers, causes wasted fuel and engine wear

Motivation: Inspiration

__Similar application: Dynamic speed display __

Motivated by effectiveness of dynamic speed sign

Motivation: ITS cycle

We design an intelligent transportation system (ITS) to show dynamic messaging

Previous Work: Infrared-imaging Idling Vehicle Detection (IVD)

• Detects idling vehicles on a heatmap sequence using Faster R-CNN
• Limitations:
  • High latency
  • Environment effect
  • Vehicle Orientation
  • Expensive and not deployable

Problem Definition

• Vehicle status :
  • __ : A vehicle is moving__
  • __ : A static vehicle with engine off__
  • __ : A static vehicle with engine on__
• Given a video visually containing vehicles and audio clips at the same moment, our model estimates a bounding box and

Method

Experiments: Dataset

• We annotated video and audio separately
• Vehicle Motion Dataset
  • Clip length: 16
  • Training: 33015 clips
  • Validation: 8252 clips
  • Test: 13271 clips
• Audio Classification
  • Sample duration: 5 seconds
  • Training: positive 8721 negative 30618
  • Validation: positive 2491 negative 8245
  • Sample rate: 48000Hz

Reduce info

Experiments: Motion Detector

The trained vehicle motion detector is capable of localizing vehicle motion on unseen data

Experiments: Audio Classifier

Fix groundtruth motion bounding box and only estimate audio classification

We observed false positives caused by loud noise makers and parallel moving vehicles

Experiments: Audio-Visual Combined Performance

Experiments: Real-Time Deployment

Smoothing prediction: window majority voting

We deploy for 14 days at a hospital*

Monitor Accuracy:

* LDS Hospital: 8th Avenue, C St E, Salt Lake City, UT 84143, USA

Conclusion

We built an algorithm and ITS to detect idling vehicles

We will test it at more locations to generalize the system