Sound Pressure Estimation Method Using Traffic Cameras and Artificial Neural Networks

Sound pressure in cities is an important information for urban planning. Measuring this information directly usually requires a complex microphone infrastructure, which may in turn require a considerable monetary investment and long term measuring periods. Cities usually have closed-circuit TV infrastructure to monitor traffic on streets, but such cameras do not have microphones. A new solution for sound pressure inference using closed-circuit TV infrastructure, i.e. for estimating sound pressure level from traffic images, is considered in this work. Specifically, we make a comparison between previously proposed non-temporal convolutional neural networks and our presently proposed solution, which uses long short-term memories (LSTM) to exploit temporal relationships within image blocks. The dataset is composed of 38 videos representing traffic under different conditions (day time, weather, and so forth), with an overall length of 995 minutes. In order to predict the average sound pressure from unseen image sequences (without audio), 130 model variations are trained using the images and audio signals from training videos within the dataset. Model performance is assessed using mean squared error and Pearson correlation between the predicted and target output signals, across ten different folds. The LSTM-based neural networks yield better results in comparison to the non-temporal architectures. The newly proposed models achieve estimation errors below those of the previously proposed models: 71.3% average correlation and mean-squared error equal to 1.14 (LSTM), in comparison to 60.2% and 1.47 (non-temporal). Regularization methods are essential for training, and the VGG16 convolutional neural networks yield the best results. Object detection architectures such as Faster R-CNN seem to have a potential for improving the prediction results. We conclude that predicting urban sound pressure from image sequences is possible. Future work topics include preparing new datasets, that represent different places and traffic viewpoints, and investigating how object detection may improve the results.