View Proposal

---

Proposer

John See

Title

Seeing the Sound: Understanding Urban Soundscapes Through Audio-Visual Perception

Goal

The aim of this project is to build a new audio-visual soundscape dataset based on an urban city of diverse environments, equipped with human-centred perceptual characteristics. The project contains distinct scopes for both students: one investigating the subjective perception of users versus objective measures, the other using computer vision and machine learning techniques to train models that can predict measured acoustic levels from the imagery

Description

This project will be undertaken by a maximum of 2 students - both with common and distinct scopes of work. Work done by both students: Collection of audio-visual data of an urban modern city containing diverse styles (i.e. from busy to quiet areas, from crowded to sparse spaces). A good candidate city for this is Putrajaya, which is also close to the campus. There is room for determining how locations within the city will be chosen as well as the time of the day (weekend/weekday, day/night). Audio samples can be collected with a Sound Level Meter while Street-view imagery (SVI) can be captured with a smartphone camera or with Google StreetView API (alternatively). This part establishes a new audio-visual soundscape dataset for the city of Putrajaya. Student 1 will study the perceived acoustic quality of the environment solely from SVI by devising a user evaluation study, showing participants the images without hearing the corresponding sound recording and asking them to estimate characteristics like sound intensity, busyness, calmness, etc. The relationship between subjective perception and objective sound-level measurements can be investigated, drawing useful insights about the reality of such perception and potential divergences. Student 2 will investigate whether computer vision and machine learning models can be used to predict acoustic and perceptual characteristics of an urban environment from SVI. Features extracted from pixels, objects, scene semantics, and/or vision-language models can be explored for use to train models that can predict the measured acoustic levels. As perceptual characteristics are dependent on Student 1's part, it would be an added bonus to have models trained for those.

Resources

Data collection: a Sound Level Meter (for sound level measurements) and a smartphone (for image/audio recordings) - binaural microphones are a bonus to have

Background

Sound is vital to a city in how it shapes public health, cultural identity and livability of urban spaces. Urban soundscapes are not determined solely by physical sound levels but also by the perception of visual characteristics, such as traffic, buildings, vegetation, people and open spaces. Previous research has demonstrated that people can form perceptions of an urban soundscape from street-view imagery alone, and that these perceptions can be related to measured acoustic conditions. Street-view imagery (SVI) provides a scalable source of information about the visual characteristics of urban environments. Computer vision techniques can automatically extract information such as buildings, roads, vehicles, vegetation, pedestrians and scene characteristics. Previous research has demonstrated that such visual features can be related to perceived soundscape indicators and used to estimate soundscape characteristics across large urban areas.

Url

Difficulty Level

Moderate

Ethical Approval

Full

Number Of Students

2

Supervisor

John See

Keywords

soundscapes, urban perception, subjective assessment, predictive models

Degrees

Bachelor of Science in Computing Science
BSc Information Systems with Data Analytics