Project proposal overview

Sustainable cities, which are connected and optimized through the use of technology, are not only an unstoppable global phenomenon, but they are also the only solution for containing and reducing the alarming environmental and socio-economic repercussions that current cities produce to our planet. European cities have been leading the way on smart/sustainable city development for over a decade. European cities are pioneers in testing and implementing innovative, sustainable, and integrated solutions to become environmentally sustainable, more efficient, and more habitable. Working at an EU and on local projects and initiatives1, city governments have fostered territorial and multi-level collaboration and increasingly developed a broader smart city vision. Digital transformation, the change associated with the application of digital technology in all aspects of human society, is also part of this vision. Cities are increasingly combining the power of data technology and the power of people, creating numerous opportunities, but also posing new challenges as citizens and public authorities are increasingly concerned about privacy, security, and accountability issues.

Nowadays, many cities have opted to strategically recycle and digitally transform themselves in response to some of the greatest global challenges: population growth, pollution, scarcity of resources, water management, and energy efficiency. Information and Communication Technologies (ICT) and Big Data Analytics have emerged as efficient solutions for smart cities management: from transport to the use of energy or water resources. Their goal is to effectively and sustainably manage cities while reducing energy consumption and CO2 emissions and increasing the well-being of their inhabitants. Within this process of digital transformation, one of the critical tasks to solve is the efficient management of urban infrastructure, i.e., the public and private resources available to the city inhabitants to ease or provide services or leisure. Specifically, in this project we will address the efficient planning, installation, and maintenance of resources such as street furniture, green spaces (e.g., greenways and street trees) and street waste containers. Street/Urban elements is a collective term for objects, green spaces and pieces of equipment installed along streets and roads for various purposes. It includes among others benches, trees, post boxes, streetlamps, traffic lights, traffic signs, bus/taxi/tram stops, waste receptacles, etc. The design and placement of street elements should consider factors such as efficiency, sustainability, aesthetics, visual identity, function, people mobility, and road safety. The street elements management of a city involves enormous costs not only economic, but also in terms of human resources, time, and environmental impact.

Key stages for the adequate management of urban infrastructures are efficient and effective urban planning and infrastructure monitoring. The former aims to ensure that all users and maintenance services have convenient and safe access to and through infrastructure at the smallest cost in time and resources, whereas the latter refers to the continuous assessment of the infrastructure status for its adequate maintenance. Generally, urban planning follows solid and well-stablished premises to place and locate urban elements in cities. However, situations such the replacement of old infrastructure by a new one with different capabilities, changes in urban regulation or scope and human interaction process may result in an outdated planification, resulting in under or over availability of urban infrastructure. Similarly, the cost in time and resources of continuous dedicated human monitoring is considerable and generally underestimated in the service budget, leading to an ineffective management process and to a late (or absent) reaction to infrastructure harm or deterioration. Initiatives involving citizens in the monitoring are effective but require deep and solid community organization platforms that are scarce in the neighbourhoods of nowadays cities. Current solutions for urban planning and monitoring consist in the installation of one sensor for each item to be catalogued, that is, following Internet of things (IoT) schemes, using, for example, RFID type sensors. The use of these sensors allows geo-positioning urban elements. However, they imply great additional costs, both for installation and maintenance, and are prone to, with high probability, become potentially obsolete. These alternatives are restricted to provide only GPS information and do not allow higher semantic abstractions to enrich the generation and completeness of the street elements catalogue, neither assessing its conservation status. To alleviate these costs and to generally provide a dynamic and affordable monitoring strategy to provide descriptive mechanisms for the adequate updating of urban infrastructure, the main objective of this project consists of the adjustment and creation of the necessary technology to automatize the generation and maintenance of a catalogue of street elements and areas using Computer Vision (CV). Computer Vision is relied on for the ongoing revolution of Artificial Intelligence (AI) applications based on visual information processing. Together with the AI field, Computer Vision is currently booming, being the focus of unprecedented initiatives and funding efforts targeting to boost a transformational technology. The main reason for this explosive growth has been the development of Machine Learning schemes (mainly Deep Convolutional Neural Networks?CNNs) that are allowing to reach success rates similar to the human ones in problems requiring the analysis of visual information (e.g., object detection or semantic segmentation [1][2]). In fact, the multitude of reports (ocde.org or European Commission4), that declare Artificial Intelligence (AI) as the currently most impacting General-Purpose Technology (GPT), rely on Computer Vision for the current revolution of applications based on visual information processing.

Effective solutions based on CV are currently a reality and are already transforming and defining the world we live in. The deep learning (DL) revolution, unprecedented in the field of CV, is taking place both in the scientific and industrial fields, thanks to the broad and open availability of research ideas to confront many of the most demanded applications but also enabling new ones for the public good. Any public or private institution in charge of the street elements management will benefit from this technology. This new technology will facilitate the low-impact planning, monitoring, updating, and redistribution of street elements according to the actual necessities in urban spaces. The technological requirements include the identification and localization of urban elements, the re-identification of those already catalogued and the ability to adapt continuously to new additional elements, both new instances of elements already present in the catalogue and completely new categories.

Specifically, we propose (1) to explore the capabilities of localizing the elements in urban scenarios using moving or wearable geopositioned cameras and image based deep learning object detection approaches; (2) to explore computer vision based object re identification approaches; (3) and to explore incremental/continual learning approaches in deep networks.