WP1: Infrastructure and datasets

This work package aims at the initial establishment and maintenance of a development framework for the remaining work packages.

T1.1: Infrastructure update and maintenance (M1-M36)

Arrangement and configuration of the available equipment, and, the acquisition of complementary equipment, for establishing the necessary infrastructure to meet the objectives of the project (servers, wearable cameras and additional sensors).

T1.2: Cameras network simulation (M1-M36)

Development of new features for the simulation environments of heterogeneous camera networks: MSS virtual reality and WiseMnet simulators. This task considers generating test scenarios using the software simulators available at the VPULab.

T1.3: Generation of datasets (M4-M32)

Support to other tasks for generating test data and defining evaluation methodologies. It includes the selection of appropriate datasets (sequences and associated ground-truth) and their generation if required.

Milestones:

• M1.1 Acquisition of required hardware for WP2 (T1.1, M6 rescheduled M12)
• M1.2 Development of testing scenarios for WP3 (T1.2, M12 rescheduled M15)
• M1.3 Generation of datasets for WP3 (T1.3, M16)
• M1.4 Development of testing scenarios for case studies in WP4 (T1.2, M24)
• M1.5 Generation of datasets for case studies in WP4 (T1.3, M32)
• M1.6 Refinement of testing scenarios for WP3 and WP4 (T1.2, M36)

Deliverables:

• D1.1 System infrastructure (M12 and updates every 12 months if required)
• D1.2 Camera simulation (M12, M24 and M36)
• D1.3 Evaluation datasets (M12 and updates every 12 months if required).

WP.2: Feasibility studies

To perform a study of current technologies for applications related to heterogeneous camera networks where camera mobility plays a key role. Such studies will be performed on public data-sets. If required, small scenarios will be recorded.

T2.1: People tracking for active vision (M1-M9 rescheduled M15)

To evaluate state-of-the-art approaches for target tracking in an active vision application (e.g., tracking a teacher during a lecture) where the field of view of a single camera is continuously adapted to the environment, thus deciding what to observe over time.

T2.2: Object detection for collision detection (M1-M9 rescheduled M15)

To evaluate state-of-the-art approaches for detection in two case studies using a single moving camera: collision detection with a wearable camera (that may help handicapped people to avoid obstacles) and predicting collisions for traffic monitoring environments using on-board cameras and/or PTZ monitoring.

T2.3: Scene categorization for life-logging (M1-M9 rescheduled M15)

To evaluate state-of-the-art approaches for object detection, scene identification and semantic segmentation in applications for single wearable cameras such as life-logging.

T2.4: Multi-target tracking for UAV monitoring (M1-M9 rescheduled M15)

To evaluate state-of-the-art approaches for multiple target detection and tracking in data captured from a single camera on-board of an aerial unmanned vehicle.

Milestones:

• M2.1 Preliminary selection of datasets and algorithms (T2.1, T2.2, T2.3 and T2.4, M3 rescheduled M15)
• M2.2 Implementation and performance evaluation (T2.1, T2.2, T2.3 and T2.4, M9 rescheduled M15)

Deliverables:

• D2 Feasibility studies: algorithms and findings (M9 rescheduled M15)

WP.3: Enabling technologies

To obtain research contributions to the current state-of-the-art in mobile camera networks. We consider two tasks devoted to single cameras (scene identification and semantic segmentation) and another two tasks for multiple mobile cameras (multi-view matching and cooperative detection/tracking).

T3.1: Scene identification (M9-M30)

To propose new methods for obtaining high-level descriptions of a scene and to arrange scenes into semantically coherent clusters. The goals of this task will be adapted accordingly to the conclusions of T2.3.

T3.2: Semantic segmentation (M9-M30)

To propose new methods to use descriptions of semantic segments to drive object detection approaches. The goals of this task will be adapted accordingly to the conclusions of T2.3.

T3.3: Multi-view matching (M9-M30)

To propose new methods or strategies for online multi person/object association for each camera in multi-camera settings to provide complementary information to the multi-view approaches. This task will be adapted the conclusions achieved in T2.1 and T2.2.

T3.4: Cooperative detection and tracking (M9-M30)

To exploit interactions among multiple cameras to optimize the overall performance (accuracy) via aggregation or cooperative fusion schemes. The goals of this task will be adapted to the conclusions of T2.1, T2.2 and T2.4.

Milestones:

• M3.1 First version of semantic segmentation algorithms (T3.2, M16)
• M3.2 First version of multi-view matching algorithms (T3.3, M16)
• M3.3 First version of cooperative algorithms (T3.4, M16)
• M3.4 First version of scene identification algorithms (T3.1, M20)
• M3.5 Final version of semantic segmentation algorithms (T3.1, M28)
• M3.6 Final version of scene identification algorithms (T3.2, M28)
• M3.7 Final version of multi-view matching algorithms (T3.3, M28)
• M3.8 Final version of cooperative algorithms (T3.4, M28)

Deliverables:

• D3 Technologies for mobile camera networks (M18 and M30)

WP.4: Deployment of applications

To develop demonstrators of the research contributions for selected case studies. Moreover, solutions for the identified practical issues will be provided. These demonstrators will provide applications both for developers (for evaluation and testing) as well as for final users (use cases to be defined with the help of the Observing Partners). This workpackage makes use of the results in WP1, WP2 and WP3

T4.1: Case studies: implementation and testing (M24-M32)

Using WP2 and WP3 outcomes, the final use cases will be clearly defined (scenario) and the WP3 technology will be applied to enhance the results obtained in WP2 with a single camera by making use of heterogeneous camera networks/configurations. This task also considers the implementation of the software required for each case study.

T4.2: Real-time and network issues (M28-M36)

To extend the demonstrators developed in the task T4.1 by providing solutions to the practical issues arising from the deployment of the demonstrators. This task will consider real-time and latency as the main challenges to address by applying in-node and in-network optimization strategies (e.g., exploring the accuracy-cost trade-off or by dynamic task allocation based on available resources).

T4.3 Sensor issues (M28-M36)

To extend the demonstrators developed in the task T4.1 by providing solutions to the practical issues arising from the limitations of the visual sensors. As initial approach, this task will consider the use of sensors embedded in single mobile (wearable) cameras such as the depth-based Tango sensor and GPS geolocalization.

Milestones:

• M4.1 Initial results and issues for the uses cases (T4.1, M28)
• M4.2 Solutions for real-time and network issues (T4.2, M36)
• M4.3 Solutions for using complementary sensors (T4.3, M36)

Deliverables:

• D4 Deployment and application scenarios (M36)

WP.5: Management and dissemination

To coordinate the project activities, follow-up progress and the dissemination.

T5.1: Management (M1-M36)

This task has the following activities: monthly follow up of Project progress and achievements, workplan milestones and outcomes deadlines control, Workplan updates, corrective actions and administrative issues.

T5.2: Dissemination (M1-M36)

This task coordinates the compilation and internal publication of intermediate results, as well as planning the publications in journals and conferences. It will also coordinate the dissemination of results via the project web page and Newsletters every six months.

T5.3 Workshop organization (M1-M36)

This task is devoted to the organization of one-day workshops, short-courses and tutorials for each year of the project. The target audience are researchers, developers and professionals from observing partners.

Milestones:

• M5.1 Organization of the first developer workshop (T5.3, M7 -cancelled-)
• M5.2 Organization of the second developer workshop (T5.3, M14 -to reschedule-)
• M5.3 Organization of the third developer workshop (T5.3, M21-to reschedule-)

Deliverables:

• D5 Results report (M9 -rescheduled M12- and updates every 9 months)

This workplan was updated 02/07/2018 and 29/09/2017 adjusting the original workplan after the official project approval notification in order to adjust to the budget and dates.

Last update 29/09/2018