LIDAR and spectral data segmentation and classification for autonomous vehicles simulation
ARMINES / MINES ParisTech PhD 2018
Keywords: LiDAR, classification, mobile mapping, materials, multi-spectral, deep learning.
Context: 3D Mobile mapping
Mines ParisTech has developed a technique for 3D mobile mapping, integrated in a platform called L3D2 [GOULETTE06]. L3D2 is a vehicle equipped with a precise geographical location system (GPS, IMU), a laser scanner attached to the rear of the vehicle, and cameras. This device collects 3D point clouds from urban and road environments, describing with precision the elements present along the traveled paths (roads, roundabouts, facades, trees, cars, etc.). 3D models of the scanned scenes are obtained after processing the acquired point clouds.
The image below shows an original 3D point cloud produced by the MINES ParisTech system in a Parisian street. The great number of geometric details visible in the data can be observed. On the right, the urban scene has been automatically segmented and classified.
Many applications based on relevant information extracted from 3D point clouds are possible. Scene understanding or scene change detection algorithms provide key inputs for real life applications.
The Center for Mathematical Morphology (CMM) of MINES ParisTech has an advanced expertise in the field of 3D point clouds classification [SERNA14], having obtained the best results of the state of the art in the benchmark TerraMobilita / iQmulus [VALLET15].
This PhD work will be done in close collaboration with the start-up Terra 3D, spin-off of MINES ParisTech research work on mobile mapping systems and 3D point cloud processing. The start-up transfers the research results of the laboratories CAOR and CMM of MINES ParisTech, resulting from several research projects.
PhD subject: Segmentation and classification of road scenes by analysis of LIDAR data, color and multi-spectral images
The objective of this thesis is to integrate multi-spectral data in order to improve the robustness of the semantization methods. Indeed, this information will allow a better discrimination of certain materials and thus help to remove ambiguities. It will thus be possible not only to identify the different materials, but also to determine their concentrations, adding a quantitative dimension to the description of the scene. It will also be possible to provide additional information needed for simulation purposes. For example, most analysis methods only distinguish the ground from the rest of the scene, whereas realistic rendering requires a different material to be assigned to the ground marking. MINES ParisTech has been studying hyper-spectral data for about ten years and has the required expertise to address this topic [VELASCO13]. The proposed thesis aims to push the scientific limits of these problems and to develop concrete applications. This thesis will explore the interaction between deep learning techniques and geometric approaches for automatic scene semantization.
This work will be part of the FUI REPLICA, a French federator project, in which the key actors (optis, Renault, PSA among others) collaborate in the global market for autonomous vehicle testing simulation tools.
- [GOULETTE06] François Goulette, Fawzi Nashashibi, Samer Ammoun, Claude Laurgeau, An integrated on-board laser range sensing system for On-the-Way City and Road Modelling, ISPRS International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences, 2006
- [SERNA14] Andres Serna, Beatriz Marcotegui, Detection, segmentation and classification of 3D urban objects using morphological and learning techniques, ISPRS Journal of Photogrammetry and Remote Sensing, 2014
- [VALLET15] Bruno Vallet, Mathieu Brédif, Andres Serna, Beatriz Marcotegui, Nicolas Paparoditis, TerraMobilita/iQmulus Urban Point Cloud Analysis Benchmark. Computers & Graphics, Volume 49, Pages 126-133, June 2015
- [VELASCO13] Santiago Velasco-Forero, Jesus Angulo, Classification of hyperspectral images by tensor modeling and additive morphological decomposition, Pattern Recognition, Volume 46, Issue 2, February 2013
Required general profile
- Master 2 in image processing or machine learning.
· Good relationship, rigor and autonomy · Writing and oral presentation skills. Fluent spoken and written English.
Scientific knowledge :
- 3D data processing, Image processing, Machine learning
- Good programming skills (PC development, windows or linux, in C++ and python)
Start date of the PhD: October 2018, duration 3 years.
Terra3D team as well as CAOR and CMM laboratories will be involved in this PhD. The PhD student will be mainly hosted at CMM, in the Fontainebleau MINES ParisTech site.
Institution: MINES ParisTech
Centre de Morphologie Mathématique (CMM) / Mines ParisTech
35 rue Saint-Honoré. 77305 Fontainebleau Cedex
- Beatriz Marcotegui, Full Professor (firstname.lastname@example.org)
Santiago Velasco-Forero, Permanent researcher (email@example.com)
Please, send by e-mail to (Beatriz.Marcotegui@mines-paristech.fr and Santiago.Velasco@mines-paristech.fr) with a detailed up-to-date résumé, a motivation letter, recommendation letter(s) and academic transcripts.