LEAN & COMPUTING IN CONSTRUCTION CONGRESS 3D BUILDING MODELING WITH MULTI-SOURCE DATA A STUDY OF HIGH-DENSITY URBAN AREA IN HONG KONG Chen, K. Xue, F. Lu, W. PhD Cand. RAP Assoc. Prof. iLab, Faculty of Architecture The University of Hong Kong JC3-2017, 4-7 July, 2017 Heraklion, Crete, Greece OUTLINE 1 Background & Opportunity 2 A Pilot of A Multi-source Modeling Method in HK 3 Discussion & Future Research 2 Chen, Xue, Lu: Multi-source 3D building modeling (JC3-2017, Heraklion, Greece) 1. BACKGROUND & OPPORTUNITY 3 Chen, Xue, Lu: Multi-source 3D building modeling (JC3-2017, Heraklion, Greece) 1.1 BACKGROUND  3D modeling from measurement (as-built BIM†)(Volk et al.,2014)  The digital foundation of smart city development, e.g.  Smart construction and human-machine cooperation  Healthy aging and accessibility development  Energy consumption and waste emissions reduction  Facility management & built heritage conservations, etc.  Popular (semi-)auto methods & data types  Photogrammetry, point cloud-based, GIS-based, rules, etc.  Image (video), point clouds, GIS data layers, knowledge, etc. An example of photogrammetry: Kowloon Wall City (Source: patrick-@sketchfab.com) An example of point cloud: Pompei City (Source: MAP-Gamsau lab, CNRS, France)  Not new, but still very challenging in scenarios of CIM‡  Unsatisfactory semantic/abstraction   huge size, poor reusability, & poor details in models  Obstacles: Complexity, incomplete data, noises, etc.  Especially for high-rise high-density urban areas Chen, Xue, Lu: Multi-source 3D building modeling (JC3-2017, Heraklion, Greece) An example of GIS-based: 3D 4 Berlin (Open Data, source: berlin.de) †: Building information modeling ‡: City information modeling 1.1 BACKGROUND Smart governance Smart living Smart environment Camera Matching by features; Structure from Motion(SfM) Data-driven (many) Meshing; Texturing Simultaneous localization and mapping (SLAM) Laser scanner Stitched point cloud Semantically rich 3D building models (in CIM) Mesh surface Point cloud ModelDriven (few) BIM component Database 5 (Some icons from Wikipedia: CC-BY; CC-BY-NC) Chen, Xue, Lu: Multi-source 3D building modeling (JC3-2017, Heraklion, Greece) A previous study (Xue et al., 2016) 1.2 PREVIOUS STUDY & LOD  There exist (semi-)automatic data-driven methods on single-source data  Adopted computer vision methods, e.g., RANSAC†, SGM‡.  CityGML* defines LoD (Level-of-Detail) Data sources No. of buildings Auto? LoD Reference Aerial images city-scale Yes Between LoD1 and LoD2 Lafarge et al. (2008) Aerial images 3 No Between LoD2 and LoD3 Singh et al. (2014) Aerial images city-scale Yes LoD2 Haala et al. (2015) Aerial images 375 Yes Between LoD1 and LoD2 Li et al. (2016) Satellite images Univ. campus No LoD1 Fraser et al. (2001) Satellite images city-scale No LoD2 Kocaman et al. (2006) LoD0 (2.5D DTM) LiDAR 51 Yes Between LoD1 and LoD2 Maas and Vosselman (1999) LiDAR city-scale Yes Between LoD1 and LoD2 Poullis and You (2009)  LoD1 (prismatic block, LiDAR <20 Yes LoD2 Sun and Salvaggio (2013) LiDAR 29 No LoD2 Heo et al. (2013) LiDAR 39 Yes LoD2 Yan et al. (2016)  flat roof)  LoD2 (+ roof shapes, assemblies)  where the auto single-source methods can go  LoD3 (+ detailed outer surfaces, roof structures) Chen, Xue, Lu: Multi-source 3D building modeling (JC3-2017, Heraklion, Greece) †: Random sampling consensus ‡: Semi-global matching *: See Kolbe et al. (2005) : Digital terrain model 6 1.2 PREVIOUS STUDY & LOD  Capability of multi-source methods was still limited to generating LoD2 models  If not involving manual work Data sources No. of buildings Auto? LoD Reference Satellite images; LiDAR 170 Yes LoD1 Sohn and Dowman (2007) Aerial images; LiDAR <30 Yes Between LoD1 and LoD2 Rottensteiner and Jansa (2002) LiDAR; Camera city-scale No Between LoD2 and LoD3 Frueh and Zakhor (2003) Aerial images; LiDAR <50 Yes LoD2 Cheng et al. (2011) Aerial images; LiDAR 88 No N/A (Roof only) Awrangjeb et al. (2013) Aerial images; LiDAR 107 No LoD2 Siddiqui et al. (2016) Aerial images; LiDAR; 2D vector map 170 No Between LoD2 and LoD3 Zhang et al. (2005) LiDAR; Building footprints 529 N.A. LoD2 OpenStreetMap; Shuttle radar mission city-scale No LoD1 Alexander et al. (2009) Over et al. (2010) OpenStreetMap; Aerial images >700 Yes LoD2 Laser scanner; Ground plans 83 Yes LoD2 Vosselman and Dijkman (2001) LiDAR; Building footprints 370 Ledoux and Meijers (2011) LiDAR; Topographic map 109 Yes LoD2 No LoD1 LiDAR; Land cadaster maps; Topographic map; Digital Chen, Xue, Lu: Multi-source 3D building modeling Greece) ~2,300 No LoD2 orthophotos; Building footprints; Non-spatial data (JC3-2017, Heraklion, Rumpler et al. (2012) You and Lin (2011) Agugiaro (2014) 7 1.3 OPPORTUNITY  LiDAR and digital maps are extensively available, but  Unsatisfactory semantic/abstraction discovery  “For-profit” models (e.g., Google) are not open  Existing LoD2 methods still have some flaws  many methods only illustrated in low-density areas  Opportunity of a multi-source method Yan et al. (2016)’s space-cutting method  Object recognition from point clouds (LiDAR data)  Facilitated by topographic map  Some a priori rules about alignment (horizontal, parallel, perpendicular, etc.) as architectural knowledge  E.g., parallel to the major edges of footprints (existing in topographic map) Chen, Xue, Lu: Multi-source 3D building modeling (JC3-2017, Heraklion, Greece) LiDAR: 1 p/m2 16 p/m2 40p/m2 Alexander et al. (2009)’s aspect regulation 8 2. A PILOT OF A MULTI-SOURCE MODELING METHOD IN HK 9 Chen, Xue, Lu: Multi-source 3D building modeling (JC3-2017, Heraklion, Greece) 2.1 THE PILOT PROBLEM  The pilot area  A 4 km2 square area in Central & Western District, Hong Kong  1,819 prisms (1,361 blocks) of densely distributed buildings The pilot area (Source: OpenStreetMap)  Three inputs  Point clouds from LiDAR (4 p/m2 , format: .las)  Building footprints & heights from topographic map (format: .json)  A list of architectural knowledge about rooftop elements from Airborne LiDAR data (.las format) literature reviews (hard coded rules with parameters)  One output  All building models in LoD2 in the area (format: .dae & .json) Chen, Xue, Lu: Multi-source 3D building modeling (JC3-2017, Heraklion, Greece) The ib1000 topographic 10 map & data layers (ArcGIS format) 2.2 A CLOSE LOOK  LoD1 building models can be quickly generated in ArcGIS based on the topographic map  Extrusion of footprint  LiDAR data contains geometric measurements of LoD1 models in ArcGIS rooftop elements  It fits well onto the map (both in HK1980 grid system)  The topographic map can export all necessary data in GeoJSON  Easy for in-house program development  Knowledge-based rectification of rooftop elements  Must be in parallel or perpendicular (0°, 90°, 180°, 270°) LiDAR point clouds are able to bring rooftop elements with one or more directions of major edges of footprint Chen, Xue, Lu: Multi-source 3D building modeling (JC3-2017, Heraklion, Greece) 11 Extracting information from the map 2.3 THE PROCESS (STEPS 1~2)  Step 1: Filter the points of rooftop of each building with its footprint and roof level  Reduce the size of point cloud from ~107 (urban-scale) to ~103 (building-scale)  Step 2: Cluster the directions of the longest edges in footprint  Find one, or more if there are more clusters, direction for each building An example at HKU campus (Source: Google) Step 1: Filtering LiDAR points, computing normal Chen, Xue, Lu: Multi-source 3D building modeling (JC3-2017, Heraklion, Greece) Step 2: Clustering guiding directions from footprints (in dashed lines) 12 2.3 THE PROCESS (STEPS 3~5)  Step 3: Planar primitives discovery by RANSAC  May contains a few unrealistic shapes  Step 4: Rectification of the primitives with respect to the guiding directions of each building (summarized from observations)  If there are more than one guiding directions, the closest one is used  Step 5: Create 3D rooftop elements from those are close to  (i) guiding directions and (ii) the horizontal plane for each building An example building from Step 1,2 Step 3: RANSAC Step 4: Rectification (Schnabel et al., 2007) Chen, Xue, Lu: Multi-source 3D building modeling (JC3-2017, Heraklion, Greece) Step 5: Create 3D rooftop elements Reference image (Source:13 Google) 2.4 RESULT OF THE PILOT TEST  Pilot test  The 1,361 blocks of buildings in 4 km2  By an in-house plugin for CloudCompare 2.9 (single thread)  (Available soon)  On a notebook computer, i7 2.6GHz mobile CPU Result of the example area of HKU (dark=LoD1; light=rooftop in LoD2; photo source: dragageshk.com)  Time cost  Clustered guiding directions of all buildings: < 1.0 s  Generated 1,114 out of 1,361 building models: 319.7 s  Manual processing time: 0.0 s (geo-referencing by aircraft’s GPS)  Quality  Information richness: LoD2 in general The area in Google Map/Earth  Geometric error (RMSE): 0.06~0.20m for segmented points  Acceptable, since the accuracy of LiDAR is 0.30 m (horizontal) and 0.10 m (vertical) Chen, Xue, Lu: Multi-source 3D building modeling (JC3-2017, Heraklion, Greece) 14 3. DISCUSSION & FUTURE RESEARCH 15 Chen, Xue, Lu: Multi-source 3D building modeling (JC3-2017, Heraklion, Greece) 3.1 DISCUSSION  Pros:  The proposed method is efficient  ~0.3 s per building, linear increasing overall time when scaling up to a district or a city  Should be attributing to the efficiency of the nested RANSAC  The error is acceptable  Because odd shapes were dropped by the rules and the guiding directions  Cons:  Limited to planar primitives temporarily, not fitting precisely for curve shaped roofs  The airborne LiDAR contains little data of facades in high-density areas  There can be more inputs to fuse, e.g., point clouds from synthetic aperture radar (SAR), freely online photos, SLAM prototype models, etc.  Limitation: The pilot result is only visually tested yet Chen, Xue, Lu: Multi-source 3D building modeling (JC3-2017, Heraklion, Greece) 16 3.2 FUTURE RESEARCH  Advancing LoD2 models to LoD3 with dense point clouds  E.g., unmanned aerial vehicles (UAVs), Simultaneous localization and mapping (SLAM)  With more architectural/construction knowledge to explore  Semantic enrichment for applications, e.g.  Thermal properties  energy saving  A/C units, satellite dish, pipes, and green  facility management Regression / Rules / Supervised learning Albedo (reflectance) map, green roof map, etc. Chen, Xue, Lu: Multi-source 3D building modeling (JC3-2017, Heraklion, Greece) 17 REFERENCES  Alexander, C., Smith-Voysey, S., Jarvis, C., and Tansey, K. (2009). Integrating building footprints and LiDAR elevation data to classify roof structures and visualise buildings. Computers, Environment and Urban Systems, 33(4), 285-292.  Kolbe, T. H., Gröger, G., and Plümer, L. (2005). CityGML: Interoperable access to 3D city models. In: P. van Oosterom, S. Zlatanova, E. M. Fendel, ed., Geo-information for Disaster Management. Springer Berlin Heidelberg, pp. 883–899, doi: 10.1007/3-540-27468-  Schnabel, R., Wahl, R., and Klein, R. (2007). Efficient RANSAC for point‐cloud shape detection. Comput. Graph. Forum, 26(2), pp. 214–226, doi: 10.1111/j.1467-8659.2007.01016.x, 214-226.  Volk, R., Stengel, J., and Schultmann, F. (2014). Building Information Modeling (BIM) for existing buildings— Literature review and future needs. Automation in Construction, 38: 109-127.  Xue, F., Chen, K., Liu, D., Niu, Y., and Lu, W. (2016). An optimization-based semantic building model generation method with a pilot case of a demolished construction. In International Conference on Advancement of Construction Management & Real Estate, CRIOCM 2016. (proceedings in press)  Yan, J., Jiang, W., and Shan, J. (2016). A global solution to topological reconstruction of building roof models from airborne LiDAR point clouds. ISPRS Annals of Photogrammetry, Remote Sensing and Spatial Information Sciences, 379-386. 18 Chen, Xue, Lu: Multi-source 3D building modeling (JC3-2017, Heraklion, Greece) LEAN & COMPUTING IN CONSTRUCTION CONGRESS THANK YOU!