Personalized Walkability Assessment for Pedestrian Paths An As-built BIM Approach Using Ubiquitous Augmented Reality (AR) Smartphone and Deep Transfer Learning CRIOCM’18, Guiyang, China 22 May 2018 Anna Zetkulic on behalf of Xue, F. Chiaradia, A.J.F. Webster, C.J. Liu, D. Xu, J. & Lu, W.S. Outline 1 Introduction 2 An As-built BIM Approach 3 A Pilot Study 4 Discussion & Future Work F Xue et al.: Personalized walkability assessment, 24-27 Aug 2018, CRIOCM 2 Section 1 INTRODUCTION F Xue et al.: Personalized walkability assessment, 24-27 Aug 2018, CRIOCM 3 1.1 Smart city, personalized walkability  Smart city development  Settled by the government of many modern cities  Over 200 cities in China  Smart living/ transportation The rising of smart cities around the world Photo source: siemens.com  Aims at making life more efficient, more controllable, economical, productive, integrated and sustainable [1]  A pillar of smart city  Personalized walkability  Meeting individual walking requirements of residents Walkable?  Essential for smart living in smart cities  Demanding automatic (real-time, cheap) assessment o To handle the possible changes in paths F Xue et al.: Personalized walkability assessment, 24-27 Aug 2018, CRIOCM Personalized walkability for smart living Photo source: pixarba.com 4 1.2 Existing methods for Personalized Walkability Assessment (PWA)  Existing assessment methods suffers from (at least one)  Level of details  Automation of assessment  Personalized requirements  So, we propose an as-built BIM approach for addressing the difficulties Method Input Process by Level of detail Automation Personalized Example Observational Audits Walking Characteristics Human experts ★★★★☆ ★ ☆ [2] StreetView (e.g., Google) Human experts ★★★★☆ ★★ ☆ [3] GIS-based GPS records Computers ★★ ★★★★☆ ★★ [4] As-built BIM 3D point clouds Smartphone & computer ★★★☆ ★★★★☆ ★★★★★ – F Xue et al.: Personalized walkability assessment, 24-27 Aug 2018, CRIOCM 5 1.3 As-built BIM  BIM (building information model/modeling) BIM “M”  A digital representation of physical & functional characteristics of a facility. [5] “I” “B”  A shared … resource for information about a facility, forming a reliable basis for decisions during its life cycle from inception onward. [5]  Evolved from CAD (computer-aided design) [6] An evolution view of CAD/BIM [6]  Why BIM, not GIS? Unit element Emphasis BIM Component (3D, meanings, relations) Temporal (Records, changes) GIS Data layer (mainly shapes) Spatial (Fitting to the globe)  Semantically richer than GIS for PWA  As-built (or as-is) BIM  As-designed  as-planned  as-built  as-demolished  Actual, current (real-time) conditions for PWA F Xue et al.: Personalized walkability assessment, 24-27 Aug 2018, CRIOCM A quick comparison between BIM and GIS 6 Section 2 AN AS-BUILT BIM APPROACH F Xue et al.: Personalized walkability assessment, 24-27 Aug 2018, CRIOCM 7 2.1 The conceptual framework  A three-step “pipeline”  1. Actual path  As-is 3D point cloud  2. As-is 3D point cloud  As-built BIM  3. As-built BIM  PWA; recommendation F Xue et al.: Personalized walkability assessment, 24-27 Aug 2018, CRIOCM 8 2.2 Technical details under the hood of Step 2  Step 2: As-is 3D point cloud  As-built BIM  2.1 Deep transfer learning [7-8] o As-is 3D point cloud  Semantically segmented points o Method: Pre-trained deep learning models (PointNet)  2.2 Object modeling o Segmented points  3D objects with meanings (semantic) o Data-driven shape fitting o Method: Improved RANSAC [9]  2.3 As-built BIM creation o 3D objects  complete BIM with topology and relations o Model-driven o Method: Global optimization with constraints [10] F Xue et al.: Personalized walkability assessment, 24-27 Aug 2018, CRIOCM 9 Section 3 A PILOT STUDY AROUND HKU F Xue et al.: Personalized walkability assessment, 24-27 Aug 2018, CRIOCM 10 3.1 A street scene near HKU  A narrow path  1(a)  Guardrail  Obstacles (a) A scene of Bonham Road, Hong Kong (b) As-is cloud of 569,344 points through AR scanning  1: Phone scanning  1(b) point cloud  2: As-built BIM  2(a) segment  2(b) modeling (a) 3D point classification ( e.g., the points labeled as (b) As-built BIM consisting of semantic objects (walls “manmade terrain” were detached as the pavement) omitted in this view) F Xue et al.: Personalized walkability assessment, 24-27 Aug 2018, CRIOCM  2(b) BIM 11 3.2 Analysis for PWA  3: Assessment  3(a) geo-referencing  3(b) slope grade  3(c) tilt grade  3(d) footway width (a) Geo-referenced first-person view (c) The tilt grades of the paved footway (b) The slope grades (i.e., tan θ) of the paved footway (d) The actual widths of the pavementwith obstacles F Xue et al.: Personalized walkability assessment, 24-27 Aug 2018, CRIOCM 12 3.3 PWA results and recommendations  Examples of five types of pedestrians Walking characteristic Calculated value No. of steps 0 * 1:50.0~58.8 Slope grade 1:47.6~66.7 Tilt grade† ‡ Footway width 45~199 cm Clearance Good Overall walkability (the worst) Wheelchair ♿ OK OK OK Failed OK Failed Type of pedestrians Luggage 🛄🛄 Senior 👴👴 Stroller OK OK OK Limited OK Limited OK OK OK Limited OK Limited OK OK OK OK OK OK Exercise 🏃🏃 OK OK OK OK OK OK *: Reference maximum slope grade: 1:8~12 (wheelchairs); †: Reference maximum tilt grade of pavement: 1:15 (wheelchairs); ‡: Reference minimum width: 70~90 cm (wheelchairs), 40~70 cm (strollers), and 30~60 cm (baggage).  Recommendation on possible obstacle removal Major obstacles Light pole Minor obstacles Inoffensive obstacles (None) Meter pole, drainage pipe #1, #2, and concrete trace on the wall F Xue et al.: Personalized walkability assessment, 24-27 Aug 2018, CRIOCM 13 Section 4 DISCUSSION & FUTURE WORK F Xue et al.: Personalized walkability assessment, 24-27 Aug 2018, CRIOCM 14 4.1 Discussion  The proposed as-built BIM approach was confirmed  Can be automatic o Real-time o Inexpensive  Rich details  Personalized  Could be useful for other applications, too (see right)  Yet still preliminary in  Test data set The spatial-temporal matrix of the interests of BIM, GIS, CIM. CV  Deep learning model  3D object fine-tuning for better BIM  Completion and automation in PWA analysis F Xue et al.: Personalized walkability assessment, 24-27 Aug 2018, CRIOCM 15 4.2 On-going and future work  Semantic prioritization  Identifying available urban semantics  Confirming most demanded semantics  Data-driven 3D object modeling  Geometric regularity, e.g., symmetry  Interactive machine learning  Model-driven as-built BIM creation  New semantic registration methods  More adaptive objects  Everyday smartphone APP  To make an impact F Xue et al.: Personalized walkability assessment, 24-27 Aug 2018, CRIOCM 16 References           [1] European Commission (2014). Smart living. https://ec.europa.eu/docsroom/documents/13407/attachments/2/translations/en/renditions/native [2] Sun, G., Webster, C., and Chiaradia, A. (2017). Objective assessment of station approach routes: Development and reliability of an audit for walking environments around metro stations in China. Journal of Transport & Health, 4: 191-207. [3] Griew, P., Hillsdon, M., Foster, C., Coombes, E., Jones, A., and Wilkinson, P. (2013). Developing and testing a street audit tool using Google Street View to measure environmental supportiveness for physical activity. International Journal of Behavioral Nutrition and Physical Activity, 10(1): 103 [4] Carlson, J. A., Saelens, B. E., Kerr, J., Schipperijn, J., Conway, T. L., Frank, L. D., Chapman, J. E., Glanz, K., Cain, K. L., and Sallis, J. F. (2015). Association between neighborhood walkability and GPS-measured walking, bicycling and vehicle time in adolescents. Health & Place, 32: 1-7. [5] National Institute of Building Sciences. (2015). National Building Information Modeling Standard. Version 3, Retrieved from https://www.nationalbimstandard.org/ [6] Penttilä, H. (2007). Early architectural design and BIM. Computer-Aided Architectural Design Futures (CAADFutures 2007), 291-302. [7] Qi, C. R., Su, H., Mo, K., and Guibas, L. J. (2017). Pointnet: Deep learning on point sets for 3D classification and segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition: 652-660. doi:10.1109/CVPR.2017.16 [8] Hackel, T., Savinov, N., Ladicky, L., Wegner, J. D., Schindler, K., and Pollefeys, M. (2017). Semantic3D.Net: A new large-scale point cloud classification benchmark. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2017(IV-1-W1): 91-98. [9] Chen, K., Lu, W., Xue, F., Tang, P., and Li, L. H. (2018). Automatic building information model reconstruction in high-density urban areas: Augmenting multi-source data with architectural knowledge. Automation in Construction, 93: 22-34. [10] Xue, F., Lu, W., and Chen, K. (2018). Automatic generation of semantically rich as-built building information models using 2D images: A derivative-free optimization approach. Computer-Aided Civil and Infrastructure Engineering, in press, doi:10.1111/mice.12378. Chen, K., Lu, W., Xue, F., Tang, P., & Li, L. H. (2018a). Automatic building information model reconstruction in high-density urban areas: Augmenting multi-source data with architectural knowledge. Automation in Construction, 93, 22-34.  Chen, K., Lu, W. S. ., Xue, F, Zheng, L. Z., & Liu, D. D. (2018b). Smart Gateway for Bridging BIM and Building. In Proceedings of the 21st International Symposium on Advancement of Construction Management and Real Estate (pp. 1307-1316). Springer, Singapore.  Harty, C., Goodier, C. I., Soetanto, R., Austin, S., Dainty, A. R., & Price, A. D. (2007). The futures of construction: a critical review of construction future studies. Construction Management and Economics, 25(5), 477-493.  Gröger, G., Kolbe, T., & Czerwinski, A. (2007). Candidate OpenGIS CityGML Implementation Specification (City Geography Markup Language). Open Geospatial Consortium Inc, OGC.  National Geographic Society. (2012). GIS (geographic information system). http://www.nationalgeographic.org/encyclopedia/geographic-information-system-gis/  Niu, Y., Lu, W., Xue, F., Liu D., Chen, K., Fang, D., & Anumba, C. (2018). Towards the “Third Wave”: An SCO-enabled occupational health and safety management system for construction. Safety Science. Under review F Xue et Personalized walkability assessment, 24-27 Aug 2018 , CRIOCM 17  al.: Rowley, J. (2007). The wisdom hierarchy: representations of the DIKW hierarchy. Journal of information science, 33(2), 163-180.  Thank You ! 谢 谢！