Urban Semantics in BIM & GIS From source to sea 15 October 2018 Xi’an Jiaotong-Liverpool University Dr. Frank Xue Research Assistant Professor Dept. of REC, HKU HKURBANlab—iLab, HKU Outline 1 ‘Source’: Semantics focused in BIM & GIS 2 One ‘stream’: Semantic registration 3 ‘Sea’: For smarter prospects F. Xue: Semantics in BIM & GIS 2 Section 1 ‘SOURCE’ SEMANTICS FOCUSED IN BIM & GIS F. Xue: Semantics in BIM & GIS 3 1.1 Background  Global urbanization  By 2050, 65% world’s population will live in cities (WHO, 2015)  Irreversible; Even faster in China  Leads to urban vulnerability (a.k.a. ‘urban diseases’)  ‘Dead’ space/landscape, low familiarity with surroundings,  Poor waste treatment, environment (air, water) pollution, China’s and global urbanization rates source: gov.cn  Heritage destruction, aging town blocks, inefficient traffic,  Disasters (earthquake, climate change), resource crisis, …  Demands smarter and more resilient development  (a) Smarter decision supports in multiple disciplines  (b) On basis of accurate, timely urban semantics F. Xue: Semantics in BIM & GIS Global urban vulnerability level (Birkmann et al, 2016) source: nature.com 4 1.1 Urban semantics  Information is the meaning of data (Rowley & Hartley, 2017)  Abstracted, inferred from data  Answering interrogative questions (what, who, where, when)  Semantics is a subset of information (Floridi, 2005) Data: Digital pixels (0~255 R, G, B) Information: Car, building, tree, …  Urban semantics  Geometric / Non-geometric facts: o Size, location / function, materials, history, etc.  Instructions (how-to): o Manufacturing, installation, access  Urban semantics databases  BIM & GIS Information map (Floridi, 2005) F. Xue: Semantics in BIM & GIS 5 1.2 BIM  BIM (building information model/modeling) “M” “I”  A digital representation of physical & functional characteristics of a facility. (NIBS, 2015) “B”  A shared … resource for information about a facility, forming a reliable basis for decisions during its life cycle from inception onward. (NIBS, 2015) An evolution view of CAD/BIM (Penttilä, 2007)  Evolved from CAD (computer-aided design) (Penttilä, 2007)  Essence  Urban semantics database  Component (unit facility) based o “Family” and “instance”  A quiz: Which is not BIM? F. Xue: Semantics in BIM & GIS A B C D E F 6 1.2 Semantics focused in BIM  BIM LOD (Level of Development)  Previously “Level of Detail” Arch.  LOD 100：For concept presentation Eng.  LOD 200：For design development  LOD 300：For 2D documentation Const. O&M Demo. o LOD 350 construction 3D documents  LOD 400：For construction stage  LOD 500：For facilities management  Focused semantics What is called LOD Source: PracticalBIM.net  Temporal development  Abstracted family > exact geometry No LOD : Data only F. Xue: Semantics in BIM & GIS data, needs processing to info., then to BIM 7 1.3 GIS  GIS (geographic information system)  A computer system for capturing, storing, checking, and displaying data related to positions on Earth’s surface (NGS, 2012) “I” “G”  Evolved from DBMS (database management system)  Essence  Urban semantics database  Data tables (layers) based o Independent objects (rows) in each table GIS interpretation Source: US Government Accountability Office o A few discussion on component-based GIS, too  A quiz: Which is not GIS? F. Xue: Semantics in BIM & GIS A B C D E F 8 1.3 Semantics focused in GIS  GIS LOD (Level of Detailing)  Defined in CityGML (by Open GIS Consortium)  LOD0 : Region and landscape  LOD1 : + Prismatic buildings model (flat roof)  LOD2 : + Roof and thematic surfaces  LOD3 : + Detailed exterior (wall and roof)  LOD4 : + Interior (indoor) GIS Level of Detailing (Gröger et al., 2007)  Focused semantics  Spatial details  Exact geometry > abstracted concepts  BIM/GIS integration  In both academia and industry F. Xue: Semantics in BIM & GIS BIM-GIS integration Source: ESRI.com 9 Section 2 ONE ‘STREAM’ SEMANTIC REGISTRATION F. Xue: Semantics in BIM & GIS 10 2.1 The task of semantic registration  A dilemma of urban semantics in BIM/GIS  Inadequacy: Poor semantics in the models  Overload: Rich online open BIM/GIS resources o With fact-nongeometric & instructional  Semantic registration Google Map/Earth? ~LOD2 BIMobject.com has >300,000 parametric BIM objects  Registering semantics to low LOD models  Input 1: Geometric measurement  Input 2: Semantic components  Performance metrics Semantic registration  Computational time  Precision = true positive / registered  Recall = true positive / actual F. Xue: Semantics in BIM & GIS Semantic registration as a process 11 2.2 A derivative-free optimization (DFO) approach  Semantic registration is a decision task  Can be automated through optimization  Problem formulation (Xue et al., 2018a)  Input 1: E.g., 3D point cloud  Variables (X): transformation parameters DFO-based Semantic registration (Xue et al., 2018a; 2018b)  Objective function (f): minimum geometric error  Constraints (C): Topological regularity  DFO algorithms (Conn et al., 2009)  Solves problems comprising too complex derivatives  Succeeded in many science and engineering problems o E.g., Protein folding (Nicosia & Stracquadanio, 2008), aircraft wing design (Lee, et al., 2008) F. Xue: Semantics in BIM & GIS Comparison of DFO algorithms for BBOB-2009 (Auger et al., 2010) Source: Inria 12 2.3.1 An outdoor case (Xue et al., 2018a)  2D photo + free BIM objects  LOD3/300 3D models  Time: 2.5h  Automatic, error tolerant, recoverable from wrong objects  Precision: 0.92  Segmentation-free, topological relationships involved  Recall: 0.92 (a) A photo of a demolished building (c) Approximate building mode Door portico Tree × 2 Wall × 2 Windows × 2 (b) Semantic components from web F. Xue: Semantics in BIM & GIS (d) Semantic/topological links (Language: C++, Ruby; Data formats: SketchUp, Bmp, Google earth) 13 2.3.2 An indoor case (Xue et al., 2018b)  Point cloud + BIM objects  LOD 4/500 indoor model  Time: 6.44s  Automatic, saved 98% time from manual modeling  Precision: 1.0  RMSE = 3.87cm, equal to experienced modelers  Recall: 1.0 F. Xue: Semantics in BIM & GIS (Language: C++, CLR; Data formats: Autodesk Revit, Stanford polygon) 14 2.4 Other semantics ‘streams’ at iLab  LOD1 + point cloud = LOD2 (Chen et al., 2018)  Non-geometric semantic learning  Semantic discovery (Xue et al., 2018c)  Real-time motions, behavior (Niu et al., 2018) a  BIM localization (Wang et al., 2018) F. Xue: Semantics in BIM & GIS b c  Smart facility (Xu et al., 2018)  Smart walkability (Xue et al., 2018d) 15 Section 3 ‘SEA’ FOR SMARTER PROSPECTS F. Xue: Semantics in BIM & GIS 16 3.2 Target semantics, e.g., building facts (Chen et al., 2018b) Construction Operation & maintenance (O&M) F. Xue: Semantics in BIM & GIS Geometric • Site information (coordinate’s data and layout) • Building spaces (floor, zones, rooms, openings) • Utility lines • Dimension of building components • Building services (location, relationship) • Building spaces (floor, zones, rooms, openings) • Utility lines • Specification of exterior enclosure products • Furnishing Non-geometric • Construction materials (status, quality, category, manufacturer) • Precast elements (quality, category, manufacturer) • Equipment attributes (ID, type, status) • Financial data • Location of labor, materials, and machine • Project performance data • Construction schedule • Construction activity status • Site environment • Building services (identification number, manufacturer) • Status of mechanical, electrical, & plumbing equipment • Maintenance record • Indoor environment • Attributes of replaced components • Maintenance status • Maintenance schedule • Operation records 17 3.2 List of semantics standards (Wang et al., 2018) Research/ industry Pratt (2004) Belsky et al. (2016) Application Scenario BIM object contents exchange Semantic enrichment for BIM objects Parametric BIM object modelling Chen and Wu (2013) Open Geospatial Consortium (OGC, 2007) Object Parameters Functional type; Geometry; Attributes; Relations between objects; Behavioural rules. Function; Geometry; Material; Identity; Aggregation relationships; Composition relationships. Basic Object Data (Identification, Classification, Geometry, Quantities, and Phasing); Representation data (Material) Object data description in CityGML for virtual 3D city and landscape Geometrical, Topological, Semantic, and Appearance properties. Modelling and professional analysis (e.g. thermal) RIBA, UK (2014) Object data description defined in NBS BIM Object Standard Identification (number, name, type, description); Classification (OmniClass code and description); Geometry; Material; Quantities; Manufacturer; Cost; Phasing; LEED, Thermal and Structural Properties, etc. Authorship, Identification (name, Uniclass code, and product link), Manufacturer, NBS description, and reference, etc. NIBS, USA (2012) Information Collection via Cobie to improve handover to owner-operator Authorship, Identification (created by, category, Description, type, code, etc.) Manufacturer, Warranty, Geometry, Material CIBSE, UK (2016) Product description for manufacturer defined in Product Data Templates(PDTs) Manufacturer, Construction, Application, Dimension, Performance, Electrical, Controls, Sustainability, Operations and Maintenance Autodesk Revit (2017) F. Xue: Semantics in BIM & GIS 18 3.3 Urban semantics technologies for smarter city Year GIS (spatial) BIM  BIM, GIS, CIM, Robotics/CV  Complementary and overlapping  On the same urban objects (temporal) o With focused semantics Second Day  Integrated urban semantics is then CIM  Digital twin of the built environment  Recognizable by machines  For smarter city applications Robotics/CV(real-time) Comp. Room Building  4D, nD, temporal (building): BIM Area/city The spatial-temporal matrix of the interests of BIM, GIS, CIM. CV F. Xue: Semantics in BIM & GIS  4D, nD, temporal (area): CIM  Spatial analysis (area): GIS  Real-time control: Robotics/CV 19 References                        Auger, A., Finck, S., Hansen, N., and Ros, R. (2010). BBOB 2009: Comparison tables of all algorithms on all noisy functions, INRIA. Birkmann, J., Welle, T., Solecki, W., Lwasa, S., & Garschagen, M. (2016). Boost resilience of small and mid-sized cities. Nature News, 537(7622), 605. Chen, K., Lu, W., Xue, F., Tang, P., & Li, L. H. (2018a). Automatic building information model reconstruction in high-density urban areas: Augmenting multi-source data with architectural knowledge. Automation in Construction, 93, 22-34. Chen, K., Lu, W. S. ., Xue, F, Zheng, L. Z., & Liu, D. D. (2018b). Smart Gateway for Bridging BIM and Building. In Proceedings of the 21st International Symposium on Advancement of Construction Management and Real Estate (pp. 1307-1316). Springer, Singapore. Conn, A. R., Scheinberg, K., & Vicente, L. N. (2009). Introduction to derivative-free optimization. SIAM. Harty, C., Goodier, C. I., Soetanto, R., Austin, S., Dainty, A. R., & Price, A. D. (2007). The futures of construction: a critical review of construction future studies. Construction Management and Economics, 25(5), 477-493. Gröger, G., Kolbe, T., & Czerwinski, A. (2007). Candidate OpenGIS CityGML Implementation Specification (City Geography Markup Language). Open Geospatial Consortium Inc, OGC. Lee, D. S., Gonzalez, L. F., Periaux, J. & Srinivas, K. (2008), Robust design optimisation using multi-objective evolutionary algorithms. Computers & Fluids, 37(5), 565-583. National Institute of Building Sciences. (2015). National Building Information Modeling Standard. Version 3, Retrieved from https://www.nationalbimstandard.org/ National Geographic Society. (2012). GIS (geographic information system). http://www.nationalgeographic.org/encyclopedia/geographic-information-system-gis/ Nicosia, G. & Stracquadanio, G. (2008), Generalized pattern search algorithm for peptide structure prediction. Biophysical Journal, 95(10), 4988-4999. Niu, Y., Lu, W., Xue, F., Liu D., Chen, K., Fang, D., & Anumba, C. (2018). Towards the “Third Wave”: An SCO-enabled occupational health and safety management system for construction. Safety Science. 111, 213-223. Penttilä, H. (2007). Early architectural design and BIM. Computer-Aided Architectural Design Futures (CAADFutures 2007), 291-302. Rowley, J. (2007). The wisdom hierarchy: representations of the DIKW hierarchy. Journal of information science, 33(2), 163-180. Rowley, J., & Hartley, R. (2017). Organizing knowledge: an introduction to managing access to information. Routledge. Wang, J., Lu, W., Liu, D., & Xue, F. (2018). The Heterogeneity of BIM Objects in Different Construction Contexts. In Construction Research Congress 2018 (pp. 210-220). World Health Organization. (2015). Global Report on Urban Health—Executive Summary. http://www.who.int/kobe_centre/measuring/urban-global-report/en/ Xu, J., Chen, K., Xue, F., & Lu, W. (2018). 3D point clouds for architecture, engineering, construction, and operation: A SWOT analysis. Working paper Xue, F., Lu, W., Chen, K. (2018a). Automatic generation of semantically rich as-built building information models using 2D images: A derivative-free optimization approach. Computer-Aided Civil and Infrastructure Engineering, in press. doi: 10.1111/mice.12378 Xue, F., Lu, W., Chen, K. & Zetkulic, A. (2018b). From ‘semantic segmentation’ to ‘semantic registration’: A derivative-free optimization-based approach for automatic generation of semantically rich as-built building information models (BIMs) from 3D point clouds. Journal of Computing in Civil Engineering. Under review Xue. F., Chen, K., Lu, W. (2018c). Architectural Symmetry Detection from 3D Urban Point Clouds: A Derivative-Free Optimization (DFO) Approach. CIB W78 2018, accepted. Xue, F., Chiaradia, A., Webster, C., Liu, D., Xu, J., Lu, W. (2018d). Personalized walkability assessment for pedestrian paths: An as-built BIM approach using ubiquitous augmented reality (AR) smartphone and deep transfer learning. Proceedings of the 23rd International Symposium on the Advancement of Construction Management and Real Estate. In press Zins, C. (2007). Conceptual approaches for defining data, information, and knowledge. Journal of the Association for Information Science and Technology, 58(4), 479-493. F. Xue: Semantics in BIM & GIS 20 THANK YOU ! F. Xue: Semantics in BIM & GIS 21