Semantic Enrichment for BIM & GIS A Computational Perspective 计算视角下的 BIM 及 GIS 语义充实研究 Coll. of Civil Eng., SZU 深圳大学 土木工程学院 22 May 2018 Frank Xue 薛帆 Research Assistant Professor 助理教授（研究） Dept. of REC, HKU 香港大学 房地产及建设系 iLab, HKURBANlab, HKU 香港大学 HKURBANlab – iLab Aim and scope  Aim of this presentation 目的  To introduce the HKURBANlab – iLab 自报家门  To revisit the concepts about information 审视基础概念  To discuss a novel research topic 探讨一个新课题  To share several recent studies 分享若干新进展  To engage critiques and debates 希批评指正  To promote collaborations (and citations) 促进合作  Scope 范畴  Extension: Urban information databases (BIM, GIS) 外延：城市信息库  Intention: Semantic enrichment 内涵：语义充实  Methods: Computational methods 方法：计算方法 F Xue: Semantic enrichment for BIM & GIS, 22 May 2018, SZU 2 Outline 简介 1 Introduction to iLab 背景 2 Background & Opportunities 正文 3 Semantic Enrichment for BIM & GIS 讨论 4 Discussion & Future Work F Xue: Semantic enrichment for BIM & GIS, 22 May 2018, SZU 3 Section 1 INTRODUCTION TO ILAB ILAB介绍 — “一波强行植入” F Xue: Semantic enrichment for BIM & GIS, 22 May 2018, SZU 4 1.1 HKURBANlab  Faculty of Architecture, HKU 建筑学院  3 Departments: Arch., REC, DUPAD  2 Divisions: Landscape Arch., Arch. Conservation  HKURBANlab 实验中心  Newly branded research arm of FoA  1 Academician (CAS), 10 full professors  12 labs on o Urban planning; Property rights; o Chinese architecture; Rural; Sustainability; Conservation; Virtual Reality; … o Health; o Fabrication and materials; o iLab (data and information); F Xue: Semantic enrichment for BIM & GIS, 22 May 2018, SZU www.arch.hku.hk 5 1.1 iLab: The urban big data hub  iLab 实验室  Urban big data hub  multi-dimensional and multi-disciplinary urban big data collection, storage, analysis, and presentation to inform decisionmaking in urban development iLabHKU fac.arch.hku.hk/iLab  Focusing on information technology (IT) o Geographical Information Systems (GIS) o Global Positioning Systems (GPS) o Urban Remote Sensing (URS) o Building Information Model (BIM) o Internet of Things (IoT) o virtual design and construction (VDC) o integrated project delivery (IPD) F Xue: Semantic enrichment for BIM & GIS, 22 May 2018, SZU 6 1.2 The research team  Lab director 主任  Dr. Wilson Lu  Full-time team members 成员  1 RAP, 1 PostDoc, 1 SRA, 3 RAs, 7 PhD candidates  Research themes 主题  Urban big data (BIM, GIS, IoT, …)  Construction project management  Construction waste management Lunch-time gathering  International construction o Corporate social responsibility F Xue: Semantic enrichment for BIM & GIS, 22 May 2018, SZU 7 1.3 About myself  A mixed background 背景  Engineering  BEng in Automation 2004  MSc in Computer Science 2007  Computer Science  PhD in System Engineering 2012  AI, DFO, ML  PostDoc in Construction Management 2016  Research interests 兴趣  ISE, CEM, EIE  Economics  SCM  Computation and urban semantics in BIM  Applied operations research  Machine learning and visualization for construction  On-going research projects 在研  PI: RGC (17201717), HKU (201702159013, 201711159016)  Co-I: NSFC (71671156), NSSFC (17ZDA062), HKU PTF F Xue: Semantic enrichment for BIM & GIS, 22 May 2018, SZU 8 1.4 Job vacancies at iLab, HKU  Research Assistant (2~4 openings)  $16,575/month  PostDoc(~1 opening) 博后  ~$30,000/month  PhD (2~3/year) 博士  100% funded scholarship o $16,330/month  HKU PhD Fellowship (UPF) o Above + $70,000 (one time) Empty seats for you  HK PhD Fellowship Scheme (HKPF) o $20,000/month + $10,000/year conference + $42,100 (annual fee of 1st year)  Inquiry: Dr. Wilson Lu F Xue: Semantic enrichment for BIM & GIS, 22 May 2018, SZU 9 Section 2 BACKGROUND & OPPORTUNITIES 背景和机遇 — “时势造英雄” F Xue: Semantic enrichment for BIM & GIS, 22 May 2018, SZU 10 2.1 BIM, GIS, and humanity’s future  Global urbanization 全球城市化  By 2050, 65% world’s population will live in cities (WHO, 2015)  Irreversible; Even faster in China  Leads to urban vulnerability (a.k.a. ‘urban diseases’)  Poor resource (water, power) management, inefficient traffic,  Poor waste treatment, environment (air, water) pollution, China’s and global urbanization rates source: gov.cn 国家新型城镇化规划（2014－2020年）  Disasters (earthquake, storm, climate change),  Heritage destruction, …  For the future of humanity 为了明天  Smart, sustainable, and resilient city development o On decision support platforms like BIM & GIS F Xue: Semantic enrichment for BIM & GIS, 22 May 2018, SZU Global urban vulnerability level (Birkmann et al, 2016) source: nature.com 11 2.1 BIM, GIS, and the construction industry  Construction is known as a “backward industry” 现状  Low productivity, labor-intensive (v.s. aging workers)  Fatality, occupational hazards, management (e.g., cost overrun)  Meets new information communication tech. (ICT) 机遇  To fuse as urban big data o BIM, RFID, LiDAR, GPS, UAV, CV, VR/AR, smart phones…  To extract urban semantic information  Is now adopting 为了今日  BIM and GIS models  For effective (productive, automatic, age friendly) and efficient USA’s gross value-added by sectors source: economist.com Efficiency eludes the construction industry (safer, profitable, on-time, sustainable) AECO industry  A consensus of global research institutes (e.g., Harty et al., 2007) F Xue: Semantic enrichment for BIM & GIS, 22 May 2018, SZU Recent advances in ICT 12 2.2 Concepts – BIM  BIM (building information model/modeling) 概念 “M” “I”  A digital representation of physical & functional characteristics of a facility. (NIBS, 2015) “B”  A shared … resource for information about a facility, forming a reliable basis for decisions during its life cycle from inception onward. (NIBS, 2015)  Evolved from CAD (computer-aided design) (Penttilä, 2007) An evolution view of CAD/BIM (Penttilä, 2007)  Essence 本质  Urban information database  Component (unit facility) based  A quiz: BIM or not? 练习  How to measure the info.? F Xue: Semantic enrichment for BIM & GIS, 22 May 2018, SZU A B C D E F 13 2.2 Level of Development (LOD) of BIM  BIM LOD 发展指数  Previously “Level of Detail”  Information metric by temporal stages  Levels* 分级 Arch.  LOD 100：For concept presentation  LOD 200：For design development Eng. Const.  LOD 300：For 2D documentation o LOD 350 construction 3D documents  LOD 400：For construction stage What is called LOD Source: PracticalBIM.net O&M  LOD 500：For facilities management  * Still not accepted universally Demo. F Xue: Semantic enrichment for BIM & GIS, 22 May 2018, SZU No LOD data, needs processing to info., then to BIM 14 2.3 Concepts – GIS  GIS (geographic information system) 概念  A computer system for capturing, storing, checking, and displaying data related to positions on Earth’s surface (NGS, 2012) “I” “G”  Evolved from DBMS (database management system)  Essence 本质  Urban information database  Data tables (layer) based  A quiz: GIS or not? 练习 GIS interpretation Source: US Government Accountability Office A F Xue: Semantic enrichment for BIM & GIS, 22 May 2018, SZU B C D E F 15 2.3 Level of Detailing (LOD) of GIS  GIS LOD 细节指数  Defined in CityGML (by open GIS consortium)  Information metric by spatial details  Levels* 分级  LOD0 : Region and landscape  LOD1 : + Prismatic buildings model (flat roof)  LOD2 : + Roof and thematic surfaces GIS Level of Detailing (Gröger et al., 2007)  LOD3 : + Detailed exterior (wall and roof)  LOD4 : + Interior (indoor)  * Still not accepted universally, neither  But, what is information after all, behind these metrics? F Xue: Semantic enrichment for BIM & GIS, 22 May 2018, SZU Google Map/Earth? ~LOD2 16 2.4 Concepts – information 信息  Information 概念 Why  Many definitions in different fields (e.g., philosophy; comm.)  The DIKW pyramid 相关概念 How  Data is sensory stimuli (Rowley, 2007) (or signals; Zins, 2007)  Information is the description, meaning of data (Rowley & Hartley, 2017) o Abstracted, inferred from data o Answering interrogative questions (what, who, where, when) o For supporting decision-making The DIKW pyramid What Who When As-is Where Howmany  Knowledge is processed, organized or structured information o Reasonable  Wisdom* is evaluated understanding of knowledge o Shared, for future F Xue: Semantic enrichment for BIM & GIS, 22 May 2018, SZU The DIKW flow Source: Wikipedia 17 2.4 Concepts – semantics 语义  Semantics 概念  Origin in linguistics and philosophy, study of meaning o Opposed to syntax (e.g., SVO)  A subset of information (Floridi, 2005) o Including facts and instructions (how-to) o Opposed to environmental  In BIM/GIS, an object has (Xue et al, 2018) 分类  Individual’s semantics o Geometric: E.g., shape, size, position, texture Information map (Floridi, 2005) for (int i = 0; i < 10; ++i) { sum += i; … } o Non-geometric: E.g., type, materials, function, assembly order  Relational semantics o E.g., dependency, topology, joints Over a box of building F Xue: Semantic enrichment for BIM & GIS, 22 May 2018, SZU On a plane of road 18 2.4 List of semantics of buildings (Chen et al., 2018b) Geometric Construction Site information 建设 (coordinate’s data and layout) Building spaces (floor, zones, rooms, openings) Utility lines Dimension of building components Operation & maintenance (O&M) 营运 Building services (location, relationship) Building spaces (floor, zones, rooms, openings) Utility lines Specification of exterior enclosure products Furnishing Non-geometric  Required Construction materials (status, quality, category, semantics 对比 manufacturer) Precast elements (quality, category, manufacturer)  So many Equipment attributes (ID, type, status) Financial data  Both geometric Location of labor, materials, and machine and nonProject performance data geometric Construction schedule Construction activity status  Changing over Site environment time Building services (identification number, manufacturer) Status of mechanical, electrical, & plumbing equipment Maintenance record Indoor environment Attributes of replaced components Maintenance status Maintenance schedule Operation records F Xue: Semantic enrichment for BIM & GIS, 22 May 2018, SZU 19 2.4 List of semantics standards (Wang et al., 2018) Research/ industry Pratt (2004) BIM object contents exchange Belsky et al. (2016) Semantic enrichment for BIM objects Parametric BIM object modelling Chen and Wu (2013) GIS Application Scenario Open Geospatial Consortium (OGC, 2007) Autodesk Revit (2017) RIBA, UK (2014) Object data description in CityGML for virtual 3D city and landscape Modelling and professional analysis (e.g. thermal) Object data description defined in NBS BIM Object Standard Information Collection via Cobie to improve handover to owneroperator Product description for CIBSE, UK (2016) manufacturer defined in Product Data Templates(PDTs) F Xue: Semantic enrichment for BIM & GIS, 22 May 2018, SZU NIBS, USA (2012) Object Parameters Functional type; Geometry; Attributes; Relations between objects; Behavioural rules. Function; Geometry; Material; Identity; Aggregation relationships; Composition relationships. Basic Object Data (Identification, Classification, Geometry, Quantities, and Phasing); Representation data (Material) Geometrical, Topological, Semantic, and Appearance properties. Identification (number, name, type, description); Classification (OmniClass code and description); Geometry; Material; Quantities; Manufacturer; Cost; Phasing; LEED, Thermal and Structural Properties, etc. Authorship, Identification (name, Uniclass code, and product link), Manufacturer, NBS description, and reference, etc. Authorship, Identification (created by, category, Description, type, code, etc.) Manufacturer, Warranty, Geometry, Material Manufacturer, Construction, Application, Dimension, Performance, Electrical, Controls, Sustainability, Operations and Maintenance  BIM vs GIS 对比  Seems that BIM community cares more than GIS  App-oriented  Implemented as BIM o Parameters of comp. GIS o Map layers (data tables) 20 2.4 Status quo of urban semantics in BIM/GIS  Lacking in semantics in general 普遍缺乏  GIS o Rich (up to LOD3/300) for iconic buildings o Poor (LOD1/100) for most buildings  BIM o Rich (up to LOD4/400) for new buildings o Poor (LOD1/100) (copy-and-paste from GIS) Hong Kong 3D map (95%: LOD1, 5%: LOD2~3) Source: LandsD  Failed to enable smart, sustainable, and resilient city apps. o Often requires LOD4/500 o Plus many semantic objects in environments E.g., walkable 3D network, for baby strollers, lifts, …  Any integration? 整合？ F Xue: Semantic enrichment for BIM & GIS, 22 May 2018, SZU Berlin buildings 3D map (>95%: LOD1, <5%: LOD2) Source: osmbuildings.org 21 2.4 Integrating BIM and GIS with urban semantics Year GIS (spatial) BIM  BIM, GIS, CIM, Robotics/CV  Complementary and overlapping  On the same urban objects (temporal) o With emphasized semantic info. Second Day  Integration is feasible CIM  Via urban objects (BIM-centric)  Via locations (GIS-centric)  Enriching each other Robotics/CV(real-time)  Barriers by commercial companies (say, ESRI vs Autodesk) Comp. Room Building Area/city  Is semantic enrichment possible? The spatial-temporal matrix of the interests of BIM, GIS, CIM. CV F Xue: Semantic enrichment for BIM & GIS, 22 May 2018, SZU 22 2.4 Semantic enrichment: Potentials (Chen et al., 2018a)  Shared urban semantics in GIS, BIM, RS, AR, etc. F Xue: Semantic enrichment for BIM & GIS, 22 May 2018, SZU 23 2.4 Semantic enrichment: Data & parts (Xu et al., 2018)  Increasingly available data and components 数据越来越多 SAR (Synthetic Aperture Radar ) Spaceborne (Xue et al., 2018a) Airborne Airborne Point Cloud Sensing LiDAR (Light Detection And Ranging)  BIM models and objects  BIMobject.com o >300,000  3DWarehouse Terrestrial Mobile Spaceborne o >3,000,000  Possible to enrich semantics 可能的  What is …? Photogrammetry Airborne  When? Mobile F Xue: Semantic enrichment for BIM & GIS, 22 May 2018, SZU  How? 24 2.5 Concepts – semantic enrichment 语义充实  An example in linguistics 语言学例子  Bob: I bet my dog would like you.  Alice: Sorry, I have a pet allergy.  另一个例子  篮球裁判说： 这是个好球  For BIM/GIS/CV, semantic enrichment 本研究中  看台观众说：  The process of adding new semantics to existing objects  篮球售货员说 o For as-designed, as-altered, as-built, as-demolished BIM (GIS) o On abstract meanings Of pixels (3D points), geometric primitives, and components 这是个好球 ：这是个好球  BIM/GIS  E.g., as-designed  as-built (LOD 3/300  LOD 4/500)  Prevail in BIM/GIS manual modeling/ automatic processing Over a box of building o A.k.a. annotation, labeling, scene understanding (+relational) o The core of modeling, in fact F Xue: Semantic enrichment for BIM & GIS, 22 May 2018, SZU On a plane of road 25 2.5 When to enrich semantics? 需求  To meet temporal requirements 为不同时间  As-built (as-is) BIM o LOD 350  400 to enable (real-time) CEM (smart const.) o LOD 400  500 to enable O&M (disaster, smart city)  To meet spatial requirements 为不同空间  3D GIS o LOD3 for 3D map (e.g., VR flight simulation) o LOD4 for indoor-outdoor navigation  To meet multi-disciplinary requirements 为不同领域  Scene understanding for CV (e.g., CEM, smart city)  Smart/rational decision making for robotics (e.g., construction industrialization) F Xue: Semantic enrichment for BIM & GIS, 22 May 2018, SZU 26 2.5 How to enrich semantics? 方法  Two subclasses in computational perspective 两大类  Data-driven (scan-to-BIM): From data, processing to semantics Reference measurements  Model-driven (scan-vs-BIM): From other models, copy-and-edit  Same as automatic/semi-automatic BIM modeling Pre-trained processors Rules, examples (a) Data-driven: A computational perspective F Xue: Semantic enrichment for BIM & GIS, 22 May 2018, SZU (Some icons from Wikipedia: CC-BY; CC-BY-NC) (b) Model-driven: A computational perspective 27 2.5 Challenges 难点  Inputs 输入  Data-driven: Noise, huge amount in point clouds, uncontrolled real-world scenes  Model-driven: Availability of standard components  Processing of semantic enrichment 处理  Rule-based data-driven o Fails on complex/irregular objects  Machine learning-based data-driven o Fails without big data training examples o Fails with biased training examples  Model-driven o Computational complexity due to huge search space F Xue: Semantic enrichment for BIM & GIS, 22 May 2018, SZU 28 2.5 Opportunity of adopting DFO 新机遇  Brutal-force search is impractical  Thanks to off-the-peg derivative-free optimization (DFO) algorithms for solving such black-box problems  Surrogate methods o CMA-ES (Covariance matrix adap- tation with evolution strategy) and its variants are competitive  Trust-region methods  Metaheuristics (GA, PSO, VNS, etc.)  Hyper-heuristics, data mining  … and Monte Carlo F Xue: Semantic enrichment for BIM & GIS, 22 May 2018, SZU Comparison of algorithms for BBOB-2009 (Black-Box Optimization Benchmarking, higher is better) (Auger et al., 2010) Source: Inria 29 Section 3 SEMANTIC ENRICHMENT FOR BIM & GIS BIM和GIS的语义充实 — “热腾腾新鲜出炉” F Xue: Semantic enrichment for BIM & GIS, 22 May 2018, SZU 30 3.1 Semantic enrichment in recent papers 近期成果  List of six automatic cases 六个  LOD1/100 + LiDAR LOD2/200 o Rule-based data-driven (Chen et al., 2018a)  LOD1/100 + LiDAR LOD2+/200+ o Machine learning-based data-driven (Xue et al., 2018e)  2D photo + BIM components (LOD3/300)  LOD3/300 o DFO-based model-driven (Xue et al., 2018a)  2D photo/3D point cloud + BIM comp. (LOD4/500)  LOD4/500 As flexible, extensible as toy clay o DFO-based model-driven (Xue et al., 2018a; 2018b)  LOD4/400 + multiple real-time sensor data  real-time LOD4/400+ o Rule (automata)-based data-driven (Niu et al., 2018)  Building’s symmetry hierarchy in point clouds (Xue et al., 2018d) F Xue: Semantic enrichment for BIM & GIS, 22 May 2018, SZU 31 3.2 Case 1: For LOD2/200 (Chen et al., 2018a) 屋顶  LOD1 box models + LiDAR point cloud = LOD2 buildings  Data driven + architectural regularity (Language: C++; Data formats: COLLADA, Las, csv) F Xue: Semantic enrichment for BIM & GIS, 22 May 2018, SZU 32 3.2 Case 1: For LOD2/200 (Chen et al., 2018a) 屋顶  Step 1. RANSAC; Step 2. rectification  A “top-down” approach, tested on over 1,300 buildings (Language: C++; Data formats: COLLADA, Las, csv) F Xue: Semantic enrichment for BIM & GIS, 22 May 2018, SZU 33 3.3 Case 2: Beyond LOD2/200 (Xue et al., 2018e) 非几何  Non-geometric semantics on rooftops  Estimating albedo from Intensity, data-driven (Language: C++, R; Data formats: GeoJSON, Las, csv) F Xue: Semantic enrichment for BIM & GIS, 22 May 2018, SZU 34 3.3 Case 2: Beyond LOD2/200 (Xue et al., 2018e) 非几何  Non-geometric semantics on rooftops  A preliminary decision tree model for predicting green roofs, data-driven (Language: C++, R; Data formats: GeoJSON, Las, csv) F Xue: Semantic enrichment for BIM & GIS, 22 May 2018, SZU 35 3.4 Case 3: For LOD3/300 (Xue et al., 2018a) 外部细节  2D photo + free BIM objects  LOD3/300 models  Automatic, segmentation-free, DFO-based, model-driven  Recycling existing BIM/CAD resources (a) A photo of a demolished building (c) Approximate building mode Door portico Tree × 2 Wall × 2 Windows × 2 (b) Semantic components from web (d) Semantic/topological links (Language: C++, Ruby; Data formats: SketchUp, Bmp, Google earth) F Xue: Semantic enrichment for BIM & GIS, 22 May 2018, SZU 36 3.4 Case 3: For LOD3/300 (Xue et al., 2018a) 外部细节  Nonlinear optimization problem formulation  SSIM (input 2D photos, 3D-to-2D projection of BIM)  Constrained by topological relationships SSIM = structure ⋅ luminance ⋅ contrast = ( 2 µ ˆ µ A +c1 )( 2σ ˆ +c1 ) A AA , 2 2 2 ( µ ˆ + µ A +c1 )(σ ˆ+σ 2A +c2 ) A A C Example scaling_max (a) A photo of a demolished building Door portico Tree × 2 Example value Notes [1.5, 1.5, 1.5] xyz coordinates scaling_min [0.8, 0.8, 0.8] Ibid. CI z_rotation_max π/2 z_rotation_min 0 on_top_of ‘Ground’ Adjacency, connectivity CR contains_on Wall × 2 Windows × 2 (b) Semantic components from web ‘Wall’ min_separation ‘0.5 m’ F Xue: Semantic enrichment for BIM & GIS, 22 May 2018, SZU Containment or intersection Separation (Language: C++, Ruby; Data formats: SketchUp, Bmp, Google earth) 37 3.4 Case 3: For LOD3/300 (Xue et al., 2018a) 外部细节  Problem solving  Fully-automatic, DFO-based, model-driven  Rich semantics: Geometry, topology, functions, materials  Occasional errors in recognition F Xue: Semantic enrichment for BIM & GIS, 22 May 2018, SZU (Language: C++, Ruby; Data formats: SketchUp, Bmp, Google earth) 38 3.5 Case 4: For LOD4/500 (Xue et al., 2018a; 2018b) 室内  PCD/2D photos + BIM objects  Indoor (for LOD 4/500)  Automatic  Model-driven  Semantic  Accurate  Efficient  COBIMG  DFO  A quick demo (Language: C++, CLR; Data formats: Autodesk Revit, Stanford polygon) F Xue: Semantic enrichment for BIM & GIS, 22 May 2018, SZU 39 3.5 Case 4: For LOD4/500 (Xue et al., 2018b) 室内  Nonlinear optimization problem formulation  Data: PCD scanned by Google Tango phone 𝑓𝑓(X) = RMSE(BIM(X), Pin ) ≈ RMSE(PX , Pin ) ≈ RMSE(P′X , P′in ) = �Σ𝑝𝑝∈P′in nndist (𝑝𝑝, P′X )⁄m′ 2 ≈ RMSE(P′in , P′X ) f X = RMSE BIM X , Pin subject to C(X) ≤ 0. minimize = �Σ𝑝𝑝∈P′X nndist2 (𝑝𝑝, P′in )�‖P′X ‖ (Language: C++, CLR; Data formats: Autodesk Revit, Stanford polygon) F Xue: Semantic enrichment for BIM & GIS, 22 May 2018, SZU 40 3.5 Case 4: For LOD4/500 (Xue et al., 2018b) 室内  Problem solving  CMA-ES-driven Autodesk Revit plugin (Language: C++, CLR; Data formats: Autodesk Revit, Stanford polygon) F Xue: Semantic enrichment for BIM & GIS, 22 May 2018, SZU 41 3.5 Case 4: For LOD4/500 (Xue et al., 2018b) 室内  Indoor modeling  Accurate: 3.87 cm, 100% recall  Fast: 6.44 s  Rich semantics: Product, assembly, etc. Modeler No. 1 2 3 Experience Correctness (out of 8) 8 Expert (3 years) Average (1 8 year) Beginner 8 RMSE (cm) 3.79 Time cost (s) 363.9 3.90 335.4 4.22 691.1 8 3.87 6.44 8 3.87 ~ 246.0 COBIMG -Revit COBIMG-Revit + annotation F Xue: Semantic enrichment for BIM & GIS, 22 May 2018, SZU (Language: C++, CLR; Data formats: Autodesk Revit, Stanford polygon) 42 3.5 Case 4: For LOD4/500 (Xue et al., 2018a) 室内  2D photos: Two from smartphone  Process: CMA-ES-driven SketchUp plugin  Accurate: 3.9 cm, in 2.5 hours (97% time on projection) (b) Indoor case: A scene in a furniture store (384×512 pixels; taken by a smart phone camera with 28mm effective focal length; resolution down sampled) (Language: C++, Ruby; Data formats: SketchUp, BMP) F Xue: Semantic enrichment for BIM & GIS, 22 May 2018, SZU 43 3.6 Case 5: Beyond LOD4/400 (Niu et al., 2018) 实时  World is changing, very fast  BIM/GIS can be “deaf and blind” if not changes over time (Chen et al., 2015; Xue et al., 2018c)  i-Core enabled, cloud service compatible a b c  Demo1 (logistics)  Demo2 (Crane hoist) F Xue: Semantic enrichment for BIM & GIS, 22 May 2018, SZU 44 3.6 Case 5: Beyond LOD4/400 (Niu et al., 2018) 实时  Based on IoT and rules (finite-state machine)  The real-time model o s accurate  Crane o Motions o Safety alerts o Efficiency  Beam o Swings o Rotations F Xue: Semantic enrichment for BIM & GIS, 22 May 2018, SZU 45 3.7 Symmetry: An on-going work (Xue et al., 2018d) 对称  The universal symmetries  a result of economical, manufacturing, functional, aesthetic, and mechanical considerations (a) Reflection (Mirror) (The Taj Mahal, India) (b) Rotation (The Pentagon, USA) (e) Scaling × rotation (f) Rotation × translation (The Pantheon dome, Italy) (The Gherkin, UK) (c) Translation (The Great Wall, China) (d) Translation × scaling (Fractal-like) (Hindu temples) (g) Translation × reflection (h) Cluster of homogeneous (Sugar Hill Project, USA) symmetries (Tulou, China) F Xue: Semantic enrichment for BIM & GIS, 22 May 2018, SZU 46 3.8 Summary of the five cases 小结  Presented a series of semantic enrichment methods  Accuracy up to  cm accurate (in xyz)  s accurate (in time)  Automation and efficiency  Fully, inexpensive (e.g., saving 98% modeling time)  Very fast (s level)  Semantics  Rich  Resulting LODs  LOD2/200 to 4/500 and beyond F Xue: Semantic enrichment for BIM & GIS, 22 May 2018, SZU 47 Section 4 DISCUSSION & FUTURE WORK 总结展望 — “尚未成功，仍需努力” F Xue: Semantic enrichment for BIM & GIS, 22 May 2018, SZU 48 4.1 Discussion  Semantic enrichment for BIM and GIS  Resulting in real-time, accurate, and rich urban semantics o Also enables fully automatic BIM modeling  Is highly demanded o In different temporal (BIM) and spatial (GIS) scales o In various smart city applications  Is feasible in different LODs  Can be n-dimensional  Drawbacks  Data-driven: Limited by rules, training data  Model-driven: Limited by standard components Auto-BIM modeling by one click  Killer (downstream) applications F Xue: Semantic enrichment for BIM & GIS, 22 May 2018, SZU 49 4.2 On-going and future work  Semantic prioritization  Identifying available urban semantic databases  Seeking semantics for killer applications  Data-driven  Symmetry  Interactive machine learning  Improved heuristics, like multiple starts  Model-driven  Algorithm benchmarking  Component-free  More than building elements F Xue: Semantic enrichment for BIM & GIS, 22 May 2018, SZU 50 4.3 Potential collaborations  Inter-disciplinary inter-institutional research  BIM, GIS, CV, RS, AR, …  Joint research funding  Hong Kong o HK ITF Midstream Research Fund (MRF) $5~10 millions (<50% can go to Mainland) o HK RGC Collaborative Research Fund (CRF) o NSFC/RGC Joint Research Scheme ~$1M + RMB 0.8M  Shenzhen o Guangdong - Hong Kong Technology Cooperation Funding Scheme (TCFS)  Greater Bay Area o 重点技术联合创新基金 (still inception) F Xue: Semantic enrichment for BIM & GIS, 22 May 2018, SZU 51 References Auger, A., Finck, S., Hansen, N., and Ros, R. (2010). BBOB 2009: Comparison tables of all algorithms on all noisy functions, INRIA. Birkmann, J., Welle, T., Solecki, W., Lwasa, S., & Garschagen, M. (2016). Boost resilience of small and mid-sized cities. Nature News, 537(7622), 605.  Chen, K., Lu, W., Xue, F., Tang, P., & Li, L. H. (2018a). Automatic building information model reconstruction in high-density urban areas: Augmenting multi-source data with architectural knowledge. Automation in Construction, 93, 22-34.  Chen, K., Lu, W. S. ., Xue, F, Zheng, L. Z., & Liu, D. D. (2018b). Smart Gateway for Bridging BIM and Building. In Proceedings of the 21st International Symposium on Advancement of Construction Management and Real Estate (pp. 1307-1316). Springer, Singapore.  Harty, C., Goodier, C. I., Soetanto, R., Austin, S., Dainty, A. R., & Price, A. D. (2007). The futures of construction: a critical review of construction future studies. Construction Management and Economics, 25(5), 477-493.  Gröger, G., Kolbe, T., & Czerwinski, A. (2007). Candidate OpenGIS CityGML Implementation Specification (City Geography Markup Language). Open Geospatial Consortium Inc, OGC.  National Institute of Building Sciences. (2015). National Building Information Modeling Standard. Version 3, Retrieved from https://www.nationalbimstandard.org/  National Geographic Society. (2012). GIS (geographic information system). http://www.nationalgeographic.org/encyclopedia/geographic-information-system-gis/  Niu, Y., Lu, W., Xue, F., Liu D., Chen, K., Fang, D., & Anumba, C. (2018). Towards the “Third Wave”: An SCO-enabled occupational health and safety management system for construction. Safety Science. Under review  Penttilä, H. (2007). Early architectural design and BIM. Computer-Aided Architectural Design Futures (CAADFutures 2007), 291-302.  Rowley, J. (2007). The wisdom hierarchy: representations of the DIKW hierarchy. Journal of information science, 33(2), 163-180.  Rowley, J., & Hartley, R. (2017). Organizing knowledge: an introduction to managing access to information. Routledge.  Wang, J., Lu, W., Liu, D., & Xue, F. (2018). The Heterogeneity of BIM Objects in Different Construction Contexts. In Construction Research Congress 2018 (pp. 210-220).  World Health Organization. (2015). Global Report on Urban Health—Executive Summary. http://www.who.int/kobe_centre/measuring/urban-global-report/en/  Xu, J., Chen, K., Xue, F., & Lu, W. (2018). 3D point clouds for architecture, engineering, construction, and operation: A SWOT analysis. Working paper  Xue, F., Lu, W., Chen, K. (2018a). Automatic generation of semantically rich as-built building information models using 2D images: A derivative-free optimization approach. Computer-Aided Civil and Infrastructure Engineering, in press.  Xue, F., Lu, W., Chen, K. & Zetkulic, A. (2018b). From ‘semantic segmentation’ to ‘semantic registration’: A derivative-free optimization-based approach for automatic generation of semantically rich as-built building information models (BIMs) from 3D point clouds. Journal of Computing in Civil Engineering. Under review  Xue, F., Chen, K., Lu, W., Huang, GQ. (2018c). Linking radio-frequency identification to Building Information Modeling: Status quo, development trajectory and guidelines for practitioners. Automation in Construction, in press.  Xue. F., Chen, K., Lu, W. (2018d). Architectural Symmetry Detection from 3D Urban Point Clouds: A Derivative-Free Optimization (DFO) Approach. CIB W78 2018, accepted.  Xue, F., Chen, K., Lu, W. (2018e). Semantic enrichment for building rooftop modeling in BIM and GIS. Working paper.  Zins, C. (2007). Conceptual approaches for defining data, information, and knowledge. Journal of the Association for Information Science and Technology, 58(4), 479-493. F Xue: Semantic enrichment for BIM & GIS, 22 May 2018, SZU 52   Thank You ! 谢 谢！