Graduation project 2014
Exploring Regularities for Improving Façade Reconstruction
from Point Cloud
Kaixuan Zhou
Supervisors
Dr. Ben Gorte
Dr. Sisi Zlatanova
Pirouz Nourian
Client
Cyclomedia
Challenge the future
*
Content
•Introduction
•Wall and hole extraction (relevant objects)
•Regularity identification and application
•Quality analysis
•Conclusion
Challenge the future
*
Introduction
• LoD3
•
•
Façade details
Applications
• Serious games: Fire brigade training
• Luminance calculations
• Point Clouds- (Semi)-Automatic
•
•
Terrestrial LiDAR
Panoramic images
Courtesy of Karlsruhe Institute of
Technology
Cyclomedia point cloud
Challenge the future
*
Introduction
• Problems of Data
• Relevant objects
• Wall and holes
•
Occlusions
• Out of Scope
• Not bad for terrestrial point cloud
•
Noise and varying Densities
• Imperfectness of recognition algorithms
•Regularities
•Features shared within one object and
among objects
Challenge the future
*
Introduction
• Problems of Data
• Relevant objects
• Wall and holes
•
Occlusions
• Out of Scope
• Not bad for terrestrial point cloud
•
Noise and varying Densities
•Imperfectness of recognition algorithms
•Regularities
•Features shared within one object and
among objects
Challenge the future
*
Introduction
• Problems of Data
• Relevant objects
• Wall and holes
•
Occlusions
• Out of Scope
•
Noise and varying Densities
•Imperfectness of recognition algorithms
•Regularities
•Features shared within one object and
among objects
low
Density
high
big
Size of hole
small
Challenge the future
*
Introduction
• Problems of Data
• Relevant objects
• Wall and holes
•
Occlusions
• Out of Scope
• Not bad for terrestrial point cloud
•
Noise and varying Densities
•Imperpectness of recogintion algorithms
•Regularities
•Features shared within one object and
among objects
Challenge the future
*
Introduction
• Problems of Data
• Relevant objects
• Wall and holes
•
Occlusions
• Out of Scope
• Not bad for terrestrial point cloud
•
Noise and varing Densities
•Imperpectness of recogintion algorithms
•Regularities
•Features shared within one object and
among objects
Challenge the future
*
Introduction
• Noise, varying density and imperfectness of algorithm can affect
quality of extracted separate objects.
• Research Question:
• In which way regularities can be identified from point cloud
and applied in order to improve quality of 3D facade
reconstruction?
Challenge the future
*
Content
•Introduction
•Wall and hole extraction (relevant objects)
•Regularity identification and application
•Quality analysis
•Conclusion
Challenge the future
*
Wall and Hole extraction
●
●
Facade of Faculty of Applied Sciences (TN)
RANSAC Plane fitting and Wall extraction( knowledge rule: vertical
and largest plane)
Challenge the future
*
Holes extraction
● Rasterization
● Rasterization
● Dilation and erosion closing(varying density)
● Connected component labeling
● Hole points tracing
Challenge the future
*
Holes extraction
● Advantage
●
Robust to noise and varying densities.
●
Tracing point back from original point cloud avoid loss of information
Challenge the future
*
Content
•Introduction
•Wall and hole extraction (relevant objects)
•Regularity identification and application
•Quality analysis
•Conclusion
Challenge the future
*
Regularity identification and application
•Different regularities in the Facade
•Principle of regularities identification and application from feature
and clustering method
•The procedures of regularity identification and application
Challenge the future
*
Different regularities(9 Cases)
•Local Regularity: regularities within one hole
•Orthogonal and parallel orientation
•Global regularities: regularities among holes
•Global regularities among similar holes
•Similar holes share
•Same boundary
•Same orientation
•Position alignment
•Same Line alignment
•Same distance
•Global regularities among different holes
•The boundaries of different holes share
•Parallel and orthogonal orientation
•Same length
•Position
•Same Line alignment
•Same distance
Challenge the future
*
Different regularities(9 Cases)
•Local Regularity: regularities within one hole
•Orthogonal and parallel orientation
•Global regularities: regularities shared among holes
•Global regularities among similar holes
•Similar holes share
•Same boundary
•Same orientation
•Position alignment
•Same Line alignment
•Same distance
•Global regularities among different holes
•Different holes share
•Parallel and orthogonal orientation
•Same length
•Position
•Same Line alignment
•Same distance
Challenge the future
*
Different regularities(9 Cases)
•Local Regularity: regularities within one hole
•Orthogonal and parallel orientation
•Global regularities: regularities shared among holes
•Global regularities among similar holes
•Similar holes share
•Same boundary
•Same orientation
•Position alignment
•Same Line alignment
•Same distance
•Global regularities among different holes
•Different holes (Boundaries) share
•Parallel and orthogonal orientation
•Same length
•Position
•Same Line alignment
•Same distance
Challenge the future
*
Principle of regularity identification and
application
Objects sharing regularities have similar certain features
Clustering method can find the similar objects in the feature space
The weighted center is chosen as representative of cluster used for representing all
member in cluster
Challenge the future
*
Principle of regularity identification and
application
Hierarchical Clustering (Group similar objects in feature space)
Challenge the future
*
The procedures of regularity identification
and application
Find features to present each regularity for clustering
Procedure (1): local regularity and global regularities among
similar holes identification and application
Procedure (2): global regularities between different holes
identification and application
Challenge the future
*
The procedures of regularity identification
and application: Procedures(1)-- Overview
Separated Hole points
ICP
Group of similar
holes
contains
Transformation
matrices
Registered similar
holes
Regularities
applied
Regularized group of
similar holes
Challenge the future
*
The procedures of regularity identification
and application: Procedures(1)-- Overview
ICP(iterative closest points)similar objects identification and registration
•
Score: distance of closest pairs
Challenge the future
*
The procedures of regularity identification
and application: Procedures(1)-- Overview
Register holes points
contains all boundary information
Transformation matrices
contain orientation and position information
Challenge the future
*
The procedures
of regularity
identification and
application:
Procedures(1)– Detail
Challenge the future
*
The procedures of regularity identification and
application: Procedures(1)– Detail
RANSAC line fitting: boundary line candidates
Challenge the future
*
The procedures of regularity identification and
application: Procedures(1)– Detail
Local regularity: orthogonal and
parallel orientation. θ
Parallel
when
orientation: i   j  0,  , 2
Orthogonal
when
orientation:
i   j 

2
,
3
;
2
r≥ 0 and θ in the interval (−π, π]
Challenge the future
*
The procedures of regularity identification and
application: Procedures(1)– Detail
Same boundary: (r, θ)
Same
boundary
regularity when:
 ri ,i    rj , j 
Challenge the future
*
The procedures of regularity identification and
application: Procedures(1)– Detail
Same orientation:
Same
orientation
regularity when:
 i ,  i     j ,  j 
Challenge the future
*
The procedures of regularity identification and
application: Procedures(1)– Detail
Translation in 2D:
Aligned orientation:
Translation in
direction:
Common case
σ
Challenge the future
*
The procedures of regularity identification and
application: Procedures(1)– Detail
Line alignment regularity
Distance between objects
regularity when:
t i x  t j x or t i y  t j y
t x ij  t x jk , or t y ij  t y jk
Challenge the future
*
The procedures of regularity identification
and application:Procedures(2)
Challenge the future
*
The procedures of regularity identification
and application:Procedures(2)
Façade point cloud from Faculty of Architecture and Built
Environment (BK)
Challenge the future
*
The procedures of regularity identification
and application:Procedures(2)
Parallel and orthogonal orientation:
•
•
All orientations of boundary from all groups are
considered
The regularity identification and application is
the same with local regularity.
Challenge the future
*
The procedures of regularity identification
and application:Procedures(2)
Same length:
•
Lengthes of boundary share a same direction
from all groups are clustered respectively .
Same length regularity
when:
L i  L j
Challenge the future
*
The procedures of regularity identification
and application:Procedures(2)
Position regularities: preserve the chains established in previous position
regularities among similar objects
Same line alignment among similar objects
Same distance among similar objects
Challenge the future
*
The procedures of regularity identification
and application:Procedures(2)
Chains are restrianed to move to its
orthogonal direction
Challenge the future
*
The procedures of regularity identification
and application:Procedures(2)
Only Boundaries with similar direction with
Chains to taken into account for position
regularites
Challenge the future
*
The procedures of regularity identification
and application:Procedures(2)
The regularities are found between
prominent group (Target)and one of other
groups(Source) each time
Challenge the future
*
L
The procedures of regularity identification
and application:Procedures(2)
Same line alignment
regularity when:
Same distance:
 ri ,i    rj , j 
(1)
r1  r2
Challenge the future
*
Quality analysis
• Effects of provided procedure for applying regularity
• Match with original point cloud
Challenge the future
*
Quality analysis
• Effects of provided procedure for applying regularity
•
TN Facade(Case1-Case6)
Challenge the future
*
Quality analysis
• Effects of provided procedure for applying regularity
•
BK Facade(Case1-Case9)
Challenge the future
*
Quality analysis
• Match with original point clouds
Challenge the future
*
Conclusion and future work
Conclusions:
Hole extraction
• Rasterization approach
• Robust to various density, noise
• No loss information of edge
Local and global regularities identification and application
• Provide 9 cases of regularities to explore
• Feature and clustering method to extract regularities
• ICP to find different groups of similar holes
• Register points and Transformation matrices
• RANSAC line fitting to find the boundary lines
• Chains to preserve the established connections
Quality
• Procedures provided works fine with these two datasets
• Good match with original point cloud
• Improve results from noise, various densities and imperfectness of algorithms
Challenge the future
*
Conclusion and future work
Future work:
•
The regularities of whole façade: including extrusions, intrusions, doors. Even
for a whole building with several facades
•
Special cases of regularities can be also applied: orientations of similar objects
share orthogonal orientations. The similar objects shares mirror reflection
regularity. Position regularities are explored among all groups
•
Occlusion problem needs to be fixed. For example, ICP can not identify
partially matched objects
•
Thresholds need to be limited and set adaptively. The relations between
thresholds can be derived in order to reduce number of threshold
Challenge the future
*
Appendix
Step
Wall extraction
TN
BK
RANSAC
plane fitting
DistanceThreshold=0.1m
DistanceThreshold=0.1m
Rasterization
Resolution=0.05m
Resolution=0.05m
ICP
Iteration times =20
The maximum distance
between closest pairs=0.005m
Iteration times =20
The maximum distance
between closest pairs=0.005m
RANSAC line
fitting
RansacDisThreshold=0.005m
Minimum number of points in
model=points*0.005
RansacDisThreshold=0.005m
Minimum number of points in
model=points*0.005
Local
regularityCase1
Clustering cut-off value =0.05(
2.8 )
Clustering cut-off value =0.05(
2.8 )
One incomplete holes:
Clustering cut-off value =0.1(
5.7 )
Same
boundary
regularityCase2
Orientation
regularityCase3
Position
regularity (2
types)-Case4,
Case5
Clustering cut-off value
==0.05
Clustering cut-off value =0.05
Clustering cut-off value
==0.05( 2.8 )
Clustering cut-off value
==0.05( 2.8 )
Clustering cut-off value
==0.10m
Clustering cut-off value
==0.15m
Orthogonal
and parallel
regularityCase6
Length
regularityCase7
Clustering cut-off value =0.05(
2.8 )
Clustering cut-off value =0.05(
2.8 )
Clustering cut-off value
=0.05m
Clustering cut-off value
=0.05m
Hole extraction
First procedure of
regularity
Thresholds
Second procedure
of regularity
Challenge the future
*
Appendix
Segmentation
Algorithm
Source
RANSAC plane fitting
PCL
Dilation and erosion
Supervisor
Connected components
labeling
Supervisor
Hierachical Clustering
ALGLIB
ICP
PCL
RANSAC Line fitting
PCL
Hole extraction
Regularities
Challenge the future
*
Challenge the future
*