Estimation of projection matrices from a sparse set of feature points for 3D tree reconstruction from multiple images

Štefan Kohek, Damjan Strnad, Borut Žalik, Simon Kolmanič


3D reconstruction of trees is an important task for tree analysis but the most affordable approach to capture real objects is with a camera. Although, there already exist methods for 3D reconstruction of trees from multiple photographs, they mostly handle only self-standing trees captured at narrow angles. In fact, dense feature detection and matching is in most cases only the first step of the reconstruction and requires a large set of features and high similarity between individual pictures. However, capturing trees in the orchard is in most cases possible only at wider angles between the individual pictures and with overlapping branches from other trees, which prevents reliable feature matching. We introduce a new approach for estimating projection matrices to produce 3D point clouds of trees from multiple photographs. By manually relating a smaller number of points on images to reference objects, we substitute the missing dense set of features. We assign to each image a projection matrix and minimize the projection error between the images and reference objects using simulated annealing. Thereby, we produce correct projection matrices for further steps in 3D reconstruction. Our approach is tested on a simple application for 3D reconstruction of trees to produce a 3D point cloud. We analyze convergence rates of the optimization and show that the proposed approach can produce feasible projection matrices from a sufficiently large set of feature points. In the future, this approach will be a part of a complete system for tree reconstruction and analysis.


3D reconstruction;simulated annealing;projection matrix

Full Text:



J. R. Rosell, J. Llorens, R. Sanz, J. Arno, M. Ribes-Dasi, J. Masip, A. Escolà, F. Camp, F. Solanelles, F. Gràcia, et al., “Obtaining the Three-Dimensional Structure of Tree Orchards from Remote 2D Terrestrial Lidar Scanning,” Agricultural and Forest Meteorology, vol. 149, no. 9, pp. 1505–1515, 2009.

N. M. Elfiky, S. A. Akbar, J. Sun, J. Park, and A. Kak, “Automation of Dormant Pruning in Specialty Crop Production: An Adaptive Framework for Automatic Reconstruction and Modeling of Apple Trees,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 65–73, 2015.

C. Li, J. Zheng, C. Dang, and H. Zhou, “A Method of 3D Reconstruction from Image Sequence,” in Image and Signal Processing, 2009. CISP’09. 2nd International Congress on, pp. 1–5, IEEE, 2009.

R. I. Hartley and P. Sturm, “Triangulation,” Computer Vision and Image Understanding, vol. 68, no. 2, pp. 146–157, 1997.

D. G. Lowe, “Object Recognition from Local Scale-Invariant Features,” in Computer Vision, 1999. The proceedings of the seventh IEEE international conference on, vol. 2, pp. 1150–1157, Ieee, 1999.

H. Bay, A. Ess, T. Tuytelaars, and L. Van Gool, “Speeded-up Robust Features (Surf),” Computer Vision and Image Understanding, vol. 110, no. 3, pp. 346–359, 2008.

P. Tan, G. Zeng, J. Wang, S. B. Kang, and L. Quan, “Image-Based Tree Modeling,” ACM Transactions on Graphics, vol. 26, 2007.

P. Tan, T. Fang, J. Xiao, P. Zhao, and L. Quan, “Single Image Tree Modeling,” ACM Transactions on Graphics, vol. 27, pp. 1081–1087, 2008.

B. Neubert, T. Franken, and O. Deussen, “Approximate Image-Based Tree-Modeling Using Particle Flows,” ACM Transactions on Graphics, vol. 26, no. 3, 2007.

A. Zamuda, J. Brest, B. Bošković, and V. Žumer, “Differential Evolution for Parameterized Procedural Woody Plant Models Reconstruction,” Applied Soft Computing, vol. 11, no. 8, pp. 4904–4912, 2011.

C. Wu, L. He, X. Du, S. Chen, and K. Ni, “3D Reconstruction of Chinese Hickory Tree for Dynamics Analysis,” Biosystems Engineering, vol. 119, pp. 69 – 79, 2014.

D. Zhang, N. Xie, S. Liang, and J. Jia, “3D Tree Skeletonization from Multiple Images Based on PyrLK Optical Flow,” Pattern Recognition Letters, vol. 76, pp. 49 – 58, 2016.

S. M. Seitz and C. R. Dyer, “Photorealistic Scene Reconstruction by Voxel Coloring,” International Journal of Computer Vision, vol. 35, no. 2, pp. 151–173, 1999.

A. Whitehead and G. Roth, “Estimating Intrinsic Camera Parameters from the Fundamental Matrix using an Evolutionary Approach,” EURASIP Journal on Advances in Signal Processing, vol. 2004, no. 8, p. 412751, 2004.

Q. Ji and Y. Zhang, “Camera Calibration with Genetic Algorithms,” IEEE Transactions on Systems, Man, and Cybernetics, vol. 31, no. 2, pp. 120–130, 2001.

S. Kirkpatrick, C. D. Gelatt, M. P. Vecchi, et al., “Optimization by Simulated Annealing,” Science, vol. 220, no. 4598, pp. 671–680, 1983.

D. Strnad and Š. Kohek, “Novel Discrete Differential Evolution Methods for Virtual Tree Pruning Optimization,” Soft Computing, pp. 1–13, 2015.

N. Metropolis, A. W. Rosenbluth, M. N. Rosenbluth, A. H. Teller, and E. Teller, “Equation of State Calculations by Fast Computing Machines,” The Journal of Chemical Physics, vol. 21, no. 6, pp. 1087–1092, 1953.

R. Hartley, R. Gupta, and T. Chang, “Stereo from Uncalibrated Cameras,” in Computer Vision and Pattern Recognition, 1992. Proceedings CVPR’92., 1992 IEEE Computer Society Conference on, pp. 761–764, IEEE, 1992.

P. C. Schuur, “Classification of Acceptance Criteria for the Simulated Annealing Algorithm,” Mathematics of Operations Research, vol. 22, no. 2, pp. 266–275, 1997.

Š. Kohek, N. Guid, S. Tojnko, T. Unuk, and S. Kolmanič, “Eduapple: Interactive Teaching Tool for Apple Tree Crown Formation,” HortTechnology, vol. 25, no. 2, pp. 238–246, 2015.

R. Storn and K. Price, “Differential Evolution – A Simple and Efficient Heuristic for Global Optimization over Continuous Spaces,” Journal of Global Optimization, vol. 11, no. 4, pp. 341–359, 1997.

J. Brest, S. Greiner, B. Boskovic, M. Mernik, and V. Žumer, “Self-adapting Control Parameters in Differential Evolution: A comparative Study on Numerical Benchmark Problems,” IEEE Transactions on Evolutionary Computation, vol. 10, no. 6, pp. 646–657, 2006.

T. Robič and B. Filipič, “Demo: Differential Evolution for Multiobjective Optimization,” in International Conference on Evolutionary Multi-Criterion Optimization, pp. 520–533, Springer, 2005.



  • There are currently no refbacks.

Copyright (c) 2017 Periodicals of Engineering and Natural Sciences (PEN)

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.

ISSN: 2303-4521

Digital Object Identifier DOI: 10.21533/pen

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License