Smart Check 3D: An Industrial Inspection System Combining 3D Vision with Automatic Planning of Inspection Viewpoints Integrated Imaging and Vision Techniques for Industrial Inspection pp 377-392 | Cite as Nicola Carlon (1) Email author (
[email protected]) Nicolò Boscolo (1) Stefano Tonello (1) Emanuele Menegatti (2) 1. IT+Robotics Srl, Padova, Italy 2. Intelligent Autonomous Systems Laboratory, University of Padova, Padova, Italy Chapter First Online: 25 September 2015 856 Downloads Part of the Advances in Computer Vision and Pattern Recognition book series (ACVPR)
Abstract In this chapter, we describe an industrial inspection system composed by a 3D vision system, mounted on a manipulator robot arm, able to perform quality and completeness inspection on a complex solid part. The novelty of the system is in the deep integration among three software modules: the visual inspection system, the 3D simulation software, and the motion planning engine of the manipulator robot. This enables an automatic off-line programming of the robot path by specifying in the system the desired inspection tasks. The system automatically generates the needed points of view in order to perform 3D reconstruction and automatic visual inspection. Moreover, the motion planning system can reorder the inspection points in order to optimize the inspection cycle time. The core of this system was developed in the European Project “Thermobot,” and currently, it is been engineered to be deployed in an industrial production plant.
Notes Acknowledgments This research has been funded by the European Unions 7th Framework (FP7/2007–2013) under Grant Agreement No. 284607, Thermobot project, and 3D Complete project (Grant Agreement No. 262009).
References 1.
Malamas EN, Petrakis EGM, Zervakis M, Petit L, Legat JD (2003) A survey on industrial vision systems, applications and tools. Image Vis Comput 21(2):171–188 CrossRef (https://doi.org/10.1016/S0262-8856(02)00152-X) Google Scholar (http://scholar.google.com/scholar_lookup? title=A%20survey%20on%20industrial%20vision%20systems%2C%20applications%20and%20tools&author=EN.%20Malamas&author=E GM.%20Petrakis&author=M.%20Zervakis&author=L.%20Petit&author=JD.%20Legat&journal=Image%20Vis%20Comput&volume=21&iss ue=2&pages=171-188&publication_year=2003)
2.
So EWY, Munaro M, Michieletto S, Tonello S, Menegatti E (2013) 3Dcomplete: efficient completeness inspection using a 2.5D color scanner. Comput Ind 64(9):1237–1252 CrossRef (https://doi.org/10.1016/j.compind.2013.03.014) Google Scholar (http://scholar.google.com/scholar_lookup? title=3Dcomplete%3A%20efficient%20completeness%20inspection%20using%20a%202.5D%20color%20scanner&author=EWY.%20So&a uthor=M.%20Munaro&author=S.%20Michieletto&author=S.%20Tonello&author=E.%20Menegatti&journal=Comput%20Ind&volume=64& issue=9&pages=1237-1252&publication_year=2013)
3.
Keyes D (2007) Petaflop/s, seriously. Lect Notes Comput Sci 4873:2 CrossRef (https://doi.org/10.1007/978-3-540-77220-0_2) Google Scholar (http://scholar.google.com/scholar_lookup? title=Petaflop%2Fs%2C%20seriously&author=D.%20Keyes&journal=Lect%20Notes%20Comput%20Sci&volume=4873&pages=2&publicat ion_year=2007)
4.
Pretto A, Tonello S, Menegatti E (2013) Flexible 3D localization of planar objects for industrial bin-picking with monocamera vision system. In: IEEE international conference on automation science and engineering (CASE), 2013, pp 168–175 Google Scholar (https://scholar.google.com/scholar? q=Pretto%20A%2C%20Tonello%20S%2C%20Menegatti%20E%20%282013%29%20Flexible%203D%20localization%20of%20planar%2 0objects%20for%20industrial%20binpicking%20with%20monocamera%20vision%20system.%20In%3A%20IEEE%20international%20conference%20on%20automation%20sci ence%20and%20engineering%20%28CASE%29%2C%202013%2C%20pp%20168%E2%80%93175)
5.
So E, Munaro M, Michieletto S, Menegatti E, Tonello S (2013) 3Dcomplete: efficient completeness inspection using a 2.5D color scanner. Comput Ind Spec Issue 3D Imaging Ind 64(9):1237–1252 Google Scholar (https://scholar.google.com/scholar? q=So%20E%2C%20Munaro%20M%2C%20Michieletto%20S%2C%20Menegatti%20E%2C%20Tonello%20S%20%282013%29%203Dco mplete%3A%20efficient%20completeness%20inspection%20using%20a%202.5D%20color%20scanner.%20Comput%20Ind%20Spec%20Is sue%203D%20Imaging%20Ind%2064%289%29%3A1237%E2%80%931252)
6.
Singh AP, Latombe JC, Brutlag DL (1999) A motion planning approach to flexible ligand binding. In: ISMB, pp 252–261 Google Scholar (https://scholar.google.com/scholar? q=Singh%20AP%2C%20Latombe%20JC%2C%20Brutlag%20DL%20%281999%29%20A%20motion%20planning%20approach%20to%2 0flexible%20ligand%20binding.%20In%3A%20ISMB%2C%20pp%20252%E2%80%93261)
7.
Lien JM, Bayazit B, Sowell RT, Rodriguez S, Amato NM (2004) Shepherding behaviors. In: Proceedings. ICRA’04. IEEE international conference on robotics and automation, 2004, vol 4, pp 4159–4164 Google Scholar (https://scholar.google.com/scholar? q=Lien%20JM%2C%20Bayazit%20B%2C%20Sowell%20RT%2C%20Rodriguez%20S%2C%20Amato%20NM%20%282004%29%20Shep herding%20behaviors.%20In%3A%20Proceedings.%20ICRA%E2%80%9904.%20IEEE%20international%20conference%20on%20robotics %20and%20automation%2C%202004%2C%20vol%204%2C%20pp%204159%E2%80%934164)
8.
Mettin U, Shiriaev AS, Freidovich LB, Sampei M (2010) Optimal ball pitching with an under actuated model of a human arm. In: IEEE international conference on robotics and automation (ICRA), 2010, pp 5009–5014 Google Scholar (https://scholar.google.com/scholar? q=Mettin%20U%2C%20Shiriaev%20AS%2C%20Freidovich%20LB%2C%20Sampei%20M%20%282010%29%20Optimal%20ball%20pitc hing%20with%20an%20under%20actuated%20model%20of%20a%20human%20arm.%20In%3A%20IEEE%20international%20conference %20on%20robotics%20and%20automation%20%28ICRA%29%2C%202010%2C%20pp%205009%E2%80%935014)
9.
LaValle SM (2006) Planning algorithms. Cambridge University Press, Cambridge Google Scholar (https://scholar.google.com/scholar? q=LaValle%20SM%20%282006%29%20Planning%20algorithms.%20Cambridge%20University%20Press%2C%20Cambridge)
10.
Schwartz JT, Sharir M (1983) On the piano movers problem. II. General techniques for computing topological properties of real algebraic manifolds. Adv Appl Math 4(3):298–351 MATH (http://www.emis.de/MATH-item?0554.51008) MathSciNet (http://www.ams.org/mathscinet-getitem?mr=712908) CrossRef (https://doi.org/10.1016/0196-8858(83)90014-3) Google Scholar (http://scholar.google.com/scholar_lookup? title=On%20the%20piano%20movers%20problem.%20II.%20General%20techniques%20for%20computing%20topological%20properties% 20of%20real%20algebraic%20manifolds&author=JT.%20Schwartz&author=M.%20Sharir&journal=Adv%20Appl%20Math&volume=4&iss ue=3&pages=298-351&publication_year=1983)
11.
LaValle SM, Kuffner JJ (2001) Randomized kinodynamic planning. Int J Robot Res 20(5):378–400 CrossRef (https://doi.org/10.1177/02783640122067453) Google Scholar (http://scholar.google.com/scholar_lookup? title=Randomized%20kinodynamic%20planning&author=SM.%20LaValle&author=JJ.%20Kuffner&journal=Int%20J%20Robot%20Res&v olume=20&issue=5&pages=378-400&publication_year=2001)
12.
Kavraki LE, Svestka P, Latombe JC, Overmars MH (1996) Probabilistic roadmaps for path planning in high-dimensional configuration spaces. IEEE Trans Robot Autom 12(4):566–580 CrossRef (https://doi.org/10.1109/70.508439) Google Scholar (http://scholar.google.com/scholar_lookup?title=Probabilistic%20roadmaps%20for%20path%20planning%20in%20highdimensional%20configuration%20spaces&author=LE.%20Kavraki&author=P.%20Svestka&author=JC.%20Latombe&author=MH.%20Over mars&journal=IEEE%20Trans%20Robot%20Autom&volume=12&issue=4&pages=566-580&publication_year=1996)
13.
Carpin S, Pillonetto G (2005) Motion planning using adaptive random walks. IEEE Trans Robot 21(1):129–136 CrossRef (https://doi.org/10.1109/TRO.2004.833790) Google Scholar (http://scholar.google.com/scholar_lookup? title=Motion%20planning%20using%20adaptive%20random%20walks&author=S.%20Carpin&author=G.%20Pillonetto&journal=IEEE%20 Trans%20Robot&volume=21&issue=1&pages=129-136&publication_year=2005)
14.
Karaman S, Frazzoli E (2011) Sampling-based algorithms for optimal motion planning. Int J Robot Res 30(7):846–894 CrossRef (https://doi.org/10.1177/0278364911406761) Google Scholar (http://scholar.google.com/scholar_lookup?title=Samplingbased%20algorithms%20for%20optimal%20motion%20planning&author=S.%20Karaman&author=E.%20Frazzoli&journal=Int%20J%20Ro bot%20Res&volume=30&issue=7&pages=846-894&publication_year=2011)
15.
SM LaValle (2003) From dynamic programming to RRTs: algorithmic design of feasible trajectories. In: Control problems in robotics. Springer, Berlin, pp 19–37 Google Scholar (https://scholar.google.com/scholar? q=SM%20LaValle%20%282003%29%20From%20dynamic%20programming%20to%20RRTs%3A%20algorithmic%20design%20of%20fe asible%20trajectories.%20In%3A%20Control%20problems%20in%20robotics.%20Springer%2C%20Berlin%2C%20pp%2019%E2%80%9 337)
16.
Perez A, Platt R, Konidaris G, Kaelbling L, Lozano-Perez T (2012) LQR-RRT*: optimal sampling-based motion planning with automatically derived extension heuristics. In: IEEE international conference on robotics and automation (ICRA), 2012, pp 2537–2542 Google Scholar (https://scholar.google.com/scholar? q=Perez%20A%2C%20Platt%20R%2C%20Konidaris%20G%2C%20Kaelbling%20L%2C%20LozanoPerez%20T%20%282012%29%20LQR-RRT%2A%3A%20optimal%20samplingbased%20motion%20planning%20with%20automatically%20derived%20extension%20heuristics.%20In%3A%20IEEE%20international% 20conference%20on%20robotics%20and%20automation%20%28ICRA%29%2C%202012%2C%20pp%202537%E2%80%932542)
17.
Glassman E, Tedrake R (2010) A quadratic regulator-based heuristic for rapidly exploring state space. In: IEEE international conference on robotics and automation (ICRA), 2010, pp 5021–5028 Google Scholar (https://scholar.google.com/scholar? q=Glassman%20E%2C%20Tedrake%20R%20%282010%29%20A%20quadratic%20regulatorbased%20heuristic%20for%20rapidly%20exploring%20state%20space.%20In%3A%20IEEE%20international%20conference%20on%20rob otics%20and%20automation%20%28ICRA%29%2C%202010%2C%20pp%205021%E2%80%935028)
18.
Aboaf EW, Drucker S, Atkeson CG (1989) Task-level robot learning: juggling a tennis ball more accurately. In: IEEE international conference on robotics and automation, 1989. Proceedings 1989, pp 1290–1295 Google Scholar (https://scholar.google.com/scholar? q=Aboaf%20EW%2C%20Drucker%20S%2C%20Atkeson%20CG%20%281989%29%20Tasklevel%20robot%20learning%3A%20juggling%20a%20tennis%20ball%20more%20accurately.%20In%3A%20IEEE%20international%20co nference%20on%20robotics%20and%20automation%2C%201989.%20Proceedings%201989%2C%20pp%201290%E2%80%931295)
19.
Plaku E, Bekris KE, Chen BY, Ladd AM, Kavraki LE (2005) Sampling-based roadmap of trees for parallel motion planning. IEEE Trans Robot 21(4):597–608 Google Scholar (https://scholar.google.com/scholar? q=Plaku%20E%2C%20Bekris%20KE%2C%20Chen%20BY%2C%20Ladd%20AM%2C%20Kavraki%20LE%20%282005%29%20Samplin gbased%20roadmap%20of%20trees%20for%20parallel%20motion%20planning.%20IEEE%20Trans%20Robot%2021%284%29%3A597%E 2%80%93608)
20.
Tonello S, Zanetti GP, Finotto M, Bortoletto R, Tosello E, Menegatti E (2012) Workcellsimulator: a 3D simulator for intelligent manufacturing. In: Simulation, modeling, and programming for autonomous robots. Springer, Berlin, pp 311–322 Google Scholar (https://scholar.google.com/scholar? q=Tonello%20S%2C%20Zanetti%20GP%2C%20Finotto%20M%2C%20Bortoletto%20R%2C%20Tosello%20E%2C%20Menegatti%20E% 20%282012%29%20Workcellsimulator%3A%20a%203D%20simulator%20for%20intelligent%20manufacturing.%20In%3A%20Simulation %2C%20modeling%2C%20and%20programming%20for%20autonomous%20robots.%20Springer%2C%20Berlin%2C%20pp%20311%E2 %80%93322)
21.
Bouguet J-Y, Perona P (1998) Camera calibration from points and lines in dual-space geometry. In: Proceedings of 5th European conference on computer vision, pp 2–6 Google Scholar (https://scholar.google.com/scholar?q=Bouguet%20JY%2C%20Perona%20P%20%281998%29%20Camera%20calibration%20from%20points%20and%20lines%20in%20dualspace%20geometry.%20In%3A%20Proceedings%20of%205th%20European%20conference%20on%20computer%20vision%2C%20pp%20 2%E2%80%936)
22.
Tsai RY (1987) A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf tv cameras and lenses. IEEE J Robot Autom 3(4):323–344 Google Scholar (https://scholar.google.com/scholar? q=Tsai%20RY%20%281987%29%20A%20versatile%20camera%20calibration%20technique%20for%20highaccuracy%203D%20machine%20vision%20metrology%20using%20off-theshelf%20tv%20cameras%20and%20lenses.%20IEEE%20J%20Robot%20Autom%203%284%29%3A323%E2%80%93344)
23.
So EWY, Michieletto S, Menegatti E (2012) Calibration of a dual-laser triangulation system for assembly line completeness inspection. In: IEEE international symposium on robotic and sensors environments, pp 138–143 Google Scholar (https://scholar.google.com/scholar? q=So%20EWY%2C%20Michieletto%20S%2C%20Menegatti%20E%20%282012%29%20Calibration%20of%20a%20duallaser%20triangulation%20system%20for%20assembly%20line%20completeness%20inspection.%20In%3A%20IEEE%20international%20s ymposium%20on%20robotic%20and%20sensors%20environments%2C%20pp%20138%E2%80%93143)
24.
Eitzinger C, Ghidoni S, Menegatti E (2013) Thermobot: towards semi-autonomous, thermographic detection of cracks. In: International conference on heating by electromagnetic sources, Padua Google Scholar (https://scholar.google.com/scholar? q=Eitzinger%20C%2C%20Ghidoni%20S%2C%20Menegatti%20E%20%282013%29%20Thermobot%3A%20towards%20semiautonomous%2C%20thermographic%20detection%20of%20cracks.%20In%3A%20International%20conference%20on%20heating%20by% 20electromagnetic%20sources%2C%20Padua)
25.
Dorigo M, Maniezzo V, Colorni A (1996) Ant system: optimization by a colony of cooperating agents. IEEE Trans Syst Man, Cybern Part B Cybern 26(1):29–41 CrossRef (https://doi.org/10.1109/3477.484436) Google Scholar (http://scholar.google.com/scholar_lookup? title=Ant%20system%3A%20optimization%20by%20a%20colony%20of%20cooperating%20agents&author=M.%20Dorigo&author=V.%2 0Maniezzo&author=A.%20Colorni&journal=IEEE%20Trans%20Syst%20Man%2C%20Cybern%20Part%20B%20Cybern&volume=26&issu e=1&pages=29-41&publication_year=1996)
Copyright information © Springer-Verlag London (outside the USA) 2015
About this chapter Cite this chapter as: Carlon N., Boscolo N., Tonello S., Menegatti E. (2015) Smart Check 3D: An Industrial Inspection System Combining 3D Vision with Automatic Planning of Inspection Viewpoints. In: Liu Z., Ukida H., Ramuhalli P., Niel K. (eds) Integrated Imaging and Vision Techniques for Industrial Inspection. Advances in Computer Vision and Pattern Recognition. Springer, London DOI (Digital Object Identifier) https://doi.org/10.1007/978-1-4471-6741-9_12 Publisher Name Springer, London Print ISBN 978-1-4471-6740-2 Online ISBN 978-1-4471-6741-9 eBook Packages Computer Science About this book Reprints and Permissions
Personalised recommendations
© 2017 Springer International Publishing AG. Part of Springer Nature. Not logged in Not affiliated 185.191.229.108