Idea Transcript
Image Processing and Computer Vision with MATLAB and SIMULINK
By Dr Jasmina Lazić Application Engineering, MathWorks
© 2015 The MathWorks, Inc. 1
The Computer Vision Ecosystem Simulink™ Embedded Coder™
Neural Network Toolbox™
Robotic Vision
Control System Toolbox™
Control Robotics
Computer Intelligence
Artificial Intelligence
Computer Vision System Toolbox™ Cognitive Vision
Machine Learning
Multi-variable SP Non-linear SP
Signal Processing Toolbox™
Signal Processing Computer Vision
Machine Vision
Physics
DSP System Toolbox™
SimScape™
Optics
SimMechanics™ Statistics Toolbox™
Statistics
Mathematics
Image Processing
Imaging
Geometry
MATLAB™
Optimization
Optimization Toolbox™
Neurobiology Biological Vision
SimBiology™
Smart Cameras
Image Acquisition Toolbox™
Image Processing Toolbox™ 2
Applications: Image and Video Processing
Medical imaging Surveillance Robotics Automotive safety Consumer electronics Geospatial computing Machine vision and more…
3
Agenda
• Introduction
• Feature Detect, Extract & Match
• Image Acquisition
• Object Detection and Tracking
• Explore-Enhance-Segment-Analyze • 3D Vision
• Image Registration
• Embedded Applications
4
http://www.mathworks.co.uk/products/image-video-processing
5
Agenda • Introduction
• Introduction
• Feature Detect, Extract & Match
• Image Acquisition
Demo: MATLAB live video with Microsoft Kinect • Object Detection and Tracking • Image Acquisition
Demo: Image Acquisition App
• Explore-Enhance-Segment-Analyze • 3D Vision
• Image Registration
• Embedded Applications
6
Image Acquisition
Acquire images and video directly into MATLAB and Simulink
Configure device properties
7
Agenda • Introduction
• Feature Detect, Extract & Match • Image Image Acquisition Acquisition
Demo: Live video in MATLAB Demo: Image Acquisition App Demo: Live video in SIMULINK
• Object Detection and Tracking
• Explore-Enhance-Segment-Analyze Demo: Tissue segmentation • Explore-Enhance-Segment-Analyze • 3D Vision Demo: Image analysis Apps
Demo: Digital display reader
• Image Registration
• Embedded Applications
8
Standard Segmentation Workflow
9
Standard Segmentation Workflow
Explore image
10
Standard Segmentation Workflow
Explore image Enhance image
11
Standard Segmentation Workflow
Explore image Enhance image Segment image
12
Standard Segmentation Workflow
Explore image Enhance image Segment image Analyze image
13
Demos: Explore-Enhance-Segment-Analyze
14
Image Processing Apps NEW! NEW!
Image Viewer Color Thresholder Region Analyzer
15
Optical Character Recognition
Support for English, Japanese + others
Simple interface: >> results = ocr(I)
16
Segmentation – other techniques Watershed and Active Contours (NEW)
17
Agenda • Introduction
• Feature Detect, Extract & Match • Image Acquisition
• Object Detection and Tracking • Explore-Enhance-Segment-Analyze Explore-Enhance-Segment-Analyze
Demo: Tissue segmentation Demo: Image analysis Apps Demo: Digital display reader
• 3D Vision
• Image Registration Demo: Video Mosaicking (Simulink) • Image Registration
• Embedded Applications
18
Image Registration
2D and 3D Align multi-model images
19
Image Registration
NEW: Non-rigid image registration and warping
20
Demo: Video Mosaicking
21
Agenda • Introduction
• Feature Detect, Extract & Match Geometry Estimation • Feature Detect, Extract & Match
• Image Acquisition
• Object Detection and Tracking • Explore-Enhance-Segment-Analyze
• 3D Vision • Image Image Registration Registration
Demo: Video Mosaicking (Simulink)
• Embedded Applications
22
Feature Detection, Extraction and Matching
Edge
Corner
Template
SURF
MSER
23
Point Tracker
Simple interface:
points = detectXFeatures(I); tracker = vision.PointTracker; initialize(tracker, points.Location, frame); while ~finished frame = step(videoFileReader); [points, validity] = step(tracker, frame); end
24
Geometry Estimation
RANSAC – Random Sampling and Consensus Gets geometry from a minimal sample set and checks it against all matches
25
Agenda • Introduction
• Feature Feature Detect, Detect,Extract Extract&&Match Match
• Image Acquisition
Geometry Estimation
• Object Detection and Tracking Demo: Pedestrianand Tracking • Object Detection Tracking
• Explore-Enhance-Segment-Analyze
• 3D Vision • Image Registration
• Embedded Applications
26
Object Detection and Tracking
27
Face Detection, People Detection and Foreground Detection
28
Training Image Labeler App
29
Demo: Tracking multiple objects with a Kalman Filter
30
Demo: Tracking multiple objects with a Kalman Filter
Kalman Filter interface: – Configure: kalmanFilter = configureKalmanFilter( ... 'ConstantVelocity', centroid, ... [200, 50], [100, 25], 100); – Predict: predictedCentroid = predict(kalmanFilter); – Update: c2 = correct(kalmanFilter, centroid); 31
Bag of Words for Category Classification 1. Create Bag of Features
2. Encode Input Images
3. Repeat for all classes
4. Use machine learning algorithm to discriminate between classes
32
Agenda • Introduction
• Feature Detect, Extract & Match
• Image Acquisition
• Object Object Detection Detectionand andTracking Tracking
• Explore-Enhance-Segment-Analyze
Demo: Multi-object Tracking
• 3D Vision Demo: Camera Calibrator App • 3D Vision Demo: 3D Reconstruction
Demo: Depth Mapping
• Image Registration
• Embedded Applications
33
3D Vision Pyramid with dynamic programming and sub-pixel accuracy 15
2.5 2 1.5 1 10
y (meters)
0.5 0 -0.5 -1 5
-1.5 -2 -2.5
-8 0
-6 -4 3 z (meters)
2
1
0
-1
-2
-3
-4
x (meters)
34
NEW Apps: Camera Calibrator and Stereo Camera Calibrator
Simplified workflow estimating camera intrinsic and extrinsic parameters
Removes the effects of lens distortion from an image
Automatically detects checkerboard patterns
35
Point Cloud Processing in Matlab As of R2015a
Computer Vision System Toolbox (R2014b/15a) pointCloud
Object for storing a 3-D point cloud
pcdenoise
Remove noise from a 3-D point cloud
pcdownsample
Downsample a 3-D point cloud
pcmerge
Merge two 3-D point clouds
pcregrigd
Register two point clouds with ICP algorithm
pctransform
Rigid transform a 3-D point cloud
showPointCloud
Plot 3-D point cloud
Pcread
Read a 3-D point cloud from PLY file
Pcwrite
Write a 3-D point cloud to PLY file
alignColorToDepth
Align Kinect color image to depth image
depthToPointCloud
Convert Kinect depth image to a 3-D point cloud
Computational Geometry in base Matlab
Shipping Example: 3-D Point Cloud Registration and Stitching 37
Agenda • Introduction
• Feature Detect, Extract & Match
• Image Acquisition
• Object Detection and Tracking
• Explore-Enhance-Segment-Analyze
• 3D Vision Vision
• Image Registration
Demo: Camera Calibrator App Demo: 3D Reconstruction Demo: Depth Mapping
• Embedded Applications Demo: Ball Tracker • Embedded Applications Demo: Quad-copter Model
39
Embedded Applications
More than 60 Image Processing functions support code generation
Most Computer Vision functionality supports code generation
Many features generate platformindependent code
bwdist bwlookup bwmorph bwpack bwselect bwtraceboundary bwunpack conndef edge fitgeotrans fspecial getrangefromclass histeq im2double im2int16 im2single im2uint16 im2uint8 imadjust
imbothat imclearborder imclose imcomplement imdilate imerode imextendedmax imextendedmin imfill imfilter imhist imhmax imhmin imlincomb imopen imquantize imreconstruct imregionalmax imregionalmin
imtophat imwarp iptcheckmap iptcheckconn label2rgb mean2 medfilt2 multithresh ordfilt2 padarray rgb2ycbcr strel stretchlim ycbcr2rgb imref2d imref3d affine2d projective2d
40
Demo: Ball Tracker
41
Agenda
• Introduction
• Feature Detect, Extract & Match
• Image Acquisition
• Object Detection and Tracking
• Explore-Enhance-Segment-Analyze • 3D Vision
• Image Registration
• Embedded Applications
42
Explore-Enhance-Segment-Analyze
Feature Detection 3D Vision
Image Registration Object Detection and Tracking Apps
Image Acquisition
Image Viewer
Color Thresholder
Region Analyzer
Camera Calibrator
http://www.mathworks.co.uk/products/image-video-processing
43
© 2015 The MathWorks, Inc. 46