Image Processing and Computer Vision with MATLAB and SIMULINK [PDF]

Image Processing and Computer Vision with MATLAB and SIMULINK

By Dr Jasmina Lazić Application Engineering, MathWorks

© 2015 The MathWorks, Inc. 1

The Computer Vision Ecosystem Simulink™ Embedded Coder™

Neural Network Toolbox™

Robotic Vision

Control System Toolbox™

Control Robotics

Computer Intelligence

Artificial Intelligence

Computer Vision System Toolbox™ Cognitive Vision

Machine Learning

Multi-variable SP Non-linear SP

Signal Processing Toolbox™

Signal Processing Computer Vision

Machine Vision

Physics

DSP System Toolbox™

SimScape™

Optics

SimMechanics™ Statistics Toolbox™

Statistics

Mathematics

Image Processing

Imaging

Geometry

MATLAB™

Optimization

Optimization Toolbox™

Neurobiology Biological Vision

SimBiology™

Smart Cameras

Image Acquisition Toolbox™

Image Processing Toolbox™ 2

Applications: Image and Video Processing    

   

Medical imaging Surveillance Robotics Automotive safety Consumer electronics Geospatial computing Machine vision and more…

3

Agenda

• Introduction

• Feature Detect, Extract & Match

• Image Acquisition

• Object Detection and Tracking

• Explore-Enhance-Segment-Analyze • 3D Vision

• Image Registration

• Embedded Applications

4

http://www.mathworks.co.uk/products/image-video-processing

5

Agenda • Introduction

• Introduction

• Feature Detect, Extract & Match

• Image Acquisition

 Demo: MATLAB live video with Microsoft Kinect • Object Detection and Tracking • Image Acquisition 

Demo: Image Acquisition App

• Explore-Enhance-Segment-Analyze • 3D Vision

• Image Registration

• Embedded Applications

6

Image Acquisition 

Acquire images and video directly into MATLAB and Simulink



Configure device properties

7

Agenda • Introduction

• Feature Detect, Extract & Match • Image Image Acquisition Acquisition   

Demo: Live video in MATLAB Demo: Image Acquisition App Demo: Live video in SIMULINK

• Object Detection and Tracking

• Explore-Enhance-Segment-Analyze  Demo: Tissue segmentation • Explore-Enhance-Segment-Analyze • 3D Vision  Demo: Image analysis Apps 

Demo: Digital display reader

• Image Registration

• Embedded Applications

8

Standard Segmentation Workflow

9

Standard Segmentation Workflow 

Explore image

10

Standard Segmentation Workflow  

Explore image Enhance image

11

Standard Segmentation Workflow   

Explore image Enhance image Segment image

12

Standard Segmentation Workflow    

Explore image Enhance image Segment image Analyze image

13

Demos: Explore-Enhance-Segment-Analyze

14

Image Processing Apps NEW! NEW!

Image Viewer Color Thresholder Region Analyzer

15

Optical Character Recognition



Support for English, Japanese + others



Simple interface: >> results = ocr(I)

16

Segmentation – other techniques Watershed and Active Contours (NEW)

17

Agenda • Introduction

• Feature Detect, Extract & Match • Image Acquisition

• Object Detection and Tracking • Explore-Enhance-Segment-Analyze Explore-Enhance-Segment-Analyze   

Demo: Tissue segmentation Demo: Image analysis Apps Demo: Digital display reader

• 3D Vision

• Image Registration  Demo: Video Mosaicking (Simulink) • Image Registration

• Embedded Applications

18

Image Registration  

2D and 3D Align multi-model images

19

Image Registration 

NEW: Non-rigid image registration and warping

20

Demo: Video Mosaicking

21

Agenda • Introduction

• Feature Detect, Extract & Match  Geometry Estimation • Feature Detect, Extract & Match

• Image Acquisition

• Object Detection and Tracking • Explore-Enhance-Segment-Analyze

• 3D Vision • Image Image Registration Registration 

Demo: Video Mosaicking (Simulink)

• Embedded Applications

22

Feature Detection, Extraction and Matching

Edge

Corner

Template

SURF

MSER

23

Point Tracker



Simple interface:

points = detectXFeatures(I); tracker = vision.PointTracker; initialize(tracker, points.Location, frame); while ~finished frame = step(videoFileReader); [points, validity] = step(tracker, frame); end

24

Geometry Estimation  

RANSAC – Random Sampling and Consensus Gets geometry from a minimal sample set and checks it against all matches

25

Agenda • Introduction

• Feature Feature Detect, Detect,Extract Extract&&Match Match 

• Image Acquisition

Geometry Estimation

• Object Detection and Tracking  Demo: Pedestrianand Tracking • Object Detection Tracking

• Explore-Enhance-Segment-Analyze

• 3D Vision • Image Registration

• Embedded Applications

26

Object Detection and Tracking

27

Face Detection, People Detection and Foreground Detection

28

Training Image Labeler App

29

Demo: Tracking multiple objects with a Kalman Filter

30

Demo: Tracking multiple objects with a Kalman Filter 

Kalman Filter interface: – Configure: kalmanFilter = configureKalmanFilter( ... 'ConstantVelocity', centroid, ... [200, 50], [100, 25], 100); – Predict: predictedCentroid = predict(kalmanFilter); – Update: c2 = correct(kalmanFilter, centroid); 31

Bag of Words for Category Classification 1. Create Bag of Features

2. Encode Input Images

3. Repeat for all classes

4. Use machine learning algorithm to discriminate between classes

32

Agenda • Introduction

• Feature Detect, Extract & Match

• Image Acquisition

• Object Object Detection Detectionand andTracking Tracking 

• Explore-Enhance-Segment-Analyze

Demo: Multi-object Tracking

• 3D Vision  Demo: Camera Calibrator App • 3D Vision  Demo: 3D Reconstruction 

Demo: Depth Mapping

• Image Registration

• Embedded Applications

33

3D Vision Pyramid with dynamic programming and sub-pixel accuracy 15

2.5 2 1.5 1 10

y (meters)

0.5 0 -0.5 -1 5

-1.5 -2 -2.5

-8 0

-6 -4 3 z (meters)

2

1

0

-1

-2

-3

-4

x (meters)

34

NEW Apps: Camera Calibrator and Stereo Camera Calibrator 

Simplified workflow estimating camera intrinsic and extrinsic parameters



Removes the effects of lens distortion from an image



Automatically detects checkerboard patterns

35

Point Cloud Processing in Matlab As of R2015a 

Computer Vision System Toolbox (R2014b/15a) pointCloud

Object for storing a 3-D point cloud

pcdenoise

Remove noise from a 3-D point cloud

pcdownsample

Downsample a 3-D point cloud

pcmerge

Merge two 3-D point clouds

pcregrigd

Register two point clouds with ICP algorithm

pctransform

Rigid transform a 3-D point cloud

showPointCloud

Plot 3-D point cloud

Pcread

Read a 3-D point cloud from PLY file

Pcwrite

Write a 3-D point cloud to PLY file

alignColorToDepth

Align Kinect color image to depth image

depthToPointCloud

Convert Kinect depth image to a 3-D point cloud



Computational Geometry in base Matlab



Shipping Example: 3-D Point Cloud Registration and Stitching 37

Agenda • Introduction

• Feature Detect, Extract & Match

• Image Acquisition

• Object Detection and Tracking

• Explore-Enhance-Segment-Analyze

• 3D Vision Vision   

• Image Registration

Demo: Camera Calibrator App Demo: 3D Reconstruction Demo: Depth Mapping

• Embedded Applications  Demo: Ball Tracker • Embedded Applications  Demo: Quad-copter Model

39

Embedded Applications 

More than 60 Image Processing functions support code generation



Most Computer Vision functionality supports code generation



Many features generate platformindependent code

bwdist bwlookup bwmorph bwpack bwselect bwtraceboundary bwunpack conndef edge fitgeotrans fspecial getrangefromclass histeq im2double im2int16 im2single im2uint16 im2uint8 imadjust

imbothat imclearborder imclose imcomplement imdilate imerode imextendedmax imextendedmin imfill imfilter imhist imhmax imhmin imlincomb imopen imquantize imreconstruct imregionalmax imregionalmin

imtophat imwarp iptcheckmap iptcheckconn label2rgb mean2 medfilt2 multithresh ordfilt2 padarray rgb2ycbcr strel stretchlim ycbcr2rgb imref2d imref3d affine2d projective2d

40

Demo: Ball Tracker

41

Agenda

• Introduction

• Feature Detect, Extract & Match

• Image Acquisition

• Object Detection and Tracking

• Explore-Enhance-Segment-Analyze • 3D Vision

• Image Registration

• Embedded Applications

42

Explore-Enhance-Segment-Analyze

Feature Detection 3D Vision

Image Registration Object Detection and Tracking Apps

Image Acquisition

Image Viewer

Color Thresholder

Region Analyzer

Camera Calibrator

http://www.mathworks.co.uk/products/image-video-processing

43

© 2015 The MathWorks, Inc. 46