Features Package

Features Package

BOFFeatureExtractor Module

class SimpleCV.Features.BOFFeatureExtractor.BOFFeatureExtractor(patchsz=(11, 11), numcodes=128, imglayout=(8, 16), padding=0)

Bases: object

For a discussion of bag of features please see: http://en.wikipedia.org/wiki/Bag_of_words_model_in_computer_vision

Initialize the bag of features extractor. This assumes you don’t have the feature codebook pre-computed. patchsz = the dimensions of each codebook patch numcodes = the number of different patches in the codebook. imglayout = the shape of the resulting image in terms of patches padding = the pixel padding of each patch in the resulting image.

extract(img)

This method extracts a bag of features histogram for the input image using the provided codebook. The result are the bin counts for each codebook code.

extractPatches(img, sz=(11, 11))

Get patches from a single images. This is an external access method. The user will need to maintain the list of features. See the generate method as a guide to doing this by hand. Sz is the image patch size.

generate(imgdirs, numcodes=128, sz=(11, 11), imgs_per_dir=50, img_layout=(8, 16), padding=0, verbose=True)

This method builds the bag of features codebook from a list of directories with images in them. Each directory should be broken down by image class. imgdirs = This list of directories. patchsz = the dimensions of each codebook patch numcodes = the number of different patches in the codebook. imglayout = the shape of the resulting image in terms of patches padding = the pixel padding of each patch in the resulting image. imgs_per_dir = this method can use a specified number of images per directory

i.e. min(#imgs_in_dir,imgs_per_dir)

verbose = print output

Once the method has completed it will save the results to a local file using the file name codebook.png

WARNING: THIS METHOD WILL TAKE FOREVER

getFieldNames()

This method gives the names of each field in the feature vector in the order in which they are returned. For example, ‘xpos’ or ‘width’

getNumFields()

This method returns the total number of fields in the feature vector.

load(datafile)

Load a codebook from file using the datafile. The datafile should point to a local image for the source patch image.

makeCodebook(featureStack, ncodes=128)

This method will return the centroids of the k-means analysis of a large number of images. Ncodes is the number of centroids to find.

reconstruct(img)

This is a “just for fun” method as a sanity check for the BOF codeook. The method takes in an image, extracts each codebook code, and replaces the image at the position with the code.

save(imgfname, datafname)

Save the bag of features codebook and data set to a local file.

Blob Module

class SimpleCV.Features.Blob.Blob

Bases: SimpleCV.Features.Features.Feature

A blob is a typicall a cluster of pixels that form a feature or unique shape that allows it to be distinguished from the rest of the image Blobs typically are computed very quickly so they are used often to find various items in a picture based on properties. Typically these things like color, shape, size, etc. Since blobs are computed quickly they are typically used to narrow down search regions in an image, where you quickly find a blob and then that blobs region is used for more computational intensive type image processing

above(blob)

Given a point or another blob determine if this blob is above the other blob

angle()

This method returns the angle between the horizontal and the minimum enclosing rectangle of the blob. The minimum enclosing rectangle IS NOT not the bounding box. Use the bounding box for situations where you need only an approximation of the objects dimensions. The minimum enclosing rectangle is slightly harder to maninpulate but gives much better information about the blobs dimensions.

area()

This method returns the area of the blob in terms of the number of pixels inside the contour.

aspectRatio()

This method returns the aspect ration (W/H) of the bounding box of the blob.

below(blob)

Given a point or another blob determine if this blob is below the other blob

bottomLeftCorner()

This method returns the bottom left corner of the bounding box of the blob as an (x,y) tuple.

bottomRightCorner()

This method returns the bottom right corner of the bounding box of the blob as an (x,y) tuple.

center()

This mehtod returns the center of the blob’s bounding box

Returns:

Tuple

centroid()

Return the centroid (mass-determined center) of the blob

circleDistance()

Compare the hull mask to an ideal circle and count the number of pixels that deviate as a fraction of total area of the ideal circle

contains(other)

Returns true if this blob contains the point, all of a collection of points, or the entire other blo in other

draw(color=(0, 128, 0), alpha=-1, width=-1, layer=None)

Draw the blob, in the given color, to the appropriate layer

By default, this draws the entire blob filled in, with holes. If you provide a width, an outline of the exterior and interior contours is drawn.

color = The color to render the blob. alpha = The alpha value of the rendered blob. width = The width of the drawn blob in pixels, if -1 then filled then the polygon is filled. layer = if layer is not None, the blob is rendered to the layer versus the source image.

Parameters:

color - Color object or Color tuple alpha - Int width - Int layer - DrawingLayer

drawHoles(color=(0, 128, 0), alpha=-1, width=-1, layer=None)

This method renders all of the holes (if any) that are present in the blob

color = The color to render the blob’s holes. alpha = The alpha value of the rendered blob hole. width = The width of the drawn blob hole in pixels, if w=-1 then the polygon is filled. layer = if layer is not None, the blob is rendered to the layer versus the source image.

Parameters:

color - Color object or Color tuple alpha - Int width - Int layer - DrawingLayer

drawHull(color=(0, 128, 0), alpha=-1, width=-1, layer=None)

Draw the blob’s convex hull to either the source image or to the specified layer given by layer.

color = The color to render the blob’s convex hull. alpha = The alpha value of the rendered blob. width = The width of the drawn blob in pixels, if w=-1 then the polygon is filled. layer = if layer is not None, the blob is rendered to the layer versus the source image.

Parameters:

color - Color object or Color tuple alpha - Int width - Int layer - DrawingLayer

drawMaskToLayer(layer=None, offset=(0, 0))

Draw the actual pixels of the blob to another layer. This is handy if you want to examine just the pixels inside the contour.

offset = The offset from the top left corner where we want to place the mask.

Parameters:

layer - DrawingLayer offset - Tuple

drawOutline(color=(0, 128, 0), alpha=-1, width=-1, layer=None)

Draw the blob contour the given layer – if no layer is provided, draw to the source image

color = The color to render the blob. alpha = The alpha value of the rendered blob. width = The width of the drawn blob in pixels, -1 then the polygon is filled. layer = if layer is not None, the blob is rendered to the layer versus the source image.

Parameters:

color - Color object or Color tuple alpha - Int width - Int layer - DrawingLayer

height()

This method returs the height of the bounding box of the blob.

hullRadius()

Return the radius of the convex hull contour from the centroid

isCircle(tolerance=0.050000000000000003)

Test circlde distance against a tolerance to see if the blob is circlular

isRectangle(tolerance=0.050000000000000003)

Given a tolerance, test the blob against the rectangle distance to see if it is rectangular

Parameters:

tolerance - Float

isSquare(tolerance=0.050000000000000003, ratiotolerance=0.050000000000000003)

Given a tolerance, test if the blob is a rectangle, and how close its bounding rectangle’s aspect ratio is to 1.0

Parameters:

tolerance - Float ratiotolerance - Float

left(blob)

Given a point or another blob determine if this blob is to the left of the other blob

length()

Length returns the longest dimension of the X/Y bounding box

match(otherblob)

Compare the Hu moments between two blobs to see if they match. Returns a comparison factor – lower numbers are a closer match

maxX()

This method return the maximum x value of the bounding box of the the blob.

maxY()

This method return the maximum y value of the bounding box of the the blob.

meanColor()

This function returns a tuple representing the average color of the blob

Returns:

Tuple

minRectHeight()

This is the y coordinate of the centroid for the minimum bounding rectangle

minRectWidth()

This is the y coordinate of the centroid for the minimum bounding rectangle

minRectX()

This is the x coordinate of the centroid for the minimum bounding rectangle

minRectY()

This is the y coordinate of the centroid for the minimum bounding rectangle

minX()

This method return the minimum x value of the bounding box of the the blob.

minY()

This method return the minimum y value of the bounding box of the the blob.

overlaps(other)

Returns true if this blob contains at least one point, part of a collection of points, or any part of a blob.

radius()

Return the radius, the avg distance of each contour point from the centroid

rectangleDistance()

This compares the hull mask to the bounding rectangle. Returns the area of the blob’s hull as a fraction of the bounding rectangle

rectifyMajorAxis(axis=0)

Rectify the blob image and the contour such that the major axis is aligned to either vertical=0 or horizontal=1

right(blob)

Given a point or another blob determine if this blob is to the right of the other blob

rotate(angle)

Rotate the blob given the angle in degrees most of the blob elements will be rotated, not however this will “break” drawing back to the original image. To draw the blob create a new layer and draw to that layer.

Parameters:

angle - Int or Float

topLeftCorner()

This method returns the top left corner of the bounding box of the blob as an (x,y) tuple.

topRightCorner()

This method returns the top right corner of the bounding box of the blob as an (x,y) tuple.

width()

This method returns the width of the bounding box of the blob.

BlobMaker Module

class SimpleCV.Features.BlobMaker.BlobMaker

Blob maker encapsulates all of the contour extraction process and data, so it can be used inside the image class, or extended and used outside the image class. The general idea is that the blob maker provides the utilites that one would use for blob extraction. Later implementations may include tracking and other features.

extract(img, threshval=127, minsize=10, maxsize=0, threshblocksize=3, threshconstant=5)

This method performs a threshold operation on the input image and then extracts and returns the blobs. img - The input image (color or b&w) threshval - The threshold value for the binarize operation. If threshval = -1 adaptive thresholding is used minsize - The minimum blob size in pixels. maxsize - The maximum blob size in pixels. 0=uses the default value. threshblocksize - The adaptive threhold block size. threshconstant - The minimum to subtract off the adaptive threshold

extractFromBinary(binaryImg, colorImg, minsize=5, maxsize=-1)

This method performs blob extraction given a binary source image that is used to get the blob images, and a color source image. binaryImg- The binary image with the blobs. colorImg - The color image. minSize - The minimum size of the blobs in pixels. maxSize - The maximum blob size in pixels.

extractUsingModel(img, colormodel, minsize=10, maxsize=0)

Extract blobs using a color model img - The input image colormodel - The color model to use. minsize - The minimum size of the returned features. maxsize - The maximum size of the returned features 0=uses the default value.

Parameters:

img - Image colormodel - ColorModel object minsize - Int maxsize - Int

Detection Module

class SimpleCV.Features.Detection.Barcode(i, zxbc)

Bases: SimpleCV.Features.Features.Feature

The Barcode Feature wrappers the object returned by findBarcode(), a python-zxing object.

• The x,y coordinate is the center of the code
• points represents the four boundary points of the feature. Note: for QR codes, these points are the reference rectangles, and are quadrangular, rather than rectangular with other datamatrix types.
• data is the parsed data of the code

area()

Returns the area defined by the quandrangle formed by the boundary points

draw(color=(255, 0, 0))

Draws the bounding area of the barcode, given by points. Note that for QR codes, these points are the reference boxes, and so may “stray” into the actual code.

length()

Returns the longest side of the quandrangle formed by the boundary points

class SimpleCV.Features.Detection.Chessboard(i, dim, subpixelCorners)

Bases: SimpleCV.Features.Features.Feature

This class is used for Calibration, it uses a chessboard to calibrate from pixels to real world measurements

area()

Returns the mean of the distance between corner points in the chessboard Given that the chessboard is of a known size, this can be used as a proxy for distance from the camera

draw(no_needed_color=None)

Draws the chessboard corners. We take a color param, but ignore it

class SimpleCV.Features.Detection.Corner(i, at_x, at_y)

Bases: SimpleCV.Features.Features.Feature

The Corner feature is a point returned by the FindCorners function Corners are used in machine vision as a very computationally low way to find unique features in an image. These corners can be used in conjunction with many other algorithms.

draw(color=(255, 0, 0))

Draw a small circle around the corner. Color tuple is single parameter, default Red

class SimpleCV.Features.Detection.HaarFeature(i, haarobject, haarclassifier=None)

Bases: SimpleCV.Features.Features.Feature

The HaarFeature is a rectangle returned by the FindHaarFeature() function.

• The x,y coordinates are defined by the center of the bounding rectangle
• the classifier property refers to the cascade file used for detection
• points are the clockwise points of the bounding rectangle, starting in upper left

angle()

Returns the angle of the rectangle – horizontal if wide, vertical if tall

area()

Returns the area contained within the HaarFeature’s bounding rectangle

draw(color=(0, 255, 0))

Draw the bounding rectangle, default color green

height()

Get the height of the line

length()

Returns the longest dimension of the HaarFeature, either width or height

meanColor()

Find the mean color of the boundary rectangle

width()

Get the width of the line.

class SimpleCV.Features.Detection.Line(i, line)

Bases: SimpleCV.Features.Features.Feature

The Line class is returned by the findLines function, but can also be initialized with any two points:

l = Line(Image, point1, point2) Where point1 and point2 are coordinate tuples

l.points will be a tuple of the two points

angle()

This is the angle of the line, from the leftmost point to the rightmost point Returns angle (theta) in radians, with 0 = horizontal, -pi/2 = vertical positive slope, pi/2 = vertical negative slope

draw(color=(0, 0, 255))

Draw the line, default color is blue

length()

Compute the length of the line

meanColor()

Returns the mean color of pixels under the line. Note that when the line falls “between” pixels, each pixels color contributes to the weighted average.

class SimpleCV.Features.Detection.TemplateMatch(image, template, location, quality)

Bases: SimpleCV.Features.Features.Feature

This class is used for template (pattern) matching in images The template matching cannot handle scale or rotation

draw(color=(0, 128, 0))

EdgeHistogramFeatureExtractor Module

class SimpleCV.Features.EdgeHistogramFeatureExtractor.EdgeHistogramFeatureExtractor(bins=10)

Bases: SimpleCV.Features.FeatureExtractorBase.FeatureExtractorBase

Create a 1D edge length histogram and 1D edge angle histogram.

This method takes in an image, applies an edge detector, and calculates the length and direction of lines in the image.

bins = the number of bins

extract(img)

Extract the line orientation and and length histogram.

getFieldNames()

Return the names of all of the length and angle fields.

getNumFields()

This method returns the total number of fields in the feature vector.

FeatureExtractorBase Module

class SimpleCV.Features.FeatureExtractorBase.FeatureExtractorBase

Bases: object

The featureExtractorBase class is a way of abstracting the process of collecting descriptive features within an image. A feature is some description of the image like the mean color, or the width of a center image, or a histogram of edge lengths. This feature vectors can then be composed together and used within a machine learning algorithm to descriminate between different classes of objects.

extract(img)

Given an image extract the feature vector. The output should be a list object of all of the features. These features can be of any interal type (string, float, integer) but must contain no sub lists.

getFieldNames()

This method gives the names of each field in the feature vector in the order in which they are returned. For example, ‘xpos’ or ‘width’

getNumFields()

This method returns the total number of fields in the feature vector.

classmethod load(fname)

load segmentation settings to file.

save(fname)

Save segmentation settings to file.

Features Module

class SimpleCV.Features.Features.Feature(i, at_x, at_y)

Bases: object

The Feature object is an abstract class which real features descend from. Each feature object has:

• a draw() method,
• an image property, referencing the originating Image object
• x and y coordinates
• default functions for determining angle, area, meanColor, etc for FeatureSets
• in the Feature class, these functions assume the feature is 1px

angle()

Return the angle (theta) of the feature – default 0 (horizontal)

area()

Area covered by the feature – for a pixel, 1

colorDistance(color=(0, 0, 0))

Return the euclidean color distance of the color tuple at x,y from a given color (default black)

coordinates()

Return a an array of x,y

crop()

This function returns the largest bounding box for an image.

Returns Image

distanceFrom(point=(-1, -1))

Given a point (default to center of the image), return the euclidean distance of x,y from this point

draw(color=(0, 128, 0))

With no dimension information, color the x,y point for the featuer

height()

Height of the feature – defaults to 1

length()

Longest dimension of the feature – for a pixel, 1

meanColor()

Return the color tuple from x,y

show(color=(0, 128, 0))

This function will automatically draw the features on the image and show it. It is a basically a shortcut function for development and is the same as:

>>> img = Image("logo")
>>> feat = img.findBlobs()
>>> if feat: feat.draw()
>>> img.show()

width()

Width of the feature – defaults to 1

class SimpleCV.Features.Features.FeatureSet

Bases: list

FeatureSet is a class extended from Python’s list which has special functions so that it is useful for handling feature metadata on an image.

In general, functions dealing with attributes will return numpy arrays, and functions dealing with sorting or filtering will return new FeatureSets.

Example: >>> image = Image(“/path/to/image.png”) >>> lines = image.findLines() #lines are the feature set >>> lines.draw() >>> lines.x() >>> lines.crop()

angle()

Return a numpy array of the angles (theta) of each feature. Note that theta is given in radians, with 0 being horizontal.

area()

Returns a numpy array of the area of each feature in pixels.

colorDistance(color=(0, 0, 0))

Return a numpy array of the distance each features average color is from a given color tuple (default black, so colorDistance() returns intensity)

coordinates()

Returns a 2d numpy array of the x,y coordinates of each feature. This is particularly useful if you want to use Scipy’s Spatial Distance module

crop()

Returns a nparray with the cropped features as Imges

distanceFrom(point=(-1, -1))

Returns a numpy array of the distance each Feature is from a given coordinate. Default is the center of the image.

distancePairs()

Returns the square-form of pairwise distances for the featureset. The resulting N x N array can be used to quickly look up distances between features.

draw(color=(0, 128, 0), autocolor=False)

Call draw() on each feature in the FeatureSet.

filter(filterarray)

Return a FeatureSet which is filtered on a numpy boolean array. This will let you use the attribute functions to easily screen Features out of return FeatureSets.

Some examples:

Return all lines < 200px >>> my_lines.filter(my_lines.length() < 200) # returns all lines < 200px >>> my_blobs.filter(my_blobs.area() > 0.9 * my_blobs.length**2) # returns blobs that are nearly square >>> my_lines.filter(abs(my_lines.angle()) < numpy.pi / 4) #any lines within 45 degrees of horizontal >>> my_corners.filter(my_corners.x() - my_corners.y() > 0) #only return corners in the upper diagonal of the image

height()

Returns a nparray which is the height of all the objects in the FeatureSet

length()

Return a numpy array of the length (longest dimension) of each feature.

meanColor()

Return a numpy array of the average color of the area covered by each Feature.

show(color=(0, 128, 0), autocolor=False)

This function will automatically draw the features on the image and show it. It is a basically a shortcut function for development and is the same as:

>>> img = Image("logo")
>>> feat = img.findBlobs()
>>> if feat: feat.draw()
>>> img.show()

sortAngle(theta=0)

Return a sorted FeatureSet with the features closest to a given angle first. Note that theta is given in radians, with 0 being horizontal.

sortArea()

Returns a new FeatureSet, with the largest area features first.

sortColorDistance(color=(0, 0, 0))

Return a sorted FeatureSet with features closest to a given color first. Default is black, so sortColorDistance() will return darkest to brightest

sortDistance(point=(-1, -1))

Returns a sorted FeatureSet with the features closest to a given coordinate first. Default is from the center of the image.

sortLength()

Return a sorted FeatureSet with the longest features first.

width()

Returns a nparray which is the width of all the objects in the FeatureSet

x()

Returns a numpy array of the x (horizontal) coordinate of each feature.

y()

Returns a numpy array of the y (vertical) coordinate of each feature.

HaarLikeFeature Module

class SimpleCV.Features.HaarLikeFeature.HaarLikeFeature(name=None, regions=None)

Create a single Haar feature and optionally set the regions that define the Haar feature and its name. The formal of the feature is

The format is [[[TL],[BR],SIGN],[[TL],[BR],SIGN].....] Where TR and BL are the unit coorinates for the top right and bottom left coodinates.

For example [[[0,0],[0.5,0.5],1],[[0.5.0],[1.0,1.0],-1]]

Takes the right side of the image and subtracts from the left hand side of the image.

apply(intImg)

This method takes in an integral image and applies the haar-cascade to the image, and returns the result.

setName(name)

Set the name of this feature, the name must be unique.

setRegions(regions)

Set the list of regions. The regions are square coordinates on a unit sized image followed by the sign of a region.

The format is [[[TL],[BR],SIGN],[[TL],[BR],SIGN].....] Where TR and BL are the unit coorinates for the top right and bottom left coodinates.

For example [[[0,0],[0.5,0.5],1],[[0.5.0],[1.0,1.0],-1]]

Takes the right side of the image and subtracts from the left hand side of the image.

writeToFile(file)

Write the Haar cascade to a human readable file. file is an open file pointer.

HaarLikeFeatureExtractor Module

class SimpleCV.Features.HaarLikeFeatureExtractor.HaarLikeFeatureExtractor(fname=None, do45=True)

Bases: SimpleCV.Features.FeatureExtractorBase.FeatureExtractorBase

This is used generate Haar like features from an image. These Haar like features are used by a the classifiers of machine learning to help identify objects or things in the picture by their features, or in this case haar features.

For a more in-depth review of Haar Like features see: http://en.wikipedia.org/wiki/Haar-like_features

extract(img)

This extractor takes in an image, creates the integral image, applies the Haar cascades, and returns the result as a feature vector.

getFieldNames()

This method gives the names of each field in the feature vector in the order in which they are returned. For example, ‘xpos’ or ‘width’

getNumFields()

This method returns the total number of fields in the feature vector.

readWavelets(fname, nfeats=-1)

fname = file name nfeats = number of features to load from file -1 -> All features

saveWavelets(fname)

Save wavelets to file

HueHistogramFeatureExtractor Module

class SimpleCV.Features.HueHistogramFeatureExtractor.HueHistogramFeatureExtractor(mNBins=16)

Bases: SimpleCV.Features.FeatureExtractorBase.FeatureExtractorBase

Create a Hue Histogram feature extractor. This feature extractor takes in an image, gets the hue channel, bins the number of pixels with a particular Hue, and returns the results.

mNBins - the number of Hue bins.

extract(img)

This feature extractor takes in a color image and returns a normalized color histogram of the pixel counts of each hue.

getFieldNames()

This method gives the names of each field in the feature vector in the order in which they are returned. For example, ‘xpos’ or ‘width’

getNumFields()

This method returns the total number of fields in the feature vector.

MorphologyFeatureExtractor Module

class SimpleCV.Features.MorphologyFeatureExtractor.MorphologyFeatureExtractor(thresholdOperation=None)

Bases: SimpleCV.Features.FeatureExtractorBase.FeatureExtractorBase

This feature extractor collects some basic morphology infromation about a given image. It is assumed that the object to be recognized is the largest object in the image. The user must provide a segmented white on black blob image. This operation then straightens the image and collects the data.

extract(img)

This method takes in a image and returns some basic morphology characteristics about the largest blob in the image. The if a color image is provided the threshold operation is applied.

getFieldNames()

This method gives the names of each field in the feature vector in the order in which they are returned. For example, ‘xpos’ or ‘width’

getNumFields()

This method returns the total number of fields in the feature vector.

setThresholdOperation(threshOp)

The threshold operation is a function of the form binaryimg = threshold(img)

the simplest example would be: def binarize_wrap(img):

return img.binarize()