Vision Plugin - Simple Image Processing

The vision plugin hosts, among others, simple image processing functions, that are mainly used for backward compatibility reasons. Other, more powerful image processing functions can be found in the OpenCV plugin.

Color values are specified in RGB, with each component in [0;1].

simVision.addBuffer1ToWorkImg

Description Performs a pixel wise addition between the work image and buffer buffer1, and stores the result in the work image. See also simVision.addWorkImgToBuffer1 and simVision.subtractBuffer1FromWorkImg.
Lua synopsis simVision.addBuffer1ToWorkImg(number handle)
Lua parameters
handle: handle of the vision sensor, or sim.handle_self if the object attached to this script is the vision sensor.
Lua return values
none

simVision.addWorkImgToBuffer1

Description Performs a pixel wise addition between the work image and buffer buffer1, and stores the result in buffer buffer1. See also simVision.addBuffer1ToWorkImg and simVision.subtractWorkImgFromBuffer1.
Lua synopsis simVision.addWorkImgToBuffer1(number handle)
Lua parameters
handle: handle of the vision sensor, or sim.handle_self if the object attached to this script is the vision sensor.
Lua return values
none

simVision.binaryWorkImg

Description Transforms the work image into a binary image, with optional triggering.
Lua synopsis number trigger,string packedPacket=simVision.binaryWorkImg(number handle,number threshold,number oneProportion,number oneTol,number xCenter,number xCenterTol,number yCenter,number yCenterTol,number orient,number orientTol,number roundness,bool enableTrigger,table_3 overlayColor={1.0,0.0,1.0}))
Lua parameters
handle: handle of the vision sensor, or sim.handle_self if the object attached to this script is the vision sensor.
threshold: the threshold for binary 0 or binary 1.
oneProportion: the nominal binary 1 pixels relative amount (0-1), for triggering.
oneTol: the allowed tolerance for oneProportion.
xCenter: the nominal x center of binary 1 pixels (0-1), for triggering.
xCenterTol: the allowed tolerance for xCenter.
yCenter: the nominal y center of binary 1 pixels (0-1), for triggering.
yCenterTol: the allowed tolerance for yCenter.
orient: the nominal orientation, in radians, of the bounding box around binary 1 pixels (0-1), for triggering.
orientTol: the allowed tolerance for orient.
roundness: the nominal roundness value of the bounding box around binary 1 pixels (0-1), for triggering.
enableTrigger: if true and the triggering conditions are met, then return value trigger will be true.
overlayColor: an overlay color used to visualize triggering conditions. Set to nil for no overlay.
Lua return values
trigger: true if the filter has triggered.
packedPacket: a packed packet (use sim.unpackFloatTable to unpack) containing:
a) the binary image proportion value
b) the binary image posX value
c) the binary image posY value
d) the binary image angle value
e) the binary image roundness value

simVision.blobDetectionOnWorkImg

Description Performs blob detection on the work image.
Lua synopsis number trigger,string packedPacket=simVision.blobDetectionOnWorkImg(number handle,number threshold,number minBlobSize,bool diffColor,table_3 overlayColor={1.0,0.0,1.0}))
Lua parameters
handle: handle of the vision sensor, or sim.handle_self if the object attached to this script is the vision sensor.
threshold: the intensity threshold for a pixel to be considered on.
minBlobSize: the minimum size of a blob (in relation to the total image surface).
diffColor: if true then each blob will be represented in a different color.
overlayColor: an overlay color used to visualize the blobs. Set to nil for no overlay.
Lua return values
trigger: always false. Not used.
packedPacket: a packed packet (use sim.unpackFloatTable to unpack) containing:
a) the number of detected blobs
b) the number of values returned for each blob
then for each blob:
c.1) the blob relative size
c.2) the blob orientation
c.3) the blob relative position X
c.4) the blob relative position Y
c.5) the blob bounding box relative width
c.6) the blob bounding box relative height

simVision.buffer1ToWorkImg

Description Copies buffer buffer1 to the work image. See also simVision.buffer2ToWorkImg, simVision.workImgToBuffer1, simVision.swapBuffers and simVision.swapWorkImgWithBuffer1.
Lua synopsis simVision.buffer1ToWorkImg(number handle)
Lua parameters
handle: handle of the vision sensor, or sim.handle_self if the object attached to this script is the vision sensor.
Lua return values
none

simVision.buffer2ToWorkImg

Description Copies buffer buffer2 to the work image. See also simVision.buffer1ToWorkImg, simVision.workImgToBuffer2 and simVision.swapBuffers.
Lua synopsis simVision.buffer2ToWorkImg(number handle)
Lua parameters
handle: handle of the vision sensor, or sim.handle_self if the object attached to this script is the vision sensor.
Lua return values
none

simVision.changedPixelsOnWorkImg

Description Returns the pixels that changed the work image, from one frame to the next.
Lua synopsis number trigger,string packedPacket=simVision.changedPixelsOnWorkImg(number handle,number threshold)
Lua parameters
handle: handle of the vision sensor, or sim.handle_self if the object attached to this script is the vision sensor.
threshold: the minimum intensity variation for a pixel to have changed.
Lua return values
trigger: always false. Not used.
packedPacket: a packed packet (use sim.unpackFloatTable to unpack) containing for each changed pixel:
a.1) 1 if the pixel became brighter, -1 otherwise
a.2) the x coordinate of the pixel
a.3) the y coordinate of the pixel

simVision.circularCutWorkImg

Description Performs a circular cut on the work image.
Lua synopsis simVision.circularCutWorkImg(number handle,number radius,boolean toBuffer1)
Lua parameters
handle: handle of the vision sensor, or sim.handle_self if the object attached to this script is the vision sensor.
radius: the radius of the disc, expressed proportionally to the image resolution. Pixels outside of the disc's bounds are removed.
toBuffer1: if true, then the removed pixels are copied to buffer buffer1.
Lua return values
none

simVision.colorSegmentationOnWorkImg

Description Performs a color segmentation on the work image.
Lua synopsis simVision.colorSegmentationOnWorkImg(number handle,number colorDistance)
Lua parameters
handle: handle of the vision sensor, or sim.handle_self if the object attached to this script is the vision sensor.
colorDistance: the color distance to be used as threshold.
Lua return values
none

simVision.coordinatesFromWorkImg

Description Extracts relative coordinates from the work image (based on the view field of a vision sensor).
Lua synopsis number trigger,string packedPacket=simVision.coordinatesFromWorkImg(number handle,table_2 pointCount,boolean angularSpace)
Lua parameters
handle: handle of the vision sensor, or sim.handle_self if the object attached to this script is the vision sensor.
pointCount: the desired point count along X and Y.
angularSpace: whether the point sampling happens in the cartesian or angular space (within a given direction).
Lua return values
trigger: always false. Not used.
packedPacket: a packed packet (use sim.unpackFloatTable to unpack) containing:
a) the number of points along X
b) the number of points along Y
then for each points:
c.1) the relative x coordinate of the point
c.2) the relative y coordinate of the point
c.3) the relative z coordinate of the point
c.4) the distance to the point

simVision.edgeDetectionOnWorkImg

Description Applies an edge detection filter to the work image. See also simVision.matrix3x3OnWorkImg and simVision.matrix5x5OnWorkImg.
Lua synopsis simVision.edgeDetectionOnWorkImg(number handle,number threshold)
Lua parameters
handle: handle of the vision sensor, or sim.handle_self if the object attached to this script is the vision sensor.
threshold: the threshold value.
Lua return values
none

simVision.horizontalFlipWorkImg

Description Horizontally flips the work image. See also simVision.verticalFlipWorkImg.
Lua synopsis simVision.horizontalFlipWorkImg(number handle)
Lua parameters
handle: handle of the vision sensor, or sim.handle_self if the object attached to this script is the vision sensor.
Lua return values
none

simVision.intensityScaleOnWorkImg

Description Transforms the work image into an intensity representation.
Lua synopsis simVision.intensityScaleOnWorkImg(number handle,number start,number end,boolean greyscale)
Lua parameters
handle: handle of the vision sensor, or sim.handle_self if the object attached to this script is the vision sensor.
start: the value representing the minimum intensity.
end: the value representing the maximum intensity.
greyscale: if true, the output is a grey scale image, otherwise it is a color coded intensity image.
Lua return values
none

simVision.matrix3x3OnWorkImg

Description Applies a 3X3 matrix filter to the work image.
Lua synopsis simVision.matrix3x3OnWorkImg(number handle,number passes,number multiplier,table_9 matrix)
Lua parameters
handle: handle of the vision sensor, or sim.handle_self if the object attached to this script is the vision sensor.
passes: the number of desired filter passes.
multiplier: a matrix multiplier.
matrix: the 3X3 matrix.
Lua return values
none

simVision.matrix5x5OnWorkImg

Description Applies a 5X5 matrix filter to the work image.
Lua synopsis simVision.matrix5x5OnWorkImg(number handle,number passes,number multiplier,table_25 matrix)
Lua parameters
handle: handle of the vision sensor, or sim.handle_self if the object attached to this script is the vision sensor.
passes: the number of desired filter passes.
multiplier: a matrix multiplier.
matrix: the 5X5 matrix.
Lua return values
none

simVision.multiplyWorkImgWithBuffer1

Description Multiplies (pixel wise) the work image with buffer buffer1, and stores the result in the work image.
Lua synopsis simVision.multiplyWorkImgWithBuffer1(number handle)
Lua parameters
handle: handle of the vision sensor, or sim.handle_self if the object attached to this script is the vision sensor.
Lua return values
none

simVision.normalizeWorkImg

Description Normalizes the work image.
Lua synopsis simVision.normalizeWorkImg(number handle)
Lua parameters
handle: handle of the vision sensor, or sim.handle_self if the object attached to this script is the vision sensor.
Lua return values
none

simVision.rectangularCutWorkImg

Description Performs a rectangular cut on the work image.
Lua synopsis simVision.rectangularCutWorkImg(number handle,table_2 rectangle,boolean toBuffer1)
Lua parameters
handle: handle of the vision sensor, or sim.handle_self if the object attached to this script is the vision sensor.
rectangle: the x/y size of the rectangle, expressed proportionally to the image resolution. Pixels outside of the rectangle's bounds are removed.
toBuffer1: if true, then the removed pixels are copied to buffer buffer1.
Lua return values
none

simVision.resizeWorkImg

Description Scales the work image. The resolution remains same.
Lua synopsis simVision.resizeWorkImg(number handle,table_2 scaling)
Lua parameters
handle: handle of the vision sensor, or sim.handle_self if the object attached to this script is the vision sensor.
scaling: the x/y scaling.
Lua return values
none

simVision.rotateWorkImg

Description Performs a rotation of the work image.
Lua synopsis simVision.rotateWorkImg(number handle,number angle)
Lua parameters
handle: handle of the vision sensor, or sim.handle_self if the object attached to this script is the vision sensor.
angle: the rotation angle in radians.
Lua return values
none

simVision.scaleAndOffsetWorkImg

Description Shifts and scales colors of the work image, in the RGB or HSL space: colorOut=postOffset+(colorIn+preOffset)*scaling
Lua synopsis simVision.scaleAndOffsetWorkImg(number handle,table_3 colorPreOffset,table_3 colorScaling,table_3 colorPostOffset,boolean rgbSpace)
Lua parameters
handle: handle of the vision sensor, or sim.handle_self if the object attached to this script is the vision sensor.
colorPreOffset: the pre-offset, in RGB or HSL space.
colorScaling: the scaling, in RGB or HSL space.
colorPostOffset: the post-offset, in RGB or HSL space.
rgbSpace: if true, values are in the RGB space, otherwise they are in the HSL space.
Lua return values
none

simVision.selectiveColorOnWorkImg

Description Selects a specific color in the RGB or HSL space of the work image.
Lua synopsis simVision.selectiveColorOnWorkImg(number handle,table_3 nominalColor,table_3 colorTolerance,boolean rgbSpace,boolean keepColor,boolean toBuffer1)
Lua parameters
handle: handle of the vision sensor, or sim.handle_self if the object attached to this script is the vision sensor.
nominalColor: the nominal color to select, in RGB or HSL space.
colorTolerance: the tolerance around the nominal color, in RGB or HSL space.
rgbSpace: if true, values are in the RGB space, otherwise they are in the HSL space.
keepColor: if true, the specified color is kept, otherwise it is removed.
toBuffer1: if true, the removed color is copied to buffer buffer1.
Lua return values
none

simVision.sensorDepthMapToWorkImg

Description Copies the depth buffer acquired by a vision sensor to the work image (buffer used for image processing). see also simVision.sensorImgToWorkImg and simVision.workImgToSensorImg.
Lua synopsis simVision.sensorDepthMapToWorkImg(number handle)
Lua parameters
handle: handle of the vision sensor, or sim.handle_self if the object attached to this script is the vision sensor.
Lua return values
none

simVision.sensorImgToWorkImg

Description Copies the RGB image acquired by a vision sensor to the work image (buffer used for image processing). See also simVision.sensorDepthMapToWorkImg and simVision.workImgToSensorImg.
Lua synopsis simVision.sensorImgToWorkImg(number handle)
Lua parameters
handle: handle of the vision sensor, or sim.handle_self if the object attached to this script is the vision sensor.
Lua return values
none

simVision.sharpenWorkImg

Description Sharpens the work image. See also simVision.matrix3x3OnWorkImg and simVision.matrix5x5OnWorkImg.
Lua synopsis simVision.sharpenWorkImg(number handle)
Lua parameters
handle: handle of the vision sensor, or sim.handle_self if the object attached to this script is the vision sensor.
Lua return values
none

simVision.shiftWorkImg

Description Shifts the work image.
Lua synopsis simVision.shiftWorkImg(number handle,table_2 shift,boolean wrap)
Lua parameters
handle: handle of the vision sensor, or sim.handle_self if the object attached to this script is the vision sensor.
shift: the x/y shift amount, proportional to the image resolution (0=no shift, 1=shift by x/y resolution pixels).
wrap: whether parts shifted outside the image frame wrap around.
Lua return values
none

simVision.subtractBuffer1FromWorkImg

Description Subtracts (pixel wise subtraction) buffer buffer1 from the work image. See also simVision.subtractWorkImgFromBuffer1 and simVision.addBuffer1ToWorkImg.
Lua synopsis simVision.subtractBuffer1FromWorkImg(number handle)
Lua parameters
handle: handle of the vision sensor, or sim.handle_self if the object attached to this script is the vision sensor.
Lua return values
none

simVision.subtractWorkImgFromBuffer1

Description Subtracts (pixel wise subtraction) the work image from buffer buffer1. See also simVision.subtractBuffer1FromWorkImg and simVision.addWorkImgToBuffer1.
Lua synopsis simVision.subtractWorkImgFromBuffer1(number handle)
Lua parameters
handle: handle of the vision sensor, or sim.handle_self if the object attached to this script is the vision sensor.
Lua return values
none

simVision.swapBuffers

Description Swaps buffer buffer1 with buffer buffer2. See also simVision.swapWorkImgWithBuffer1.
Lua synopsis simVision.swapBuffers(number handle)
Lua parameters
handle: handle of the vision sensor, or sim.handle_self if the object attached to this script is the vision sensor.
Lua return values
none

simVision.swapWorkImgWithBuffer1

Description Swaps the work image with buffer buffer1. See also simVision.swapBuffers.
Lua synopsis simVision.swapWorkImgWithBuffer1(number handle)
Lua parameters
handle: handle of the vision sensor, or sim.handle_self if the object attached to this script is the vision sensor.
Lua return values
none

simVision.uniformImgToWorkImg

Description Applies a uniform color to the work image (buffer used for image processing)
Lua synopsis simVision.uniformImgToWorkImg(number handle,table_3 color)
Lua parameters
handle: handle of the vision sensor, or sim.handle_self if the object attached to this script is the vision sensor.
color: a table containing the RGB-triplet for the color (in the range of 0-1).
Lua return values
none

simVision.velodyneDataFromWorkImg

Description Extracts relative Velodyne coordinates from the work image (based on the view field of a vision sensor).
Lua synopsis number trigger,string packedPacket=simVision.velodyneDataFromWorkImg(number handle,table_2 pointCount,number verticalScanAngle)
Lua parameters
handle: handle of the vision sensor, or sim.handle_self if the object attached to this script is the vision sensor.
pointCount: the desired point count along X and Y.
verticalScanAngle: the vertical scan angle.
Lua return values
trigger: always false. Not used.
packedPacket: a packed packet (use sim.unpackFloatTable to unpack) containing:
a) the number of points along X
b) the number of points along Y
then for each points:
c.1) the relative x coordinate of the point
c.2) the relative y coordinate of the point
c.3) the relative z coordinate of the point
c.4) the distance to the point

simVision.verticalFlipWorkImg

Description Vertically flips the work image. See also simVision.horizontalFlipWorkImg.
Lua synopsis simVision.verticalFlipWorkImg(number handle)
Lua parameters
handle: handle of the vision sensor, or sim.handle_self if the object attached to this script is the vision sensor.
Lua return values
none

simVision.workImgToBuffer1

Description Copies the work image to buffer buffer1. See also simVision.workImgToBuffer2, simVision.buffer1ToWorkImg, simVision.swapBuffers and simVision.swapWorkImgWithBuffer1.
Lua synopsis simVision.workImgToBuffer1(number handle)
Lua parameters
handle: handle of the vision sensor, or sim.handle_self if the object attached to this script is the vision sensor.
Lua return values
none

simVision.workImgToBuffer2

Description Copies the work image to buffer buffer2. See also simVision.workImgToBuffer1, simVision.buffer2ToWorkImg and simVision.swapBuffers.
Lua synopsis simVision.workImgToBuffer2(number handle)
Lua parameters
handle: handle of the vision sensor, or sim.handle_self if the object attached to this script is the vision sensor.
Lua return values
none

simVision.workImgToSensorDepthMap

Description Copies the work image (buffer used for image processing) to a vision sensor's depth buffer (each depth buffer pixel will be averaged from the image RGB-triplet). See also simVision.workImgToSensorImg.
Lua synopsis simVision.workImgToSensorDepthMap(number handle)
Lua parameters
handle: handle of the vision sensor, or sim.handle_self if the object attached to this script is the vision sensor.
Lua return values
none

simVision.workImgToSensorImg

Description Copies the work image (buffer used for image processing) to a vision sensor's RGB buffer. See also simVision.sensorImgToWorkImg.
Lua synopsis simVision.workImgToSensorImg(number handle)
Lua parameters
handle: handle of the vision sensor, or sim.handle_self if the object attached to this script is the vision sensor.
Lua return values
none