DeliveryBot

This activity is the where communication with ROS is established, the Talker and Listener nodes are initialized, and the image processing occurs. To allow communication with ROS as well as basic layout functionality, this activity extends RosActivity. Furthermore, for the image processing purposes, RobotController implements CameraBridgeViewBase.CvCameraViewListener2 for a constant stream of image frames. This implementation is from the OpenCV library.

This activity is written entirely in Java.

Public Parameters

As this activity contains the manipulation of many public parameters, they will be mentioned here to avoid redundancy.

• TextView ultrasonicReading: the TextView associated with the ultrasonic distance readings.
• TextView robotState: the TextView that lets the user know if it is blocked or not.
• Talker talkerNode: an instance of the Talker class. This instance is used to publish commands to ROS when deemed necessary.
• boolean sawRedBall: a flag that indicates if the robot saw the red ball or not.
• boolean sawYellowBall: a flag that indicates if the robot saw the yellow ball or not.
• boolean robotWon: a flag that indicates if the robot has won or not.
• String TAG: the activity's name tag.
• JavaCameraView javaCameraView: the camera view instance. This is what allows the frame stream input.
• Mat imgHSV, mYellowThresh, mRgba, mRedThresh: these are the Mat variables used to store different iterations of the processed images. mRgba is the original image. imgHSV is the image converted to HSV. mYellowThresh is the image with a threshold applied on it to only see the color yellow. mRedThresh is the image with a threshold applied on it to only see the color red.

BaseLoaderCallback mLoaderCallBack = new BaseLoaderCallback(this)

This function sets up the camera view javaCameraView.

startMasterChooser()

This method establishes the connection with the ROS network.

onCreate

This function sets up the activity's main layout as well as associating the appropriate variables with their views. Afterwards, javaCameraView is configured and a listener to its frames is established. onCreate then publishes "f" to the ROS network after a two-second delay by calling publishOnStart(). This is done so the robot always starts by going "freestyle".

init(NodeMainExecutor nodeMainExecutor)

This method initializes the ROS nodes of Talker and Listener.

• Parameters:
  • NodeMainExecutor nodeMainExecutor: the node executor that establishes all the desired nodes.
  • Listener lisNode: an instance of the listener node.

publishOnStart()

Publishes the message "f" on the talker node to initialize the robot with going "freestyle".

publishToAbort()

Publishes an abort message to ROS to stop the robot from moving.

onCameraViewStarted(int width, int height)

Here all the Mat variables are initialized with the appropriate sizes and color channels. This method is immediately called once a stream of images starts coming in through javaCameraView.

• Parameters:
  • int width: the incoming stream's width.
  • int height: the incoming stream's height.

onCameraViewStopped()

Once the image stream stops, this method is called to release the mRgba Mat from obtaining new images.

onCameraFrame(CameraBridgeViewBase.CvCameraViewFrame inputFrame)

Here the bulk of the image processing takes place. The original image is stored in mRgba and is then converted to HSV and saved in imgHSV. This is done as HSV images are better for color thresholding than RGB. Afterwards, two thresholds are applied on imgHSV and saved to mYellowThresh and mRedThresh for the yellow and red colors respectively. The two threshold images are then passed to findContours(), which finds any areas of color in them. If the contour area of the color yellow is larger than a specific number, a yellow circle of the same area will be drawn on the mRgba Mat. The same goes for the red ball. The color flags are used to ensure the robots only find the balls and declare themselves winners only once. Otherwise for each frame of a ball, the phone will keep sending messages. The mRgba Mat with the circles drawn on it is the final output image; this Mat is constantly updated with each input frame.

• Parameters:
  • List<MatOfPoint> yellowContours: contains a list of the contours found in the yellow threshold image.
  • List<MatOfPoint> redContours: contains a list of the contours found in the red threshold image.
  • double maxYellowArea: the area below which a contour will not be recognized in the yellow threshold image.
  • double maxRedArea: the area below which a contour will not be recognized in the red threshold image.
  • float[] yellowRadius: contains the circle radius in the yellow threshold image.
  • float[] redRadius: contains the circle radius in the red threshold image.
  • Point yellowCenter: contains the circle center for the yellow threshold image.
  • Point redCenter: contains the circle center for the red threshold image.