Augmented Reality Paint

Augmented Reality Paint is a program developed in the context of a university class. It replicates the basic functions of Microsoft Paint, with the added functionality of using the user's webcam to detect a color, be it the user's phone, the user's shirt, etc... as a pointer to paint the image.

Program functionalities

The program consists of two main scripts:

• color_segmenter.py - It is used to gather information from the user's webcam and create a mask. That mask will be used as the pointer. The mask limits will be saved as a .json file;
  
• ar_paint.py - Using the limits given by the previous script, transform the user's webcam into a pointer and painting in a white canvas. The user can also draw geometric shapes, draw with the mouse and play a coloring game!

How to use

To test this program you need to follow this steps:

1. Open a terminal and go to the directory where you created the python files.


2. Run the program "color_segmenter.py" to choose limits (b, g, r) for the color that you want to detect. After you chose the color, press "w" to save the current limits. Use the following command to run this program:
./color_segmenter.py


3. Now, run the program "ar_paint.py" to start drawing. For the basic functionality write the following command on the terminal:
./ar_paint.py -j limits.json

- While you are running the program, you can interact with the program by pressing the keys shown in the next table:



Key	Action
b	Start painting with the color blue
g	Start painting with the color green
r	Start painting with the color red
y	Start painting with the color yellow
l	Start painting with the color orange
p	Start painting with the color purple
a	Start painting with the color rainbow
+	Increase the pencil size
-	Decrease the pencil size
c	Clear the window
w	Save the current image on a PNG file
m	Start painting with the mouse
n	Start painting with the mask
v	Switch to draw on the white canvas/real frame
s	Draw a rectangle
o	Draw a circle
q	Exit the program



4. You can also use two advanced functionalities by using input arguments.
   If you want to use the Shake Prevention, write the following command:
   ./ar_paint.py -j limits.json -usp


5. If you want to use the Numeric Painting, write the following command:
   ./ar_paint.py -j limits.json -unp

Functionalities

Basic Functionality

When the user runs the program in this functionality, it's possible to paint with the mask or with the mouse.

In this mode the user can:

• Choose the color that it's used to draw (blue, green or red);
• Increase or decrease the pencil size;
• Clear the window;
• Save the current image as a PNG file;
• Switch between the white canvas/real frame;
• Draw rectangles or circles.

Default

The default mode is painting with the mouse, in color blue and with pencil size 5.

Advanced Functionalities

Shake Prevention

Sometimes the color detection might fail and the object detected with the biggest area can oscillate around the area of the window. When this happens the lines are drawn across all the window.

To prevent this from happening, you may use this functionality.

Numeric Painting

On this functionality, the program divides the window in zones. The user must paint the zones accordingly to the colors atributed to the zones (The colors are atributed randomly by the program).

In the end, the user can make an evaluation of the drawing, by pressing the space bar.

Contributors

• Lucas Rodrigues Dal'Col
• Manuel Alberto Silva Gomes
• Emanuel Krzysztoń
• João Pedro Tira Picos Costa Nunes