3.7 – Color Sensor

Goals:

By the end of this lesson, I will be able to:

1. Know the difference between HSV and RGB
2. Describe how the CoDrone EDU classifies colors
3. Calibrate my color sensor
4. Predict colors with the CoDrone EDU

Steps to Success:

1. Color Sensor Calibration
2. Color Sensor Prediction
3. Color Piano
4. Challenge 1: Multiple Samples
5. Challenge 2: Color Piano – Extended

Important: The color sensors will only work if the drone has landed on a surface. The color sensors cannot detect the surface underneath during flight.

Your drone will return color data in the HSV color space. HSV stands for hue, saturation, and value.

• Hue represents the color as a value from 0-360
• Saturation is the color’s intensity from 0-100
• Value is the lightness or darkness of the color from 0-100

For computer recognition and other robotics applications, it is better to use the HSV color space, since it separates values for colors and intensity. This is important because shadows, reflections, and other factors may cause certain colors to look very different. The HSV color space takes this into account for more accurate results.

Let’s start by finding how to access color data from the remote controller. First, make sure that you are not in the “LINK” state. To access color data from the controller, we need to be in remote control flight mode. Once in the remote control state, press either the “P” or “S” button until you see the screen in the image below. From this screen you can see HSV values (Hue, Saturation, Value) as well as the current color seen by the drone!

Place the drone on top of each of the 8 color cards that you received in the box.

Note: If the color is predicted incorrectly or return an unknown color, you can calibrate the sensor by following the instructions in this lesson. Although this lesson is for the JROTC edition of the drone, it applies to CoDrone EDU.

Click on the circular icon that is on the right side of Blockly to open the color training tool. Next, click on the New color data button to start color training process.

The color training tool will prompt you to enter a name for your color data set.  Name your color set codrone_colors, or any other name that you will remember.

To train your color data, you need to collect color samples for each color that you have in mind (indigo, orange, beige, red, black, etc.). To simplify what colors we are going to be collecting data from, we will use the color cards provided by the CoDrone EDU kit.

The color training tool will prompt you to add a color to your data set by clicking the ‘+’ button.

Now, you can add a name for the color that you just added! Let’s start with ‘red’ first.

After entering a name for your color, you can now click on the color picker icon right above the name to open a pop-up window to start collecting samples.

A pop-up window will appear that will let you collect samples of the desired color. The window provides the name of your color data set, the name of the color that you will be collecting, instructions on how to properly collect samples, and a button to collect samples.

Once your drone is flat on the surface and its color sensors start illuminating, you can click on ‘Collect samples’.

Now, you can repeat the same process for the other color cards!

The color training tool tab should look like this once you finish collecting samples for each color!

If you’re unhappy with how a color turned out, you can click on the ‘X’ button that’s next to one of the colored circles to remove it from your color data set and simply add a new color with the same name. For example, the ‘red’ color data that we collected didn’t look red, so the ‘red’ color was removed and was redone. Remember, if all of your colors aren’t coming out the way that they’re supposed to, head to this lesson to re-calibrate your drone’s color sensors!

Now, let’s start testing our color data set! Click on the red ‘Test color data” to test your color data set. This will open a pop-up window.

In this pop-up window, the window will show you the actual color that your color sensors are detecting, the HSV value of the detected color, and a prediction of the name of the detected color. This prediction (or guess) is based off the color data set that you provided to Blockly.

The process that we went through previously of collecting samples of color data and giving it a name (labeling) is called “training”. You’re essentially telling the program (Blockly, in this case) that “these samples of color data that I collected should be considered as the color red” (or whatever color)! During the training phase, you can name the color any way you want, mahogany, orange, pink, etc. It all depends on the context, like the color of the surface you’re using to collect data from.

Additionally, if you were to place the drone on a surface that wasn’t used in the training phase, during the testing phase, Blockly will give you its closest guess on what the name of the detected color is.

Make sure you’ve tested your color data set to make sure it is providing the correct guesses by placing your drone on each color card.

If it isn’t giving correct guesses, click on the ‘X’ button on the top-right to modify your color data set. Otherwise, it is highly recommended to download your color data set with the gray button before moving on.

Let’s click Finish and create blocks! This will populate blocks in the “Colors” category located in the “Senior” tab.

If you forget to download your color data set, don’t worry! There’s a download button that will appear on the top-right side of the color training tool. Right next to the download button, there’s a trash can icon to remove the existing color data set and depopulate the “Colors” category.

Blockly will add new blocks for you in the “Colors” menu. There will be a new block available for every color that you trained. Open the menu now to see these blocks. Your color training is complete and ready to use!

In this example, you will learn how to use the predict_color() with your dataset. From the “Colors” menu, drag the load_color_data() block into your workspace with the dataset you just created above.

In a loop that repeats forever (while code is running), create a variable that will store the predicted color returned from predict_color(). You can only use the green predict_color() block together with load_color_data(). Finally, print the color to the console and wait 0.5 seconds. To test, run the code while placing the drone on each color card.

In this activity, you will modify your code to make your drone into an instrument! Every color will make a unique buzzer noise when it is detected. To do this, add a conditional statement that compares the predicted_color with a color label. You can find the color labels in the “Colors” menu. If the predicted color matches a designated color (like red), play a sound for a half a second!

To add more colors, click the blue gear icon and add more else if blocks. How many colors can you add?

As you add more conditional statements, remember that you can only use the labels that are in your dataset.

Saving code and saving data are separate!

If you use the “Save” option from the menu, it will only save the blocks that are in your workspace. In this case, it will only save the “Color Piano” program to your computer as an .xml file that you can load later.

⚠️ This option will not save your color data set. You must use the download option when you create the data set or after creating the data set in the color training tool.

If you forgot to download your color data, you can click the download button on the top-right side of the color training tool (a color data set must already exist in Blockly for this option to show).

If you have previously saved some code with custom color data, you will need to follow the following steps.

1. Upload the code 

Blockly code is saved as an .XML file. If you try to open it with a text editor on your computer, it will not look like familiar Block code. To see any code you have saved, you have to open it in the Blockly workspace.

2. Upload the data folder

When Blockly reads the “load color data” block, it is expecting to reference the color data that you collected. If your “Colors” menu is empty, then there is no data for your code to use! Click on “Load color data”.

NOTE: Before clicking on “Upload color data” to upload your color data set from your computer, make sure that the color data set is not a ZIP file. The ZIP file must be extracted, so your color data set is in the form of a folder.

⚠️ Make sure the name of the folder that you upload matches the name of the data set that your code is trying to load. You can find this name in the “load color data” block.

If you upload the incorrect folder, your code will not be able to find the data.

Light-up Piano

Add to your program and make the LEDs change color to match the color being detected.

Custom Landing Pads

Grab some markers and art supplies to design your own landing pads! Pick some new colors and use the color sensor calibration program to detect new colors.