Nearest-neighbor Color Detector with TCS230 and Arduino

In this tutorial you will learn how to use the TCS230 Color Sensor with an Arduino to build a trainable color detector based on a nearest-neighbor classifier.

The TCS230 Color Sensor does not directly tell you which color it sees but returns sensor readings for the red, green, blue (and clear) channels. For most practical applications, you somehow have to convert those readings into a detected color, e.g. “purple”. It can be quite tricky to reliably detect a color using thresholds on the channel readings. We therefore will apply a bit of machine learning and use a trainable nearest-neighbor classifier instead.

Required Parts

You will need an TCS230 Color Sensor and a microcontroller. I used an Arduino Uno for this project, but any other Arduino or ESP32 will work as well. We also will employ an OLED to show the detected color and other information on a little display.

Makerguides.com is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to products on Amazon.com. As an Amazon Associate we earn from qualifying purchases.

Basics of the TCS230 Color Sensor

The TCS230 is a color light-to-frequency converter. It is composed of a 8 x 8 array of photodiodes that detect light. There are 16 photodiodes with a red filter that are sensitive to red light, 16 photodiodes with a green filter for green light, 16 photodiodes with a blue filter and 16 photodiodes without a filter for “white” light.

If you look closely at the picture of the TCS230 chip below, you can actually see the 8 x 8 array photodiodes with their colored (and clear) filters.

Function of the TCS230 Color Sensor

Apart from the array of photodiodes the TCS230 contains a Current-to-frequency Converter, which converts the current flowing through a set of photodiodes into a square wave. A set is composed of 16 photodiodes for a specific color (red, green, blue) or clear and the frequency of the square wave is proportional to the current through the set of photodiodes.

To measure the intensity of one of the three colors (or clear) you select the corresponding set of photodiodes (red, green, blue, white) by setting two inputs (S2, S3) and then read the frequency at the output. The frequency range (scaling) can be controlled by two other inputs (S0, S1).

Pinout of the TCS230 Color Sensor

The following picture shows the pinout of the TCS230 chip. You can see the pins S0 and S1 for selecting the output frequency range, and the S2, S3 pins for selecting the set of photodiodes (color).

The output OUT can be enabled or disabled via the OE pin. VDD and GND are for the power supply with a supply voltage of 2.7 V to 5.5 V. The table below summarizes the pinout:

Pin	Name	Description
1,2	S0, S1	Output frequency scaling
3	OE	Enable for output frequency
4	GND	Ground
5	VDD	Voltage supply
6	OUT	Output frequency
7,8	S2, S3	Photodiode type selection

Photodiode/color selection

As mentioned above, before you can measure the intensity of a specific color channel, you first have to select it via the S2 and S3 pins. The table below shows you which setting of S2 and S3 corresponds to which color channel:

Color channel	S2	S3
Red	LOW	LOW
Blue	LOW	HIGH
Clear	HIGH	LOW
Green	HIGH	HIGH

Frequency scaling

The frequency range of the output can be varied between 100%, 20%, 2%, and off by setting the following values for the S0 and S1 control pins:

Output scaling	S0	S1	min f	max f
Power down	LOW	LOW	–	–
2%	LOW	HIGH	10kHz	12kHz
20%	HIGH	LOW	100kHz	120kHz
100%	HIGH	HIGH	500kHz	600kHz

TCS230 Color Sensor Module

Typically you don’t use the TCS230 directly but get yourself a TCS230 Color Sensor Module that has a few additional components and is easier to connect. The picture below shows the front and back a typical TCS230 Color Sensor Module:

On the front of the module you find four white LEDs for illumination and a protective black housing that contains the TCS230 chip. If you look closely you can see the TCS230 chip in the center:

Schematics of TCS230 Color Sensor Module

The TCS230 Module has essentially the same pinout as the TCS230 chip itself apart from two changes. There is no OE pin but there is an additional LED pin.

The schematics of the module reveals that OE is connected to VSS, which means the output is always enabled. The LED pin is connected to the gate of a MOSFET (Q1) which switches the four illumination LEDs (D1…D4). Since the LED pin has an internal pullup, the LEDs are active by default and you can switch them off by connecting the LED pin to ground.

Note that the S0 and S1 pins also have internal pullups and the default output scaling for the frequency is therefore at 100%. On the other hand, the S2 and S3 pins have no pullups and you must set them to select a color channel.

In the next section we connect the TCS230 Module to an Arduino and write some code to test the sensor.

Connecting the TCS230 to Arduino

The following diagram shows you how to connect the TCS230 Module to an Arduino. I use 5V as power supply but 3.3V would work as well. The control pins S0, S1, S2, and S4 of the TCS230 Module are connected to pins 8, 9, 10, 11 of the Arduino.

The table below shows you again the connections you have to make

TCS230	Arduino
VCC	5V
GND	GND
S0	8
S1	9
S2	10
S3	11
OUT	12

Note that there are two VCC and two GND pins on the TCS230. You can use either. The LED pin is not connected but you could connect it to a digital output to switch the illumination LEDs on or off.

Code for reading Colors with TCS230

In the following code, we are using a TCS230 color sensor to read the values from the different colors channels (red, green, blue, and white) and print the results to the serial monitor.

#include "aprintf.h"

const int s0 = 8;
const int s1 = 9;
const int s2 = 10;
const int s3 = 11;
const int out = 12;

void initPins() {
  pinMode(s0, OUTPUT);
  pinMode(s1, OUTPUT);
  pinMode(s2, OUTPUT);
  pinMode(s3, OUTPUT);
  pinMode(out, INPUT);
}

void setScaling(int s0state, int s1state) {
  digitalWrite(s0, s0state);
  digitalWrite(s1, s1state);
}

uint16_t readPulse(int s2state, int s3state) {
  delay(20);
  digitalWrite(s2, s2state);
  digitalWrite(s3, s3state);
  return pulseIn(out, LOW);
}

void setup() {
  Serial.begin(9600);
  initPins();
  setScaling(HIGH, LOW);  // 20 %
}

void loop() {
  uint16_t red = readPulse(LOW, LOW);
  uint16_t blue = readPulse(LOW, HIGH);
  uint16_t green = readPulse(HIGH, HIGH);
  uint16_t white = readPulse(HIGH, LOW);

  aprintf("R:%3d G:%3d  B:%3d  W:%3d\n",
          red, green, blue, white);

  delay(1000);
}

Let’s break down the code into its components for a better understanding.

Includes

The code uses the aprintf library, which simplifies the formatting of text to print. It is essentially a version of the printf() function. For more details read the How To Print To Serial Monitor On Arduino tutorial.

To install it, go to the github repo click on the green “Code” button and the select “Download ZIP”:

After that, go to “Sketch” -> “Include Library” -> “Add. ZIP library” to install it.

To use the aprintf library we need the following include:

#include "aprintf.h"

Constants

Next we define the pins used to connect the TCS230 sensor to the Arduino.

const int s0 = 8;
const int s1 = 9;
const int s2 = 10;
const int s3 = 11;
const int out = 12;

Here, s0 and s1 are for setting the frequency scaling, while s2 and s3 are for selecting the color channel. The out pin is where the sensor outputs the frequency signal corresponding to the detected color intensity.

Pin Initialization

The initPins() function sets the pin modes for each of the defined pins. The sensor pins s0, s1, s2, and s3 are set as OUTPUT, while the out pin is set as INPUT.

void initPins() {
  pinMode(s0, OUTPUT);
  pinMode(s1, OUTPUT);
  pinMode(s2, OUTPUT);
  pinMode(s3, OUTPUT);
  pinMode(out, INPUT);
}

Frequency Scaling

The setScaling() function is used to configure the frequency scaling of the output signal from the sensor. By setting the states of s0 and s1, we can adjust the resolution of the sensor’s output. More about that later.

void setScaling(int s0state, int s1state) {
  digitalWrite(s0, s0state);
  digitalWrite(s1, s1state);
}

Reading Pulses

The readPulse() function is responsible for reading the duration in microseconds of the pulse output by the sensor. It takes the states of s2 and s3 as parameters to select which color channel to read. The function waits for 20 milliseconds to stabilize the sensor between readings.

uint16_t readPulse(int s2state, int s3state) {
  delay(20);
  digitalWrite(s2, s2state);
  digitalWrite(s3, s3state);
  return pulseIn(out, LOW);
}

Note that this function does not return the frequency of the square wave but the pulse lengths (many tutorials on the TCS230 get this wrong). The more intense the color the higher the frequency but the shorter the pulse. So there is an inverse relationship and the more intense the color the lower the value returned.

If you want to convert the value to a frequency, you would have to count the number of pulses n within a second and take the inverse:

f = 1 / n

but we will work directly with the pulse lengths.

Setup Function

In the setup() function, we initialize the serial communication at a baud rate of 9600 and call the initPins() function to set up the pins. We also set the scaling to 20% by calling setScaling(HIGH, LOW), which gives us a good resolution.

void setup() {
  Serial.begin(9600);
  initPins();
  setScaling(HIGH, LOW);  // 20 %
}

If you compare the 2%, 20% and 100% scaling factors you will find that the color channel readings are the largest for the 2% scaling and the smallest for the 100% scaling. Below some example readings for the different scaling factors when the sensor was shown a green and a white object with the same illumination and distance:

100%
R: 23 G: 16  B: 25  W:  8  // green
R:  7 G:  6  B:  6  W:  3  // white

20%
R:116 G: 82  B:124  W: 35  // green
R: 32 G: 31  B: 27  W: 10  // white

2%
R:1149 G:807  B:1220  W:339  // green
R:308  G:305  B:260   W: 96  // white

The 20% scaling factor is a good compromise between resolution and robustness. Also the color readings for the 20% scaling fit easily into an integer, while for the 2% scaling you may encounter overflows when the sensor is in complete darkness. I therefore picked the 20% scaling factor.

Loop Function

In the loop() function, we read the pulse durations for red, blue, green, and white colors by calling readPulse() with the appropriate parameters. The results are stored in variables red, green, blue, and white.

  uint16_t red = readPulse(LOW, LOW);
  uint16_t blue = readPulse(LOW, HIGH);
  uint16_t green = readPulse(HIGH, HIGH);
  uint16_t white = readPulse(HIGH, LOW);

As mentioned before, the color values are inverse to the intensity of the detected colors. We print the color values to the serial monitor using the aprintf() function, which formats the output for better readability. Finally, we add a delay of 1000 milliseconds (1 second) before repeating the loop.

  aprintf("R:%3d G:%3d  B:%3d  W:%3d\n",
          red, green, blue, white);
  delay(1000);

Output Example

If you upload the code to your Arduino and open the Serial Monitor, you should see color channel readings similar to ones below:

R: 56 G:123  B:112  W: 32
R: 47 G:117  B: 93  W: 28
R: 76 G: 71  B: 86  W: 26
R: 77 G: 74  B: 72  W: 26
R:104 G: 76  B: 47  W: 24
R:105 G: 75  B: 48  W: 24
R:165 G:101  B: 85  W: 28
R: 69 G: 71  B: 68  W: 24
R: 70 G: 71  B: 68  W: 24

If you put a red object in front of the sensor the red value should become smaller, similarly if the sensor sees a blue or green object, the value for the corresponding channel should become smaller.

There is a problem, however, the values depend heavily of the ambient brightness, which makes it difficult to reliably detect colors. In the next section we therefore normalize the readings.

Brightness normalization for TCS230

The overall range of the color readings depends on the chosen scaling. With 20% you can get values somewhere between 0 and about 15000. You can easily test this by covering the sensor with your finger (complete darkness) or shining a bright light into it. Here an example of the readings I got:

R:  4 G:  3  B:  4  W:  4
R:  4 G:  4  B:  4  W:  4
R:2243 G:13215  B:10289  W:2058
R:2330 G:13986  B:10896  W:2119

This strong dependency on the ambient brightness makes it nearly impossible to define robust thresholds to detect a color. For instance, the code such as the following will not work very well:

if (red > 10 && red < 100) {
  Serial.println("red detected");
}

We therefore need to normalize the color readings. This can easily be achieved by dividing the color readings by the white channel value (brightness). Here is the corresponding function:

void normalize(uint16_t &red, uint16_t &green, uint16_t &blue, uint16_t white) {
  red = 100 * red / (white + 1);
  green = 100 * green / (white + 1);
  blue = 100 * blue / (white + 1);
}

It multiplies the color reading (red, green, blue) by 100 and then divides them white value plus 1. We add the plus one to avoid a potential division-by-zero error, and we multiply by 100 to keep the color values in a similar range (same resolution).

If you compare the color readings with and without normalization you will find the ones with normalization to be more stable. For instance, I placed the TCS230 at various distances (1cm … 4cm) to a blue object.

Without normalization, I got blue values between 43 and 169. That is a difference or variation 126 units. With normalization, on the other hand, I measured blue values between 188 and 237, which is a difference of only 49 units. So the color readings are more stable (less sensitive to ambient light), with normalization.

In the loop function you can easily add the normalization function as follows:

void loop() {
  uint16_t red = readPulse(LOW, LOW);
  uint16_t blue = readPulse(LOW, HIGH);
  uint16_t green = readPulse(HIGH, HIGH);
  uint16_t white = readPulse(HIGH, LOW);

  normalize(red, green, blue, white);

  aprintf("R:%3d G:%3d  B:%3d  W:%3d\n",
          red, green, blue, white);

  delay(1000);
}

However, the normalization is not perfect and ambient light still affects the sensor readings. Furthermore, if you want to use thresholds to detect colors other than red, green or blue, the code become complex.

For instance, purple is a combination of red, green and blue values. To detect it you would have to implement something like the following:

if(red > 250 && red < 300 && green > 330 && green  < 400 && blue > 190 && blue < 230) {
  Serial.println("purple detected");
}

This code is complex, the thresholds are difficult to pick and it will not work very well. Instead we train a simple classifier to learn which color channel readings correspond to which colors. It will be much more robust, easier to implement and more fun as well ; )

Classifying TCS230 color readings

We are going to use a nearest-neighbor-classifier to classify the color readings of the TCS230. For a better understanding how this classifier works, I collected some data. I placed the TCS230 in front of a red, green, purple and yellow square and printed the color channel readings. Here is the color grid I used:

Plotting Color Channel Readings

I repeated this process multiple times and recorded the red and green readings. (I leave the blue readings out for now but we will use them later). This is how the data samples looked like:

RED, GREEN, COLOR
175,471,red
160,400,red
...
296,221,green
303,226,green
...
193,233,yellow
207,257,yellow
...
269,358,purple
282,374,purple
...

Next, I plotted each sample in a 2D scatter plot. Each dot in the plot below represents one sample (a reading of a red and green channel value), and the color of each dot indicates, which colored square was placed in front of the sensor.

As you can see, the samples for red, purple, yellow and green objects cluster very nicely together, with the exception of a potential outlier for a purple object in the upper right corner.

Classifying the Color of an Object

With this data it is now easy to determine the color of an object. We simply take a reading of the red and green channel and find the sample closest to the reading. In the plot below the cross marks the point for a new reading and you can see that the nearest sample is purple. We therefore would classify our object as having a purple color.

This method to classify objects based on the nearest sample is called a Nearest-Neighbor Classifier or Nearest-Neighbor Search. There are many, many variations of this algorithm.

Most importantly this algorithms works with higher dimensional data as well. Which means we can use all three color channel readings to determine a color. The following plot shows some samples for five colors (red, purple, yellow, green, blue) in 3D space. Each sample here has three dimensions, the color channel readings for red, green and blue and is a dot in this 3D scatter plot:

As before the color of the dot indicates which color the sample belongs too. Determining the color of a new object in this higher-dimensional 3D space works the same as before. Simply find the sample closest to your channel readings. It is just harder to visualize, which is why I started with a 2D space.

In the next section we will implement a simple Nearest-Neighbor Classifier that works in this 3D space.

Code for Nearest-Neighbor Classifier with TCS230

The code in this section is an extension of the code above. It adds some sample data and the Nearest-Neighbor Classifier. It prints out the readings for the color channels (red, green, blue) and the detected color. The output will look like this:

{292, 376, 221}  =>  purple
{393, 293, 193}  =>  blue
{274, 261, 306}  =>  green
{206, 253, 486}  =>  yellow
{165, 428, 375}  =>  red
{166, 246, 466}  =>  ???

Note that it can detect colors like “purple” or “yellow”, which are combinations of red, green and blue values. Furthermore, if it cannot confidently detected a color, it prints “???”.

Have a quick look at the code first and then we discuss how it works.

#include "aprintf.h"

const int s0 = 8;
const int s1 = 9;
const int s2 = 10;
const int s3 = 11;
const int out = 12;

const uint16_t reject = 1000; 
const char *colors[] = { "???", "red", "green", "blue", "purple", "yellow" };
const int n_samples = 33;
const uint16_t samples[n_samples][4] = {
  // red
  { 158, 422, 358, 1 },
  { 165, 354, 311, 1 },
  { 145, 347, 307, 1 },
  { 160, 376, 321, 1 },
  { 177, 343, 309, 1 },
  { 156, 403, 343, 1 },
  // green
  { 296, 225, 303, 2 },
  { 279, 232, 311, 2 },
  { 300, 223, 313, 2 },
  { 271, 231, 295, 2 },
  { 293, 230, 309, 2 },
  { 313, 231, 320, 2 },
  // blue
  { 413, 300, 186, 3 },
  { 337, 286, 191, 3 },
  { 376, 293, 200, 3 },
  { 354, 290, 206, 3 },
  { 395, 287, 187, 3 },
  { 351, 285, 197, 3 },
  { 427, 304, 190, 3 },
  // purple
  { 271, 361, 215, 4 },
  { 274, 346, 217, 4 },
  { 268, 355, 217, 4 },
  { 268, 333, 214, 4 },
  { 297, 387, 225, 4 },
  { 284, 365, 218, 4 },
  { 270, 356, 221, 4 },
  // yellow
  { 193, 243, 450, 5 },
  { 194, 252, 464, 5 },
  { 200, 244, 433, 5 },
  { 205, 252, 442, 5 },
  { 210, 245, 357, 5 },
  { 236, 284, 436, 5 },
  { 206, 253, 473, 5 },
};


float sqr(float a) {
  return a*a;
}

float eucDist(uint16_t r, uint16_t g, uint16_t b, uint16_t *s) {
  return sqrt(sqr(r - s[0]) + sqr(g - s[1]) + sqr(b - s[2]));
}

char *classify(uint16_t r, uint16_t g, uint16_t b) {
  char *color = colors[0];
  float mindist = reject;

  for (int i = 0; i < n_samples ; i++) {
    float dist = eucDist(r, g, b, samples[i]);
    if (dist < mindist) {
      mindist = dist;
      color = colors[samples[i][3]];
    }
  }

  return color;
}

void initPins() {
  pinMode(s0, OUTPUT);
  pinMode(s1, OUTPUT);
  pinMode(s2, OUTPUT);
  pinMode(s3, OUTPUT);
  pinMode(out, INPUT);
}

void setScaling(int s0state, int s1state) {
  digitalWrite(s0, s0state);
  digitalWrite(s1, s1state);
}

uint16_t readPulse(int s2state, int s3state) {
  delay(20);
  digitalWrite(s2, s2state);
  digitalWrite(s3, s3state);
  return pulseIn(out, LOW);
}

void normalize(uint16_t &red, uint16_t &green, uint16_t &blue, uint16_t white) {
  red = 100 * red / (white + 1);
  green = 100 * green / (white + 1);
  blue = 100 * blue / (white + 1);
}

void setup() {
  Serial.begin(9600);
  initPins();
  setScaling(HIGH, LOW);  // 20 %
}

void loop() {
  uint16_t red = readPulse(LOW, LOW);
  uint16_t blue = readPulse(LOW, HIGH);
  uint16_t green = readPulse(HIGH, HIGH);
  uint16_t white = readPulse(HIGH, LOW);

  normalize(red, green, blue, white);
  char *color = classify(red, green, blue);

  aprintf("{%3d, %3d, %3d}  =>  %s\n",
          red, green, blue, color);

  delay(1000);
}

Constants for Samples, Colors and Rejection

At the beginning of the code we add three constants. The reject constant determines when the classifier decides that it cannot detect a color. More about that later.

Next we have the list of colors we are going to detect. The first “color” is “???”, which is returned when the classifier could not detect a color. You can change and extend this list to detect as many colors as you like.

Finally, we have a list of samples n_samples, where each samples contains readings for the red, green and blue channel plus a color index. The color index is simply the position in the list of colors.

const uint16_t reject = 1000; 
const char *colors[] = { "???", "red", "green", "blue", "purple", "yellow" };
const int n_samples = 33;
const uint16_t samples[n_samples][4] = {
  // red
  { 158, 422, 358, 1 },
  ...
  // green
  { 296, 225, 303, 2 },
  ...
  // blue
  { 413, 300, 186, 3 },
  ...
  // purple
  { 271, 361, 215, 4 },
  ...
  // yellow
  { 193, 243, 450, 5 },
  ...
};

For instance, the last sample { 193, 243, 450, 5 }, means we have the readings for the three color channels red==193, green==243, blue==450 and that this sample is for the color colors[5] == "yellow".

I’ll show you later how to collect these samples.

Distance Function

A nearest-neighbor classifier works by finding the nearest sample to a given reading of channel colors. We therefore need a function to compute the distance between a reading of channel colors and a sample. A common distance function for this purpose is the Euclidean distance.

The function eucDist computes the Euclidean distance between the color channel readings r, g, b and a sample s. The closer the channel readings to the sample, the smaller the Euclidean distance will be.

float sqr(float a) {
  return a * a;
}

float eucDist(uint16_t r, uint16_t g, uint16_t b, uint16_t *s) {
  return sqrt(sqr(r - s[0]) + sqr(g - s[1]) + sqr(b - s[2]));
}

The square function sqr is a helper function that computes the square of its argument a.

Classification Function

The classify function iterates through all samples, finds the sample with the smallest Euclidean distance mindist to the color channel readings r, g, b and returns the detected color as a string.

char *classify(uint16_t r, uint16_t g, uint16_t b) {
  char *color = colors[0];
  float mindist = reject;

  for (int i = 0; i < n_samples; i++) {
    float dist = eucDist(r, g, b, samples[i]);
    if (dist < mindist) {
      mindist = dist;
      color = colors[samples[i][3]];
    }
  }

  return color;
}

If the smallest distance mindist is not smaller than the reject threshold value, the first color in the colors array is returned. This is the string “???“, which indicates that the color channel readings were not close enough to one of the samples to reliable detect the color.

You can make the rejection threshold really large (1e8) and the classifier will always respond with a color but generally it is better to let the classifier tell you, if it cannot confidently classify the color of an object.

Output Example

If you compile and upload the code, you should see color channel readings and the detected color printed like this:

{263, 363, 221}  =>  ???
{292, 376, 221}  =>  purple
{284, 371, 215}  =>  purple
{392, 275, 200}  =>  ???
{392, 296, 200}  =>  blue
{393, 293, 193}  =>  blue
{303, 269, 236}  =>  ???
{280, 266, 316}  =>  green
{274, 261, 306}  =>  green
{274, 261, 306}  =>  green
{222, 302, 320}  =>  ???
{163, 420, 360}  =>  red
{163, 420, 356}  =>  red
{160, 416, 333}  =>  red
{160, 416, 353}  =>  red
{268, 237, 314}  =>  ???
{287, 237, 292}  =>  ???

In the next section I’ll show you how to adapt the code to choose the colors you want to detect.

Collecting Training Data for the Color Detector

The code above demonstrates how to implement a Color Detector using a Nearest-Neighbor Classifier. It works for my sensor, with my ambient lighting conditions and it can detect the colors “red”, “green”, “blue”, “purple”, “yellow”. It’s accuracy and the number of colors is directly related to the data samples I collected.

Typically, you need to collect your own samples to build a Color Detector that works in your environment and for the colors you are interested in. Luckily this is easy. You can use the existing code. Just comment out the line where the classifier is called and change the printing as shown below:

void loop() {
  uint16_t red = readPulse(LOW, LOW);
  uint16_t blue = readPulse(LOW, HIGH);
  uint16_t green = readPulse(HIGH, HIGH);
  uint16_t white = readPulse(HIGH, LOW);

  normalize(red, green, blue, white);
  //char *color = classify(red, green, blue);

  int index = 1;  // first color
  aprintf("{%3d, %3d, %3d, %d}",
          red, green, blue, index);

  delay(1000);
}

Now, start collecting samples by picking your first color (index=1) and put an object with that color in front of the sensor. Vary the distance and ambient light as much as possible. After a few seconds you will have enough samples printed out for your first color, e.g. “green”

// green
{296,225,303,1},
{279,232,311,1},
...
{293,230,309,1},
{313,231,320,1},

Throw out the “bad” samples, e.g. when the object was removed from the detector and save the “good” samples in a text editor. Next increment the index variable to 2 and repeat the process for your next color, e.g. “purple”

// purple
{271,361,215,2},
{274,346,217,2},
...
{284,365,218,2},
{270,356,221,2},

Keep doing this until you have sample data for all the colors you want to detect. Then copy the sample data together into one big array:

const uint16_t samples[n_samples][4] = {
  // green
  {296,225,303,1},
  {279,232,311,1},
  ...
  {293,230,309,1},
  {313,231,320,1},

  // purple
  {271,361,215,2},
  {274,346,217,2},
  ...
  {284,365,218,2},
  {270,356,221,2},

  // other colors
  ...
};

Finally adjust the n_samples constant so that it matches the number of samples in the list and you are done with the data collection and the training of your classifier.

Change the code back to call the classifier and print its response to test your newly trained classifier:

void loop() { 
  ...
  char *color = classify(red, green, blue);
  aprintf("{%3d, %3d, %3d}  =>  %s\n",
          red, green, blue, color);
  ...
}

If you find cases where the detector fails to detect a color correctly, add those cases as samples to the data set. After a while the data set could become pretty big. You can reduce it by removing duplicates or near duplicates from the data set, e.g.

{271,361,215,3},
{274,346,217,3},  <= duplicate
{268,355,217,3},
{268,333,214,3},
{274,346,217,3},  <= duplicate
{297,387,225,3},
{273,346,216,3},  <= near duplicate
{270,356,221,3},

You can do this by hand or write some code to remove the duplicates programmatically.

In the next and final section we are going to add an OLED to display the detected color on a little screen, instead of printing to the Serial Monitor.

Adding an OLED to display detected colors

Adding the OLED is straightforward. Simply connect SDA and SCL of the OLED to A4 and A5 of the Arduino (SDA->A4, SCL->A5). Since the OLED can run on 5V, we can share the power supply lines. Connect VCC to 5V and GND to GND. The picture below shows the complete wiring:

Code to Display detected Colors

Below is the code that displays the color detected by the TCS230 and our nearest-neighbor classifier on the OLED:

#include "Adafruit_SSD1306.h"

const int s0 = 8;
const int s1 = 9;
const int s2 = 10;
const int s3 = 11;
const int out = 12;

const uint16_t reject = 1000;
const char *colors[] = { "???", "red", "green", "blue", "purple", "yellow" };
const int n_samples = 33;
const uint16_t samples[n_samples][4] = {
  // red
  { 158, 422, 358, 1 },
  { 165, 354, 311, 1 },
  { 145, 347, 307, 1 },
  { 160, 376, 321, 1 },
  { 177, 343, 309, 1 },
  { 156, 403, 343, 1 },
  // green
  { 296, 225, 303, 2 },
  { 279, 232, 311, 2 },
  { 300, 223, 313, 2 },
  { 271, 231, 295, 2 },
  { 293, 230, 309, 2 },
  { 313, 231, 320, 2 },
  // blue
  { 413, 300, 186, 3 },
  { 337, 286, 191, 3 },
  { 376, 293, 200, 3 },
  { 354, 290, 206, 3 },
  { 395, 287, 187, 3 },
  { 351, 285, 197, 3 },
  { 427, 304, 190, 3 },
  // purple
  { 271, 361, 215, 4 },
  { 274, 346, 217, 4 },
  { 268, 355, 217, 4 },
  { 268, 333, 214, 4 },
  { 297, 387, 225, 4 },
  { 284, 365, 218, 4 },
  { 270, 356, 221, 4 },
  // yellow
  { 193, 243, 450, 5 },
  { 194, 252, 464, 5 },
  { 200, 244, 433, 5 },
  { 205, 252, 442, 5 },
  { 210, 245, 357, 5 },
  { 236, 284, 436, 5 },
  { 206, 253, 473, 5 },
};

Adafruit_SSD1306 oled(128, 64, &Wire, -1);

void initOLED() {
  oled.begin(SSD1306_SWITCHCAPVCC, 0x3C);
  oled.setTextColor(WHITE);
}

void display(const char *color) {
  oled.clearDisplay();

  oled.setTextSize(3);
  oled.setCursor(15,10);
  oled.print(color);
  oled.display();
}

float sqr(float a) {
  return a * a;
}

float eucDist(uint16_t r, uint16_t g, uint16_t b, uint16_t *s) {
  return sqrt(sqr(r - s[0]) + sqr(g - s[1]) + sqr(b - s[2]));
}

char *classify(uint16_t r, uint16_t g, uint16_t b) {
  char *color = colors[0];
  float mindist = reject;

  for (int i = 0; i < n_samples; i++) {
    float dist = eucDist(r, g, b, samples[i]);
    if (dist < mindist) {
      mindist = dist;
      color = colors[samples[i][3]];
    }
  }

  return color;
}

void initPins() {
  pinMode(s0, OUTPUT);
  pinMode(s1, OUTPUT);
  pinMode(s2, OUTPUT);
  pinMode(s3, OUTPUT);
  pinMode(out, INPUT);
}

void setScaling(int s0state, int s1state) {
  digitalWrite(s0, s0state);
  digitalWrite(s1, s1state);
}

uint16_t readPulse(int s2state, int s3state) {
  delay(20);
  digitalWrite(s2, s2state);
  digitalWrite(s3, s3state);
  return pulseIn(out, LOW);
}

void normalize(uint16_t &red, uint16_t &green, uint16_t &blue, uint16_t white) {
  red = 100 * red / (white + 1);
  green = 100 * green / (white + 1);
  blue = 100 * blue / (white + 1);
}

void setup() {
  initPins();
  initOLED();
  setScaling(HIGH, LOW);  // 20 %
}

void loop() {
  uint16_t red = readPulse(LOW, LOW);
  uint16_t blue = readPulse(LOW, HIGH);
  uint16_t green = readPulse(HIGH, HIGH);
  uint16_t white = readPulse(HIGH, LOW);

  normalize(red, green, blue, white);
  char *color = classify(red, green, blue);
  display(color);
 
  delay(100);
}

The main changes are the include of the Adafruit_SSD1306 Library needed to communicate with the OLED, the creation of the OLED object, a function initOLED to initialize the OLED, and a display function to print the detected color on the OLED:

#include "Adafruit_SSD1306.h"

...

Adafruit_SSD1306 oled(128, 64, &Wire, -1);

void initOLED() {
  oled.begin(SSD1306_SWITCHCAPVCC, 0x3C);
  oled.setTextColor(WHITE);
}

void display(const char *color) {
  oled.clearDisplay();

  oled.setTextSize(3);
  oled.setCursor(15,10);
  oled.print(color);
  oled.display();
}

...

If you haven’t already installed the Adafruit_SSD1306 Library, you will have to. Just install it via the Library Manager as usual:

Note that the I2C address for the OLED display is set to 0x3C in oled.begin(). Most of these small OLEDs use this address (or 0x27) but yours might be different. If you don’t see anything on the OLED, it most likely has a different I2C address and you have to change the address.

If you don’t know the I2C address have a look at the How to Interface the SSD1306 I2C OLED Graphic Display With Arduino tutorial to find it. Also the Use SSD1306 I2C OLED Display With Arduino tutorial will tell you more about how to use an OLED.

Loop Function

The only change in the loop function is that we replace the printing to Serial Monitor by calling the display function with the detected color:

void loop() {
  uint16_t red = readPulse(LOW, LOW);
  uint16_t blue = readPulse(LOW, HIGH);
  uint16_t green = readPulse(HIGH, HIGH);
  uint16_t white = readPulse(HIGH, LOW);

  normalize(red, green, blue, white);
  char *color = classify(red, green, blue);
  display(color);
 
  delay(100);
}

And that’s it! Now you should have a color detector that you can adjust (train) to detect any color you like under varying lighting conditions. The short video clip below shows my color detector in action:

You can see that it nicely detects the three colors purple, yellow and blue and prints “???”, when I am moving between colors or when the detector is too far away.

Conclusions

In this tutorial you learned how to use the TCS230 Color Sensor with an Arduino to build a trainable color detector using a nearest-neighbor classifier. This color detector works pretty well but there are many possible improvements.

If you want to detect many colors and the sensor readings are more noisy a k-nearest neighbor classifier will make the detection more robust.

Instead of normalizing the color channels readings by the white channel you could collect 4 dimensional samples (r, g ,b, w) and create a 4-dimensional nearest-neighbor classifier. Such a classifier will be more robust towards changes in ambient light but will also need more training samples.

However, you can compress the list of samples, and thereby reduce memory requirements and increase classification speed, by storing only the centers of the color clusters. Alternatively, you could remove samples from the center and keep only samples close to the class border.

Finally, we “trained” the classifier manually by collecting samples and hard-coding them. But you could also add a “training” button to the circuit and code that stores a sample in EEPROM when pressed. This would allow you to dynamically train your color detector.

If you have any questions feel free to leave them in the comment section.

Happy Tinkering ; )