Image-9 Bluescreen

Building on the earlier image manipulation and logic sections, here we see how to implement the "bluescreen" movie special effect.

Bluescreen

• Demo Google search: blue screen movie image
• Also known as Chroma Key (wikipedia)
• Video is just a series of still images, 20-60 per second
• Have image to modify
• Have back background image, pull pixels from
• Detect colored area in an image, e.g. the blue area
• In one word: paste
• Paste a new image on the blue pixels

First: A Tale of 2 Pixels

• Suppose we have 2 pixels, red and green
• In code, the variable pixel is the red one, pixel2 is the green
• Q1: In English, how to change pixel so it looks identical to pixel2?
• Q2: Write the code below to do that just for red, using variables pixel and pixel2
• Lego Theme: an idea like "looks identical" expressed by combining tiny bits of code
• Movie "Office Space" - pixel vs. pixel2 in the code

Solution code:

  //Q1: make the red/green/blue numbers be the same, the pixels will look identical

  // copy over the 3 numbers with setXXX/getXXX
  // get from pixel2, set into pixel
  pixel.setRed(pixel2.getRed());
  pixel.setGreen(pixel2.getGreen());
  pixel.setBlue(pixel2.getBlue());

Bluescreen Stop Sign Example

Say we want to change the red part of sign to look like leaves

Bluescreen Algorithm "Non Trivial"

• (on the board)
• Two images we'll call image and back
• Detect, say, red pixels in image
• For each red pixel:
  -Consider the pixel at the same x,y in back image
  -Copy that pixel from back to image
  -i.e., as above, copy RGB numbers from back pixel to image pixel
• Result: for red areas, copy over areas from back image
• Adjacent pixels in back are still adjacent in new image, so it looks right

Bluescreen Example Code - Demo

• Bluescreen plan:
• Loop over stop.jpg pixels
• Detect red pixels (avg technique * 1.5)
• We'll call this "red-high" detection
• For those pixels, copy over pixels from leaves.jpg
• For each red pixel detected:
  -Figure out x,y of pixel
  -Get pixel2 from x,y in back image
  -Copy red/green/blue from pixel2 to pixel

The above bluescreen example code detects the red part of the sign, and for those pixels, copies over the pixels from the back image. The bluescreen code has 3 key differences from the earlier examples:

1. back = new SimpleImage("leaves.jpg"); - open a second image and store it in the variable "back" (thus far we've only opened one image at a time)
2. pixel2 = back.getPixel(x, y); - say we are looking at pixel x,y in the main image. From the other image get the pixel at the same x,y and store it in the variable "pixel2". We get the x and y of the current pixel in the loop with the functions pixel.getX() and pixel.getY().
3. pixel.setRed(pixel2.getRed()); - copy the red value from pixel2 to pixel. Also do this for the other two colors. The result is that at this x,y location, the pixel is copied from the back to image.

Bluescreen Stop/Leaves Experiments - Your Try It

Try these on the above code.

1. Flip -- try replacing everything-but-the-red with leaves instead of the red, i.e. "red-low" detection
2. Sky -- try replacing the blue sky instead of the red areas of the sign. What color do you look for? Tune the * factor so only the sky is changed.
3. Shorter -- make the code a little shorter by eliminating the need for "x" and "y" variables. Put the calls to pixel.getX() and pixel.getY() right in the back.getPixel(...) call.

Solution code:

    // flip: just change < to >

    // sky: if (pixel.getBlue() > avg * 1.1) {
    // This is a great example of tuning the * factor

    // pixel2 in one line:
    // pixel2 = back.getPixel(pixel.getX(), pixel.getY());

Monkey Bluescreen Example - You Try It

Now we'll do one like the movies -- film the movie start in front of a blue screen. Replace the blue background behind the monkey with pixels from moon.jpg.

Here is our monkey movie star:

Here is our background, the famous Apollo 8 photo of the earth shown rising above the moon horizon. There is a theory that this famous image helped start the environmental movement.

• Bluescreen plan:
• Loop over monkey.jpg
• Detect blue pixels, i.e. blue-high
• For those pixels: copy over moon pixels
• (copy over the earlier example code to get started)
• Then try it with other back images: paris.jpg yosemite.jpg stanford.jpg

Solution code:

image = new SimpleImage("monkey.jpg");
back = new SimpleImage("moon.jpg");
back.setAsBig(image);

for (pixel: image) {
  avg = (pixel.getRed() + pixel.getGreen() + pixel.getBlue())/3;
  // your code here
  if (pixel.getBlue() > avg * 0.92) {  // detect blue-high
    pixel2 = back.getPixel(pixel.getX(), pixel.getY());
    pixel.setRed(pixel2.getRed());
    pixel.setGreen(pixel2.getGreen());
    pixel.setBlue(pixel2.getBlue());
  }
}

print(image);

Selecting High vs. Low

• The monkey image is especially easy to work with
• The monkey is redish and the background is blueish
• We can use red or blue to detect the monkey or background
• e.g. "blue-high" strategy = detect areas where blue is high (the background)
• e.g. "red-high" strategy = detect areas where red is high (the monkey)
• Four combinations: blue-high blue-low red-high red-low
• Choose the best strategy depending on the image
• The body code below sets bright green on the selected pixels
• Experiments to try:
   1. (demo) Write code for the red-high strategy to select the monkey
   2. (demo) Change to the flip of that (red-low), selects what?
   3. (you try it) Write code and the * factor for blue-low to select the monkey
    do we want to make the if-test more or less restrictive?
    do we want the * factor to be bigger or smaller?
   4. Finally, flip to blue-high, selects what?
• I like this series, reflecting a little of the perfect logic within the computer

Solution code:

    // red-high -> monkey
    if (pixel.getRed() > avg * 1.05) {

    // flip: red-low -> background
    if (pixel.getRed() < avg * 1.05) {

    // blue-low -> monkey
    if (pixel.getBlue() < avg * 0.95) {

    // blue-high -> background
    if (pixel.getBlue() > avg * 0.95) {

Note: back.setAsBig(image);

Looping over a main image and then grabbing pixels from a back image, there is a problem if the back image is smaller than the main image. The code will try to access a non-existent pixel, and we get an "x/y out of bounds" error. The following line, before the loop, fixes this problem by resizing the back image to be bigger if necessary. As a precaution, we'll include this line for the problems that grab x/y pixels from the back image.

  back.setAsBig(image);

Another Effect: Image Blend

A "blend" is another effect to try, making a "ghost" version of the monkey in the back image. We blend the colors from the two images instead of replacing entire pixels.

• Modify and print the back image, blending in the monkey
• Blend plan:
• 1. Loop over monkey image
• 2. if-logic to detect monkey with blue-low strategy
  (this is flip, previously we detected not-monkey background)
• 3. Blend in monkey:
  set back-pixel-red = back-pixel-red + monkey-pixel-red/2
• 4. Do it for the other 2 colors too
• 5. Print the back image
• Result is a blend of a faint monkey into the back image
• Experiment 1: play with the * factor. What does is more restrictive here? What happens when monkey-background is included?
• Experiment 2: Adjust div=2 factor, 1 2 10 20. div=2 is good to see how it works, but /10 is more artistic
• Lego analogy: same bricks as bluescreen, arranged slightly differently

Solution code:

image = new SimpleImage("monkey.jpg");
back = new SimpleImage("paris.jpg");
back.setAsBig(image);

for (pixel: image) {
  avg = (pixel.getRed() + pixel.getGreen() + pixel.getBlue())/3;
  // your code here
  if (pixel.getBlue() < avg * 0.9) {  // detect monkey (blue-low)
    div = 2;  // monkey is divided by this variable below
    pixel2 = back.getPixel(pixel.getX(), pixel.getY());
    pixel2.setRed(pixel2.getRed() + pixel.getRed()/div);
    pixel2.setGreen(pixel2.getGreen() + pixel.getGreen()/div);
    pixel2.setBlue(pixel2.getBlue() + pixel.getBlue()/div);
  }
}
print(back);