Project 1:

David Martinez - Fall 2024

CS180 - Professor Efros

Images of the Russian Empire: Colorizing the Prokudin-Gorskii photo collection

  1. Preprocessing grayscale images with RGB color filters.
  1. Apply linear grid search strategy for smaller images. (jpegs)
  1. Apply Gaussian Pyramid alignment using varying depths of recursion
  1. Bell's and whistles: FFT approach, white balancing, and filters/edge detection

Simple Grid Search

green channel shift: (-5, -2)
red channel shift: (-12, -3)

green channel shift: (3, -2)
red channel shift: (-3, -2)

green channel shift: (-3, -3)
red channel shift: (-7, -3)

Gaussian Pyramid Search Results

Emir

green shift: (-48, -32), red shift(-99, -52)

Church

green shift: (-28, -9), red shift: (-54, -3)

Harvesters

green shift: (-46, -16), red shift(-102, -20)

Icon

green shift: (-41, -23), red shift(-83, -28)

Lady

green shift: (-39, -25), red shift: (-101, -26)

Train

green shift: (-34, -13), red shift: (-64, -34)

Onion Church

green shift: (-28, -40), red shift: (-97, -44)

Self Portrait

green shift: (-72, -41), red shift: (-175, -47)

Melons

green shift: (-72, 0), red shift: (-175, -4)

Sculpture

green shift: (-52, 18), red shift: (-153, 43)

Three Generations

green shift: (-48, -28), red shift: (-115, -16)

Bells and Whistles

1. “Gray World” White Balancing

A clever trick to balance the brightness and color channel imbalance in an image to use a strategy called white balancing. There are a few ways to implement this manually using a reference pixel for bright or dark areas, but a common technique is to assume that the world, as we perceive it, averages to grey. While this is true of the RGB color field, it isn’t the best representation of our perceived colors and the probability of encountering those colors. In my code, I implemented a version of white balancing which is slightly modified to fix the brightness issues perceived in these images, which had highly imbalanced color histograms.

Example:

Ground Truth

This image appears to have a slight yellow hue with improper saturation

White Balanced

With white balancing, the sky looks more accurate and vibrant

The original image has an unbalanced blue channel which overpowers the colors of the image

With white balancing, the colors of the image are more accurate and less blue

Ground Truth

White Balanced

FFT Based Approach

The FFT cross-correlation method aligns images by leveraging frequency domain operations, which handle alignment much more efficiently than a pixel-by-pixel search. Instead of checking each possible shift individually, which is slow and computationally expensive, the FFT approach quickly converts both images to the frequency domain, multiplies one by the complex conjugate of the other, and uses an inverse FFT to find the alignment shift in one step. This approach cuts down the runtime significantly—from a brute-force time complexity of O(m×n×N2)O(m \times n \times N^2) to O(N2log⁡N)O(N^2 \log N), making it much faster, especially for larger images such as those with the tif file format. In my implementation, this method greatly improved alignment speed and accuracy, bypassing the tedious process of testing every possible shift manually.

White alignment is working, I am still getting color correction implemented

Note: These runtimes are on an M2 Mac Book Pro, so OS and hardware may affect results

Pyramid + White Balanced: 17.7 seconds

FFT + White Balanced: 5.7 seconds