Bilinear Approach to Color Constancy

When both the illumination and the reflectance are linearly modeled

$$\left\{ \begin{array}{l} I(\lambda)\approx \sum_{k=1}^m e_k ... ...mbda)\approx \sum_{j=1}^n f_j F_j(\lambda) \end{array} \right.$$

we get a bilinear model which can be used to address the color constancy issue by describing the responses of the p (p=3 for human eye) types of sensors as

$$r_i(\lambda) \int S_i(\lambda) L(\lambda) d\lambda= \int S_i(\lambda) R(\lambda) I(\lambda) d\lambda$$ =

$$\sum_{k=1}^m \sum_{j=1}^n e_k f_j \int E_k(\lambda) F_j(\lambda) S_i(\lambda) d\lambda$$ =

$$\sum_k \sum_j e_k f_j c_{jk}^i\;\;\;\;(i=1,\cdots,p)$$ =

where the c_kjⁱ's are a set of p by m by n coefficients representing the response of the ith cones to the jth basis function for reflectance illuminated by the ith basis function for the illumination, which can be determined given the kth basis functions $E_k(\lambda)$ for the illumination, the jth basis functions $F_j(\lambda)$ for reflectance, and $S_i(\lambda)$, the sensitivity of the ith sensors:

$$c_{kj}^i\stackrel{\triangle}{=}\int E_k(\lambda) F_j(\lambda)... ... d\lambda\;\;\;\;\;(i=1,2,3;\;\;k=1,\cdots,m;\;\;j=1,\cdots,n)$$

The purpose here is to use the known responses of the p sensors r_i's, as well as our knowledge, the known bilinear coefficients c_jkⁱ's, to determine the unknown weights (descriptors) f_j's for the reflectance and e_k's for the illumination so that they can be determined independently. In general unique determination of mn variables requires the same number of equations, i.e., p=mn sensors are needed. While p<mn, additional conditions must be imposed in order to make the problem solvable, such as

• The average of all surfaces in the image is gray;
• The brightest surface in the image is a uniform reflector;
• Specularities or glossy surfaces are available in the image;
• The surfaces are observed multiple times under different illuminations.

Many methods along this line have been proposed in the recent years.

To find how the sensors respond to the same surface with different illuminations and how the responses under different illuminations are related, we apply the linear model for reflectance and write the responses as

$$r_i(\lambda) \int S_i(\lambda) L(\lambda) d\lambda= \int S_i(\lambda) R(\lambda) I(\lambda) d\lambda$$ =

$$\sum_{j=1}^n \int I(\lambda) S_i(\lambda) F_j(\lambda) d\lambda f_j$$ =

$$\sum_j c_{ij} f_j \;\;\;\;(i=1,2,3)$$ =

where

$$c_{ij}\stackrel{\triangle}{=}\int I(\lambda) S_i(\lambda) F_j(\lambda) d\lambda$$

which can be obtained given the specific illumination $I(\lambda)$, the cone sensitivities $S_i(\lambda)$ and the basis functions for the reflectance $F_j(\lambda)$. If we use only n=3 terms to approximate any reflectance, the c_ij's can be written as a 3 by 3 matrix

$$C=\left[ \begin{array}{lll} c_{11} & c_{12} & c{13} \\ c_{21} & c_{22} & c{23} \\ c_{31} & c_{32} & c{33} \end{array} \right]$$

and three equations above can be written in vector form as

r = C f

The responses to the same surface but under a different illumination can also be written

r'= C' f

Now the two sets of cone responses are related by

r=CC'^-1 r'

i.e., the cone responses under one illumination is linearly related to the responses under a different illumination.

For the blue-sky illumination, we have

$$C=\left[ \begin{array}{lll} 591.5 & 223.1 & -643.1 \\ 477.6 & 376.9 & -564.5 \\ 267.6 & 487.5 & 350.4 \end{array} \right]$$

For the tungsten bulb,

$$C=\left[ \begin{array}{lll} 593.5 & 68.4 & -646.7 \\ 445.8 & 312.8 & -564.3 \\ 152.5 & 278.5 & 185.5 \end{array} \right]$$

Now the responses to an arbitrary surface illuminated by the tungsten bulb are linearly related to the responses to the same surface illuminated by the blue sky by

$$r=\left[ \begin{array}{lll} 0.812 & 0.227 & 0.065 \\ -0.080 & 1.134 & 0.128 \\ 0.043 & -0.076 & 1.809 \end{array} \right] r'$$