AWGN Channel Capacity

• Channel capacity is the maximum rate at which information can be reliably transmitted over a communication channel.
• Channel capacity is denoted as $C$ and measured in bits per channel use.
• $C$ is determined by maximizing the mutual information $I(X; Y)$ over all possible input distributions $p(x)$.

Additive White Gaussian Noise (AWGN) Channel

• The AWGN channel is modeled as $Y = X + Z$.
• $X$ represents the transmitted signal, $Z$ is the additive white Gaussian noise with distribution $Z \sim \mathcal{N}(0, N)$, and $Y$ is the received signal.
• It is assumed the transmitted signal has an average power constraint, denoted as $E(X^2) \le P$.

Capacity of AWGN Channel

• The capacity $C$ of an AWGN channel is given by $C = \frac{1}{2} log(1 + \frac{P}{N})$.
• $C$ is measured in bits per transmission.
• $P$ is the average signal power, and $N$ is the noise variance.
• The optimal input distribution that achieves this capacity is $X \sim \mathcal{N}(0, P)$.

Parallel Gaussian Channels

• A system of $k$ independent Gaussian channels is represented as $Y_i = X_i + Z_i$, for $i = 1, 2, \dots, k$.
• $Z_i$ follows a normal distribution $\mathcal{N}(0, N_i)$, where $N_i$ is the noise variance for the $i$-th channel.
• $X_i$ is subject to the power constraint $E(X_i^2) \le P_i$ for each channel.
• The total power is constrained, such that $\sum_{i=1}^{k} E(X_i^2) = \sum_{i=1}^{k} P_i \le P$.

Capacity of Parallel Gaussian Channels

• The capacity $C$ for parallel Gaussian channels is $C = \sum_{i=1}^{k} \frac{1}{2} log(1 + \frac{P_i}{N_i})$.
• The allocation of $P_i$ values is done to maximize $C$, subject to the constraint $\sum_{i=1}^{k} P_i \le P$.
• Using Lagrange multipliers, a function $J(P_i)$ is maximized: $J(P_i) = \sum_{i=1}^{k} \frac{1}{2} log(1 + \frac{P_i}{N_i}) - \lambda(\sum_{i=1}^{k} P_i - P)$.

Water-filling Solution

• Differentiating $J(P_i)$ with respect to $P_i$ and setting it to zero gives $\frac{\partial J}{\partial P_i} = \frac{1}{2} \frac{1}{N_i + P_i} - \lambda = 0$.
• Solving for $P_i$ yields $P_i = \frac{1}{2 \lambda} - N_i$.
• Introducing $v = \frac{1}{2 \lambda}$, we have $P_i = v - N_i$, where $v$ is chosen such that $\sum_{i=1}^{k} P_i = P$.

Water-filling Interpretation

• Power is allocated to each channel to equalize the effective noise level across all channels.
• The power $P_i$ allocated to channel $i$ is the water level $v$ above the noise level $N_i$.
• Channels with $N_i$ above $v$ are not used in this water-filling approach.
• Analogous to pouring water into a vessel, the water level ($v$) is uniform, and the amount of water poured in is the total power ($P$).

Capacity of Parallel Gaussian Channels with Water-filling

• The capacity $C$ with water-filling is $C = \sum_{i=1}^{k} \frac{1}{2} log(\frac{v}{N_i})$.
• The water level $v$ satisfies $\sum_{i=1}^{k} max(0, v - N_i) = P$.

Example Scenario

• Two parallel Gaussian channels have noise variances $N_1 = 1$ and $N_2 = 2$, with a total power constraint $P = 3$.
• Solving for $v$: $v - N_1 + v - N_2 = P$ gives $2v - 1 - 2 = 3$, hence $v = 3$.
• Since $v > N_1$ and $v > N_2$, both channels are used with $P_1 = v - N_1 = 3 - 1 = 2$ and $P_2 = v - N_2 = 3 - 2 = 1$.

Capacity Calculation Example

• With $P_1 = 2$, $N_1 = 1$, $P_2 = 1$, and $N_2 = 2$.
• The capacity is $C = \frac{1}{2} log(1 + \frac{P_1}{N_1}) + \frac{1}{2} log(1 + \frac{P_2}{N_2})$.
• $C = \frac{1}{2} log(1 + \frac{2}{1}) + \frac{1}{2} log(1 + \frac{1}{2}) = \frac{1}{2} log(3) + \frac{1}{2} log(1.5) \approx 1.2925$ bits.