Course information

Entropy, relative entropy and mutual information, asymptotic equipartition property, entropy rate of a stochastic process, data compression, gambling, channel capacity, differential entropy, Gaussian channel, rate distortion theory, information theory and statistics, maximum entropy, universal source coding, Kolmogorov complexity, network information theory, application of information theory on portfolio theory, inequalities in information theory.