SHE Level 4
SCQF Credit Points 20.00
ECTS Credit Points 10.00
Module Code MHH624699
Module Leader Martin MacDonald
School School of Computing, Engineering and Built Environment
Subject SCEBE - School Office
  • A (September start)
  • B (January start)

Summary of Content

This course enables the students to gain understanding of fundamental concepts of information theory in communications and illustrates the usefulness of different coding schemes for improved performance in communications.


The teaching syllabus will cover the following areas: Information theory Basics: Introduction; Uncertainty and Information; Entropy, its types and properties; Entropy of Binary and discrete memory less source - Mutual Information and its properties. Source Coding : Source Coding Theorem-prefix, variable-length and fixed-length codes, Kraft Inequality; Coding efficiency and redundancy; Construction of basic source codes- Huffman Coding, arithmetic coding, universal coding- The Lempel-Ziv Algorithm. Channel capacity and Coding: Channel Models - Binary Symmetric Channel (BSC), Discrete Memory less Channel (DMC); Channel Capacity; Channel Coding Theorem; Information Capacity Theorem and its implications, The Shannon Limit. Block Codes: Linear Block Codes- Generator and Parity check matrices, Decoding of a linear block code, Standard array and Syndrome decoding; Hamming Codes; Cyclic Codes-Generator polynomial, Encoding and Decoding of cyclic codes; Burst error correction. Convolutional Codes: Introduction; Convolutional Encoder, Finite-State Machine Code, Tree and Trellis Representation of Convolutional Codes; ML Decoding of a Convolutional Code; Viterbi Algorithm; Low density parity check codes.

Learning Outcomes

On completion of this module the student should be able to1. Describe the concepts of information theory in measuring information content (AM1)2. Construct basic source codes and compare and contrast different types of codes in terms of their performance characteristics(AM1)3. Evaluate the performance of various coding techniques over communication channels (AM1,AM4,AM5)4. Design efficient error control codes, namely block codes and convolutional codes based on the given requirements (AM1,AM4)

Teaching / Learning Strategy

The main teaching method will be based on lectures. The students will be expected to perform directed reading exercises and self-learning exercises on emerging trends in information theory and coding techniques. Tutorials will be used to reinforce the module material and to discuss the issues raised by the directed reading.

Indicative Reading

-360 1. Haykin, S., 2001. Communication Systems. 4 th ed. John Wiley & Sons. Pvt. Ltd. Journal: IEEE transactions on Information theory

Transferrable Skills

Numeracy is developed in all analytical tasks. Communication skills are developed in report writing.

Module Structure

Activity Total Hours
Lectures (FT) 56.00
Independent Learning (FT) 100.00
Practicals (FT) 28.00
Assessment (FT) 16.00

Assessment Methods

Component Duration Weighting Threshold Description
Exam (School) 1.50 20.00 n/a Mid-term test - unseen test 90 minutes duration
Exam (Exams Office) 3.00 50.00 45% Final Examination - unseen exam 3 hour duration
Coursework 1 n/a 30.00 n/a Lab exercises and written report: 1500 words