Rate-distortion optimized mode selection based on multi-path channel simulation

Davide Bertozzi, Markus Gärtner

PROPOSAL:

In recent years new transmission media such as wireless channels or the Internet became 
more interesting for video applications as the available capacity constantly increased. 
Video bitstreams in such channels, however, may incur hardly predictable errors due to 
the best effort nature of the Internet and to the unreliability of wireless links.
The problem of maintaining high video quality is exacerbated whenever hybrid video 
coding with motion compensated prediction (MCP) is used in conjunction with highly 
erroneous channels. This is mainly due to the reduced redundancy between successive 
frames which may cause the prediction loop to propagate errors to following frames. 
Therefore, some frames have to be encoded in intra-frame mode to reset the frame buffer 
of the predictor. As the demands of intra-coding in terms of bit rate are significantly 
higher than for inter-coding the optimal tradeoff between compression efficiency and 
robustness is sought. 
Optimal mode switching can be the result of  feedback information provided by the 
decoder about the current transmission error statistics. This solution has to cope with a 
long delay time and therefore excludes many potential applications. Other techniques that 
do not make use of feedback rely on the accuracy with which encoders try to estimate the 
overall distortion of the reconstructed frame at the decoder side. Although many solutions 
following this approach have been proposed (heuristic, periodic, contiguous or forced 
updates, intrinsic error vulnerability, distortion thresholds methods), only a few 
"consistent algorithms" for optimum selection are known at the moment (rate-distortion 
optimized switching mode, per-pixel distortion estimation, one-frame limited estimation). 
Furthermore, some of them suffer from restrictions and limitations which affect the 
overall estimation accuracy. 
In this project we shall investigate a modified H.263 encoder design for hybrid video 
coding which incorporates a multi-path channel simulator that estimates the macroblock 
distortion for the decoder. The basic idea is to take into consideration those errors as well 
that are  either induced by the channel or propagated by the prediction loop even beyond 
more than one frame. For this purpose, the proposed encoder architecture predicts 
distortion by running an entire channel-decoder simulation. To better characterize errors 
incurred by the video bitstream, the simulator consists of many channel models in 
parallel, corresponding to several possible error patterns affecting the transmitted data. 
The coding mode selection is based on the overall cost function which is calculated as the 
weighted average of the individual cost for each path. This way we expect to achieve a 
more realistic prediction of the distortion incurred by transmission over the actual 
channel which allows better decision between inter- and intra-frame coding. We shall 
also investigate how many of such blocks in parallel are needed and how the error 
patterns have to be designed in order to obtain a high level of accuracy.


WORKPLAN:

1. Feb. – 6. Feb.: Read relevant papers, Prepare project proposal and talk
6. Feb. – 13. Feb.: Become familiar with the H.263 software, preliminary experiments 
with the H.263 encoder, get suitable video data
13. Feb. – 20. Feb.: Implement a multi-path simulator in H.263, Consider channel error 
patterns
20. Feb. – 27. Feb.: Further refinement of the architecture, conduct measurements using 
standard sequences
27. Feb. – 6. Mar.: Acquire further measurements, write report and prepare final talk


REFERENCES:

1. R. Zhang, S. L. Regunathan, K. Rose: Optimal Intra/Inter Mode Switching for Robust 
Video Communication Over The Internet; 1999

2. G. Cote, F. Kossentini: Optimal Intra Coding Of Blocks For Robust Video 
Communication Over The Internet, EURASIP Image Communication, Special issue on 
Real-Time Video over the Internet; 1999

3. J. Y. Liao, J. D. Villasenor: Adaptive Intra Update For Video Coding Over Noisy 
Channels; Proc. IEEE Int. Conference on Image Processing, ICIP'96, Lausanne, 
Switzerland, vol. 3, pp. 763-766; Sept. 1996

4. N. Faerber, K. Stuhlmueller, B. Girod: Analysis Of Error Propagation In Hybrid Video 
Coding With Application To Error Resilience; 1999

5. B. Girod: Motion Compensating Prediction With Fractional-Pel Accuracy, IEEE 
Transactions on Communications, vol. 41, no. 4, pp. 604-612; April 1993

6. G. Cote, S. Wenger, F. Kossentini: A Standard-Based System For Robust Video 
Transmission Over The Internet; 1999

7. T. Wiegand, N. Faerber, K. Stuhlmueller, B. Girod: Error-Resilient Video 
Transmission Using Long Term Memory Motion-Compensated Prediction; 2000

8. D. Wu, Y. T. Hou, B. Li, W. Zhu, Y.Q. Zhang, H.J. Chao: An End-To-End Approach 
For Optimal Mode Selection In Internet Video Communication: Theory And Application; 
2000

9. G. Cote, S. Shirani, F. Kossentini: Optimal Mode Selection And Synghronization For 
Robust Video Communication Over Error-Prone Networks; 2000

10. Y. Wang, Q.F. Zhu: Error Control And Concealment For Video Communication - A 
Review, in Proc. IEEE, vol. 86, pp. 974-996; May 1998