Fig. 1 Media streaming over an ad hoc wireless network
[People] [Background] [Objectives] [Activities] [Findings] [Publications]
Prof. Andrea Goldsmith
Prof. Bernd Girod
Eric Setton
Rajiv Agarwal
Sachin Adlakha
Taesang Yoo
Xiaoqing Zhu
An ad-hoc wireless network is a collection of wireless nodes that self organize into a network without the help of an existing infrastructure. Some or possibly all of these nodes are mobile. Since the network can be deployed rapidly and flexibly, it is attractive to numerous potential applications, ranging from multi-hop wireless broadband Internet access to highway automation to voice and video communications for disaster areas.
On the other hand, great technical challenges abound. In particular, real-time media traffic such as voice and video typically have high data rate requirements and stringent delay constraints, whereas wireless nodes generally have limited resources in energy and bandwidth. Networking tasks such as routing also become demanding due to the lack of an infrastructure and the frequent topology changes. The goal of this project is therefore to explore this open research area of designing ad hoc wireless networks which can support delay critical media traffic.
In this project, we propose a new cross-layer framework for design of ad-hoc wireless networks to support delay-critical applications, such as conversational voice or real-time video. The framework incorporates adaptation across all layers of the protocol stack to leverage the flexibility offered by joint optimization of design parameters.
As illustrated in Fig. 2, adaptive link layer techniques will be used to adjust the capacity of individual wireless links to support delay-constrained traffic, possibly in multiple service classes; dynamic capacity assignment in the media access (MAC) layer will optimally allocate resources among various traffic flows; a congestion-optimized routing algorithm will provide multiple paths to real-time media streams; finally at the transport and application level, intelligent packet scheduling and error-resilient audio/video coding will be optimized for low-latency delivery over ad-hoc wireless networks.
The proposed framework will integrate the above components in a dynamic and iterative fashion. It allows the exchange of relevant information such as link capacities, traffic flows, packet deadlines and rate-distortion preamble of the source data across the entire protocol stack. Within this framework, we will develop a suite of new techniques for joint optimization, including:
The proposed research will systematically investigate how the above techniques should interact with each other and how they should evolve over time to continuously adapt to the changing wireless environment and traffic demands. Performance of the proposed new schemes will be ultimately evaluated in a small test bed of wireless nodes.
Results from the various research topics within the project are summarized below and are documented in publications.
We developed a mathematical framework to maximize single-user throughput in a wireless channel using symbol rate, packet length and constellation size of MQAM modulation as variables. The throughput is defined as the number of correctly received bits per second. We have studied trade-offs between the throughput and the communication range [Yoo_REPORT2005].
Previous research has studied the capacity regions of wireless ad hoc networks, which can be characterized by a collection of basic media-access (MAC) arrangements with maximum capacities on the active links. Any point in the capacity region can be obtained by time-sharing among the basic configurations. In this project, we investigate the joint optimization of capacity assignment and congestion-minimized routing. We characterize network congestion as the maximum link utilization ratio (traffic rate divided by link capacity), which is a quasi-convex objective function of both network flows and link capacities. Our scheme iteratively updates the flow and capacity assignments until the global optimum is achieved. We have compared the cross-layer scheme with an oblivious-layer approach, where capacities and flows are assigned to links in the network independently [Yoo_MMSP2004].
We extended the optimization framework to joint allocation of capacity, flow and rate of multiple video streams [Adlakha_ICC2007]. The goal of the optimization is to minimize the trade-off between encoded video distortion of all streams and overall network congestion. Performance of the proposed cross-layer scheme is compared with the oblivious-layer approach where capacity, flow and video rates are assigned at the MAC, network and application layers independently.
In our previous work of characterizing the capacity regions of wireless ad hoc networks, we have assumed the availability of basic link layer configurations. While this gives us the ultimate capacity region of the network, it requires a centralized operation to find the basic configurations and involves a large amount of off-line computation. Therefore, extending our previous work, we have developed a low-complexity distributed algorithm for searching for basic configurations. In this algorithm, each node periodically and randomly activates itself, chooses a destination, and sends a pilot signal. Each destination, upon listening to the pilot, measures its SINR and calculates the link data rate. This information is then broadcast and shared among the nodes to be used to create a new basic configuration. Finally, each node runs a simple optimization algorithm to decide if it will accept the new basic configuration or not, and update existing basic configurations accordingly. As more and more basic configurations are found, the achievable rate region will gradually expand, and eventually converge to the ultimate capacity region of the network.
We have investigated the joint optimization of user data compression in the application layer and channel coding in the physical layer. In [Ng_ICC2007], [Ng_ISIT2007] and [Ng_TIT2007],we solved the minimum expected distortion and optimal power distribution problem in Gaussian layered broadcast coding with successive refinement, for which previous results were only known in the high SNR regime or through numerical approximate solutions. The publication in [Ng_ISIT2007] won the Best Student Paper Award at the IEEE International Symposium on Information Theory 2007.
As the wireless medium is inherently time-varying due to changing channel conditions and user mobility, it is also important to consider source coding with uncertain side information. We investigate this problem in [Ng_ITW2007], and formulate the allocation of source coding rates for different fading states as a convex optimization problem.
We plan to incorporate additional network layers and study their benefits on the dynamic allocation of network resources. For example, in wireless sensor networks, it is common for the neighboring sensors to register correlated data. Therefore, it is natural to additionally include the link layer and network layer and consider distributed compression, cooperative scheduling, routing, and transmission of the correlated source data based on channel conditions and end-to-end distortion requirements [Gunduz_ISIT2007].
We have developed a congestion-optimized multipath routing algorithm, which finds multiple routes and performs optimal traffic partitioning to minimize a global congestion measure, i.e., packet delay averaged over all links. The algorithm is formulated as a convex optimization problem and can be solved efficiently to the global minimum. Experiments for video streaming over a simulated network have been carried out to verify the advantages of multipath routing over a heuristic load-balancing scheme. This work is documented in publications [Setton_ICME2004], [Setton_ICIP2004] and [Zhu_EURASIP2005].
A distributed version of the proposed routing algorithm is devised by decomposing the original objective into a sequence of minimum-cost routing problems, each of which can readily be solved by applying the classical Bellman-Ford algorithm. Experiments reported in [Zhu_ICME2005] have been carried out to compare the performance of the distributed algorithm with its centralized counterpart. The algorithm is used in [Zhu_PV2006] for a distributed scheme, performing joint routing and rate allocation of multiple video streams in a mesh network. In [Zhu_OPCOMM2006], we evaluate performance of the distributed routing and rate allocation scheme over a simulated 802.11 mesh network, with online estimation of link capacities and traffic rates.
In addition, we have investigated the impact of route update frequency on the performance of the proposed routing algorithm over a mobile network. We address the trade-off between timely route updates and overhead in link-state information collection. A hybrid scheme has been developed to combine frequent traffic re-partitioning with on-demand route updates.
Extending the rate-distortion optimized packet-scheduling framework (RaDiO) for Internet streaming, we have developed the new concept of congestion-distortion optimized streaming (CoDiO) for video streaming over bandwidth-limited channels. Here, the packet scheduler attempts to minimize the Lagrangian cost of end-to-end delay and media distortion by choosing the right transmission policy, dictating which packets to transmit, and when. The optimization is performed by an iterative descent algorithm and converges to a local minimum. Details of the algorithm and experimental results are documented in [Setton_MMSP2004].
The performance of CoDiO scheduling over a mobile ad hoc wireless network and its comparison to conventional retransmission-based schemes such as ARQ are studied in [Setton_ISCAS2005].
Given a set of routes and a set of packets to deliver with a delay constraint, we have investigated the problem of optimal rate allocation among multiple paths, either to maximize throughput or to minimize expected received video distortion.
We derived an analytical solution for rate division among multiple paths to maximize the proportion of packets meeting an end-to-end delay constraint. The optimal solution can be interpreted as water-filling of data rate relative to the number of hops in each path, where each hop is weighted by its throughput [Agarwal_REPORT2004].
An alternative approach is to jointly consider the source coder performance and the impact of network congestion, and to minimize end-to-end distortion of the transmitted media. In [Zhu_ICIP2004], we investigate such a rate allocation scheme. We use simple models for the rate-distortion trade-off of a wavelet coder and for the packet delay distribution over each path. The rate allocation is optimized for a group of video frames at a time, over multiple paths. The optimization is performed via the gradient descent method, using a heuristic scheme for initialization. Network simulations are performed to compare its performance with a heuristic scheme that only optimizes the encoder performance and balances the traffic loads proportionally among multiple paths.
To support multiple users in the ad hoc network with fair and efficient resource allocation, we have formulated the source rate allocation problem to minimize a common objective function: the total decoded distortion of all streams.
For applications with multi-camera surveillance networks, a centralized scheme is proposed in [Zhu_PCS2004] with closed-form solutions, incorporating information from the application layer, the network layer and the link layer. A delay-aware rate control scheme is studied in [Zhu_MMSP2005], where the encoder frame rate and quantizer step size are adjusted at each video sender, to achieve overall congestion-distortion optimized performance in a distributed manner. In [Baccichet_PCS2006], the network-aware rate adaptation is combined with automatic frame skipping and region-of-interest coding for each frame, depending on the content of the captured surveillance pictures. The reference pictures used in predictive coding are selected based on past acknowledgments so as to avoid error propagation in case of packet losses.
For the more general setting of multi-stream video transmission over ad hoc wireless networks, a distributed solution based on the method is studied in [Zhu_ICIP2005]. To minimize total distortion of all video streams in the network while abiding to capacity constraints each wireless link, the optimization procedure is decomposed into source rate adaptation at each video sender and link price updates relaying node. The effectiveness of the proposed scheme has been confirmed in NS-2 simulations with 802.11 wireless nodes.
The problem of joint routing and rate allocation is studied in [Zhu_PV2006], where a common trade-off between encoded video distortion and increased network congestion is used to guide the rate allocation at each stream in a decentralized manner. The set of routes for each stream is also chosen in a congestion-minimized fashion, using the distributed algorithm proposed in [Zhu_ICME2005]. We compare the performance of the proposed media-aware joint routing and rate allocation scheme against the conventional approach using TCP-Friendly Rate Control (TFRC) over simulated 802.11 networks [Zhu_OPCOMM2006].
The interactions due to interference and traffic contention among wireless links are analyzed based on a stochastic system model [Zhu_ICME2006]. In this work, the performances of various congestion control policies, using stochastic dynamic programming or a greedy heuristic, are compared for the scenario of two streams with different, time-varying video contents competing over a wireless link with time-varying capacity.
The distributed rate allocation algorithm is further extended to accommodate wireless links with heterogeneous packet loss rates or transmission speeds. The application scenario of multiple high-definition (HD) video streams sharing a single-hop wireless network is considered in [Zhu_ICIP2007]. In this case, the impact of allocated video rate on each link is conveniently characterized in terms of channel time, the fraction of time each stream occupies the common wireless radio channel for transmission. We propose a distributed channel time allocation protocol to minimize the total video distortion of all streams without incurring excessive network congestion. The protocol relies on cross-layer information exchange between link state estimation at the MAC layer, and video rate adaptation at the application layer.
In the more general case of multi-hop video streaming over wireless mesh networks, we propose a distributed allocation protocol to minimize the trade-off between total video distortion and overall network congestion [Zhu_ISMW2007]. Its performance is compared against TFRC in simulations involving simultaneous streaming of multiple HD video sequences over an 802.11a network comprising heterogeneous link speeds.
A summary of the above work is included in a recent tutorial paper on video streaming over wireless networks [Zhu_EUSIPCO2007].
In addition to heterogeneity in wireless link capacities, the video streams traversing the network differ in terms of frame rate, resolution and content complexity. To explore the best quality tradeoff among the streams, we conduct subjective viewing tests and collect viewers' mean-opinion-score (MOS) of various quality combinations of high-definition (HD) and standard-definition (SD) video snapshots [Zhu_ICIP2008]. Test results are used to derive a subjective quality model for heterogeneous video streams competing over the same network, which can guide the optimization of rate allocation among these streams.
In [Zhu_VCIP2008], we develop an extension of the cross-layer rate allocation protocol for wireless video multicast. The goal of the protocol is to minimize total video distortion of all peers in the multiple multicast trees, while avoiding excessive network utilization. For video representation, the scalable video coding (SVC) extension of the H.264/AVC standard is adopted to allow rate adapation at each peer within each multicast tree. Performance of the rate allocation protocol is compared against a heurstic scheme based on TFRC, in ns-2 network simulations of SD video streams over single or multiple multicast trees.
We further investigate the scenario where multiple wireless networks (e.g., cellular, WiMax, IEEE 802.11a/b/g) are simultaneously available for Internet access. We develop and evaluate an analytical framework for optimal video rate allocation, based on observed available bit rate and round trip time over each access network, as well as the video distortion-rate characteristics. The rate allocation is formulated as a convex optimization problem that minimizes the sum of expected distortion of all video streams. We then present a distributed approximate solution to media- and network-aware allocation. Performance of the proposed allocation scheme is compared against robust rate control based on H-infinity optimal control and two heuristic schemes employing TCP-style additive-increase-multiplicative-decrease (AIMD) rate allocation. The details of this collaboration work is reported in [Zhu_MM2007]. The paper won the Best Student Paper Award at the ACM Multimedia Conference 2007.
In collaboration with Stanford's Wireless Sensor Networks Laboratory we studied a distributed routing scheme for event driven applications in which low-bandwidth periodic data and bursty event-based data packets are routed through the networks. The two data types differ in their bit-rate requirements. Bursts of event packets can lead to significant queue build-up at certain nodes that could result in high end-to-end delay unless we distribute the load across multiple paths. The proposed routing scheme associates a cost at each node with its queue length and/or its remaining energy. These costs are in addition to the cost solely based on node positions. Incorporation of queue-related costs significantly lowers the average and the maximum delays and drastically reduces the node buffer requirements.
The details of this work can be found in the papers [Savidge_BASENETS2005] and [Savidge_COGIS2006].
In [Setton_ICME2004] and [Setton_ICIP2004], we provide a video distortion model which captures both the encoder performance and the impact of network congestion on received video quality. The model applies a parameterized encoder distortion model and uses the M/M/1 queuing model to characterize packet delay statistics over a multi-hop ad-hoc wireless network. A detailed account of the work is available at [Zhu_EURASIP2005].
Time variation and unpredictability of the wireless channel calls for error-resilient coding and transmission of the video streams. We have adopted results of the NSF project 'Wyner-Ziv Coding of Video' to develop a novel error protection scheme for video transport over ad hoc wireless networks [Zhu_VCIP2005]. Unlike layered video coding, the proposed systematic loss error protection (SLEP) scheme supports backward compatibility to legacy video coding systems, and when compared to conventional FEC methods, results in graceful degradation of video quality over a wide range of wireless channel conditions. Based on an end-to-end video distortion model, selection of the best available path and the most appropriate video description has been investigated in network simulation experiments.
We also analyze the advantages of streaming with SP and SI frames. Unlike systems that rely on periodic transmissions of I frames regardless of transmission errors, the proposed algorithm takes advantage of feedback to insert SI frames on an as-needed basis to stop error-propagation. Therefore, the amount of robustness of the stream is adjusted to the network conditions on the fly, rather than being determined during encoding. This is particularly helpful when the channel statistics are time-varying, e.g., over wireless links. In the analysis and experiments carried out in [Setton_CSVT2006] and [Setton_VCIP2005], we evaluate the rate-distortion coding efficiency of SP and SI frames, as well as the performance of adaptive streaming using SP/SI frames, as opposed to periodic I-frame insertion. We analyze the effect of bit-rate, packet losses and delay on the performance of the two systems. Simulations are conducted over a simulated network with encodings produced with our implementation of an SP/SI frame codec, recently adopted by the Joint Video Team and integrated into the reference software encoder.
We have also extended our cross-layer design framework to peer-to-peer overlay networks. For peer-to-peer multicast streaming, a source distributes real-time video to a large population of hosts by making use of their forwarding capacity rather than relying on dedicated media servers. As peers may disconnect at any time, maintaining the nodes in a tree-based overlay structure requires the design of adaptive control protocols bearing many similarities with those needed for wireless ad hoc networks.
In light of the findings highlighted in [Setton_WCOMM2005], cross-layer design is crucial to the good performance of the overall network. In [Setton_P2PMMS2005] and [Setton_ICME2006], we study a new peer-to-peer multicast protocol and analyze the gains that video coding and prioritized packet scheduling at the application layer can provide. Experiments for several hundred hosts simulated in NS-2 illustrate the benefits of sharing information from the application layer to the transport layer. We have confirmed these results in a real-time implementation which has been tested with PlanetLab experiments [Baccichet_ICME2007]. A review of our work in this area will appear in an invited paper in the Proc. IEEE [Setton_ProcIEEE2008].
We built a simple testbed to highlight the limitation of TCP for supporting video streaming over wireless in the presence of heterogeneous link speeds, using off-the-shelf components. The setup mimics a wireless home network with several laptops operating in 802.11 ad-hoc mode. In the experiments, a file transfer session over a slow link competes with an ongoing video streaming session over a fast link, both using TCP ports. Video record of the experiments are presented in the workshop The Future of TCP: Train-Wreck or Evolution?. Presentation slides and demo video can be found here.
A brief survey of some of the early findings of this project is published in the Special Issue on 'Wireless Video' in the IEEE Wireless Communication Magazine [Setton_WCOMM2005]. More recently, we highlighted the benefits of cross-layer design and media-aware resource allocation in a tutorial paper on video streaming over wireless networks [Zhu_EUSIPCO2007].
As part of our framework for maximization of user throughput over a wireless channel, we obtained a closed form solution for the optimal packet length as a function of channel quality. We also established a set of design rules. When the channel gain is high, the maximum symbol rate should be used with the maximum constellation size. When the channel gain is low, packets should be protected against errors by adding redundancy in the transmission. Finally, we demonstrated that an optimal design of parameters leads to a simple throughput formula that can be used in the analysis and design of upper layers [Yoo_REPORT2005].
In the experiments reported in [Yoo_MMSP2004], we found that our new cross-layer optimization scheme tends to assign capacity only to links with active traffic, hence achieving higher efficiency in resource utilization. Experimental results obtained through network simulations illustrate the advantages of joint capacity and flow assignment over oblivious layers. We achieve much better quality of received video (gains up to 10 dB in PSNR) by supporting a several times higher data rate over the same network. While multipath routing is essential to high data rate in the oblivious-layer design, joint capacity and flow assignment achieves efficient resource utilization regardless of the number of routes.
Similarly, when multiple video streams are present over a network, joint optimization of capacity, flow and video rate allocation can achieve significant performance gains over oblivious-layer allocation. Our simulation results in [Adlakha_ICC2007] show 4-6 folds of improvement in the maximum sustainable rate of each video stream, corresponding to up to 5 dB gains in PSNR of the video quality.
In the transmission of a delay-limited source without CSI at the transmitter, we solved the minimum expected distortion and optimal power distribution in Gaussian layered broadcast coding with successive refinement. It is found that, as SNR increases, the optimal power allocation for the higher layers remains unchanged; rather the extra power is allocated towards the lower layers. On the other hand, as the bandwidth ratio b (channel uses per source symbol) tends to zero, the power distribution that minimizes expected distortion converges to the power distribution that maximizes expected capacity. Details of this work are reported in [Ng_ICC2007], [Ng_ISIT2007] and [Ng_TIT2007].
In the scenario of source coding with uncertain side information, we formulate the allocation of source coding rates for different fading states as a convex optimization problem. It is shown that under discretized Rayleigh fading, the optimal rate allocation puts almost all rate into the base layer associated with the worst-case fading. This implies that uncertain side information yields little performance benefit over no side information. Moreover, as the source coding rate increases, the benefit of uncertain side-information decreases [Ng_ITW2007].
Experimental results in [Setton_ICME2004] for video streaming over a simulated network illustrate the advantages of multipath routing due to bandwidth aggregation. It is also shown that the proposed traffic partitioning algorithm outperforms a heuristic load-balancing scheme by better utilization of link capacity resources on different paths, especially when there are shared links among the multiple paths.
For the distributed routing algorithm, results in [Zhu_ICME2005] show that the distributed approximation of the problem leads to only a small loss of performance in terms of received video quality in comparison to the centralized solution. For mobile wireless network, our studies show that a balance between timely route updates and overhead in link state information collection can be achieved by a hybrid scheme combining frequent traffic re-partitioning with on-demand route updates.
Performance of the new congestion-distortion optimized scheduler (CoDiO) has been studied under different simulation scenarios in [Setton_MMSP2004] and [Setton_ISCAS2005]. Over a network with bottleneck links, it is compared with rate-distortion optimized scheduling (RaDiO). Since end-to-end delay is a non-decreasing function of the total transmission rate, the proposed CoDiO scheduler exhibits similar rate-distortion performance as RaDiO, while reducing the end-to-end delay significantly (by as much as 50%), hence reducing network congestion substantially. Over a mobile ad hoc wireless network, it is shown that CoDiO outperforms conventional retransmission-based schemes such as ARQ over a range of playout deadlines.
Analytical study of the rate division problem among multiple paths shows that for maximizing the proportion of the packets meeting an end-to-end delay constraint, the optimal solution can be interpreted as water-filling of data rate relative to the number of hops in each path, where each hop is weighted by its throughput [Agarwal_REPORT2004].
Simulation results in [Zhu_ICIP2004] show that by taking into consideration both the source coder performance and rate-delay constraints over each path, the proposed multipath rate allocation scheme leads to lower expected end-to-end video distortion than the heuristic scheme, which only optimizes the encoder performance and performs load-balancing among multiple paths.
For a multi-camera 802.11 surveillance network, experiments in [Zhu_PCS2004] verify that the proposed centralized rate allocation scheme outperforms a simple scheme based on fixed-quality transmission of all video sequences, as well as the TCP-Friendly (TFRC) rate control approach commonly used over the Internet. The additional benefits brought by reducing the encoding frame rate during inactive periods of surveillance video and distributed delay-aware rate control at each video stream are confirmed by simulations in [Zhu_MMSP2005]. Further experiments show performance improvement in terms of maximum sustainable video quality of all seven surveillance sequences, by virtue of region-of-interest coding, automatic frame skipping, quantizer step size adaptation and reference picture selection [Baccichet_PCS2006].
For multi-stream video transmission over general ad hoc wireless networks, experiments in [Zhu_ICIP2005] have shown that the proposed distributed rate allocation scheme based on the subgradient method can effectively react to changes in network traffic conditions and variations in video content complexities, achieving minimized video distortion for all streams.
Network simulations in [Zhu_PV2006] confirm the effectiveness of the distributed algorithm for joint routing and rate allocation. It is shown that the iterative rate allocation procedure among multiple streams converges fast to the optimal solution, and that the congestion-distortion trade-off achieved by the distributed algorithm is close to the upper bound of performance from centralized optimization.
Simulation results of multi-user video streaming over 802.11 mesh networks confirm the effectiveness of the proposed media-aware joint routing and rate allocation scheme. It is shown to achieve higher average encoded video quality while maintaining lower overall network congestion than the media-unaware TFRC approach [Zhu_OPCOMM2006].
The analysis work in [Zhu_ICME2006] shows that the distortion-congestion trade-off of the multiple video streams sharing a common network can be optimized by performing congestion control during live encoding of each of the streams, in a distributed manner. While the best performance is achieved by using stochastic dynamic programming, a greedy heuristic can attain most of the gains with reduced computational complexity.
Simulation results of multiple high-definition (HD) video streams over a shared 802.11a wireless network in [Zhu_ICIP2007] demonstrate that the proposed rate allocation scheme converges rapidly in case of abrupt changes in the network (e.g., when a new stream joins or an existing stream leaves), and yields more stable allocation results and lower packet delivery delays than TFRC. In comparison with the approach without rate adaptation, the proposed scheme is proactive in avoiding congestion by voluntarily dropping less important video packets. Therefore, it can sustain multiple video streams at acceptable received video quality even under harsher channel conditions.
Similar observations can be made from the experiments involving both single-hop and multi-hop video streaming sessions [Zhu_ISMW2007]. When different video contents are streamed over links with heterogeneous speeds, the media-aware cross-layer allocation scheme can achieve more balanced quality among the video streams, with higher average video quality and lower network congestion than TCP-Friendly Rate Control. Performance gain ranges between 0.7 dB to 2.1 dB in terms of average video quality in PSNR, depending on the number of participating video streams, distortion-rate characteristic of each stream, as well as heterogeneity in the wireless link speeds.
In a recent tutorial paper, we highlight these benefits of media-aware rate allocation and cross-layer design for high-definition video streaming over wireless home networks [Zhu_EUSIPCO2007].
In the case of video streams with heterogeneous resolutions and frame rates sharing the same wireless network, results from the subjective viewing tests in [Zhu_ICIP2008] show that the subjectively preferred allocation can be closely approximated by minimizing the total mean-squared-error (MSE) distortion of all pariticipating streams. Such allocation achieves higher mean-opinion-scores (MOS) as rated by a viewer than allocation from media-unaware TFRC. The improvement is more significant in situations involving more demanding video sequences and more stringent total network resources.
An extension of the rate allocation protocol for wireless video multicast is presented in [Zhu_VCIP2008]. Performance comparison against a TFRC-based heuristic scheme shows that the proposed media-aware allocation scheme tends to allocate higher rates for peers experiencing higher link speeds, leading to higher overall video quality.
In addition, the experimental results reported in [Zhu_MM2007] show that proactive rate allocation schemes based on either convex optimization or H-infinity robust control formulations lead to smaller rate fluctuations, lower delays and significantly reduced packet losses than reactive heuristic schemes following TCP-style allocations. The media-aware approach further exploits explicit knowledge of the video distortion-rate characteristics. It can therefore achieve more balanced video quality than the other schemes.
We observe that adding the queue size and energy costs to the cost functions of greedy routing and incorporating a packet type prioritization scheme allows various trade-offs to be made between different performance factors of the network. It is observed that the average delay of event packets in hybrid networks is significantly decreased in routing schemes with queue size considerations. The maximum delay for periodic packets is improved significantly, when node queue sizes are considered for routing decisions. This delay reduction is due to the load balancing of the network traffic, which has the additional advantage of distributing the energy consumption of the overall network. Experimental results are reported in [Savidge_BASENETS2005] and [Savidge_COGIS2006].
For low-latency video streaming over a throughput-limited network, decoded video distortion is affected by both encoder quantization and packet loss due to late arrivals caused by congestion. The influence of both are captured in our proposed video distortion model in [Setton_ICIP2004] and [Zhu_EURASIP2005]. Experimental results validate the accuracy of the model for a variety of video sequences, coding structures, playout deadlines, routing scenarios, packet loss rates for performance evaluation of video streaming over ad-hoc wireless networks.
Over an ad hoc wireless network, when packet loss ratio fluctuates over time, the proposed SLEP yields graceful degradation in the received video quality, hence superior performance compared to conventional forward error correction (FEC). When multiple paths are available for transmission, with different bit-rates, different packet loss ratios and different number of hops, it is not always straightforward to transmit on the path with the highest bandwidth or the lowest packet loss ratio or the fewest number of hops. Using a video distortion model for SLEP, we show that it is possible to select the best path among the available alternatives[Zhu_VCIP2005].
SP and SI frames provide an attractive alternative to streaming with I frames when feedback is available and propagation delay is small compared to the maximum tolerable latency. In these cases, the performance is superior both in terms of congestion-distortion and in terms of rate-distortion trade-offs. The performance gap is larger for low motion sequences as this causes larger differences between I and SP frames. It is also more pronounced at lower bit rates. In the results presented in [Setton_CSVT2006] and [Setton_VCIP2005] improvement of received video quality ranges from 0.5 dB to 4 dB in PSNR for various test sequences.
We have designed a new distributed peer-to-peer video multicast protocol targeted for low-latency streaming [Setton_ProcIEEE2008]. The analysis of dynamic and unreliable wireless ad hoc networks provides important insights to the design of a light and distributed protocol able to organize a large population of peers into a multi-tree overlay. The protocol is robust to node disconnections and easily scales up to support hundreds of peers. Control traffic accounts for just a few percent of the video traffic (for streams of 200 kilobits per second or more). Video streaming latencies on the order of seconds are achieved for up to 300 nodes.
We have extended the video distortion model, discussed above, to the overlay network formed by peers of a wired network. In this case, multiple senders transmit parts of a video stream to the same receiver. The model is useful to predict how the video quality varies as a function of the rate. Compared to the results for wireless ad hoc networks, we observed comparable performance degradation when the rate approaches the capacity of the links used for streaming.
We have developed adaptive error-resilient streaming techniques suitable for peer-to-peer networks to mitigate error-propagation through the use of H.264 SP and SI picture types. In this scenario any of the forwarding peers may optimize the trade-off between coding efficiency and robustness and transmit SI pictures on an as-needed basis only.
We have also extended CoDiO scheduling of video from the sender-driven algorithm discussed above, to a light-weight receiver-driven scheduler which optimizes retransmission requests. In this context, when a receiver detects missing video packets, it can decide which packets to request and when to maximize video distortion while minimizing the induced network congestion. Details on the experimental results are reported in [Setton_P2PMMS2005] and [Setton_ICME2006].
In the demo presented at The Future of TCP Workshop, we show the impact of a file transfer session over a slow link on an ongoing video streaming session over a fast link. TCP leads to approximately equal throughput for both sessions, despite significant difference in their underlying link transmission speeds. This leads to unbalanced resource allocation among the two application flows and severe quality degradation of the video stream.
This material is based upon work supported by the National Science Foundation under Grant CCR-0325639. Any opinions, findings and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation (NSF).
Last Modified: 06/23/2008