Resynchronization and remultiplexing for transcoding to H.264/AVC

H.264/MPEG-4 AVC standard appears highly competitive due to its high efficiency, flexibility and error resilience. In order to maintain universal multimedia access, statistical multiplexing, or adaptive video content delivery, etc., it induces an immense demand for converting a large volume of existing multimedia content from other formats into the H.264/AVC format and vice versa. In this work, we study the remultiplexing and resynchronization issue within system coding after transcoding, aiming to sustain the management and time information destroyed in transcoding and enable synchronized decoding of decoder buffers over a wide range of retrieval or receipt conditions. Given the common intention of multiplexing and synchronization mechanism in system coding of different standards, this paper takes the most widely used MPEG-2 transport stream (TS) as an example, and presents a software system and the key technologies to solve the time stamp mapping and relevant buffer management. The solution reuses previous information contained in the input streams to remultiplex and resynchronize the output information with the regulatory coding and composition structure. Experimental results showed that our solutions efficiently preserve the performance in multimedia presentation.     

Systematic lossy error protection based on H.264/AVC redundant slices and flexible macroblock ordering

INTRODUCTION In typical video transmission systems, a video signal is compressed, and the resulting bit stream is transmitted over an error-prone channel. The errors may consist of symbol errors caused by fading, as observed for wireless channels, or packet erasures caused by congestion, as observed in the Internet. If a received video packet contains errors, then the portion of the video signal contained in the packet is lost and must be concealed. Error concealment schemes alone cannot …     

RTP payload format for H.264/SVC scalable video coding

The scalable extension of H.264/AVC, known as scalable video coding or SVC, is currently the main focus of the Joint Video Team‘s work. In its present working draft, the higher level syntax of SVC follows the design principles of H.264/AVC.Self-contained network abstraction layer units (NAL units) form natural entities for packetization. The SVC specification is by no means finalized yet, but nevertheless the work towards an optimized RTP payload format has already started. RFC 3984, the RTP payload specification for H.264/AVC has been taken as a starting point, but it became quickly clear that the scalable features of SVC require adaptation in at least the areas of capability/operation point signaling and documentation of the extended NAL unit header. This paper first gives an overview of the history of scalable video coding, and then reviews the video coding layer (VCL)and NAL of the latest SVC draft specification. Finally, it discusses different aspects of the draft SVC RTP payload format, including the design criteria, use cases, signaling and payload structure.     

RTP-based broadcast streaming of high definition H.264/AVC video: an error robustness evaluation

In this work, we present an evaluation of the performance and error robustness of RTP-based broadcast streaming of high-quality high-definition (HD) H.264/AVC video. Using a fully controlled IP test bed (Hillestad et al., 2005), we broadcast high-definition video over RTP/UDP, and use an IP network emulator to introduce a varying amount of randomly distributed packet loss. A high-performance network interface monitoring card is used to capture the video packets into a trace file. Purpose-built software parses the trace file, analyzes the RTP stream and assembles the correctly received NAL units into an H.264/AVC Annex B byte stream file, which is subsequently decoded by JVT JM 10.1 reference software. The proposed measurement setup is a novel, practical and intuitive approach to perform error resilience testing of real-world H.264/AVC broadcast applications. Through a series of experiments, we evaluate some of the error resilience features of the H.264/AVC standard, and see how they perform at packet loss rates from 0.01 % to 5%. The results confirmed that an appropriate slice partitioning scheme is essential to have a graceful degradation in received quality in the case of packet loss. While flexible macroblock ordering reduces the compression efficiency about 1 dB for our test material, reconstructed video quality is improved for loss rates above 0.25%.     

RTP-based broadcast streaming of high definition H.264/AVC video: An error robustness evaluation

In this work, we present an evaluation of the performance and error robustness of RTP-based broadcast streaming of high-quality high-definition (HD) H.264/AVC video. Using a fully controlled IP test bed (Hillestad et al., 2005), we broadcast high-definition video over RTP/UDP, and use an IP network emulator to introduce a varying amount of randomly distributed packet loss. A high-performance network interface monitoring card is used to capture the video packets into a trace file. Purpose-built software parses the trace file, analyzes the RTP stream and assembles the correctly received NAL units into an H.264/AVC Annex B byte stream file, which is subsequently decoded by JVT JM 10.1 reference software. The proposed measurement setup is a novel, practical and intuitive approach to perform error resilience testing of real-world H.264/AVC broadcast applications. Through a series of experiments, we evaluate some of the error resilience features of the H.264/AVC standard, and see how they perform at packet loss rates from 0.01% to 5%. The results confirmed that an appropriate slice partitioning scheme is essential to have a graceful degradation in received quality in the case of packet loss. While flexible macroblock ordering reduces the compression efficiency about 1 dB for our test material, reconstructed video quality is improved for loss rates above 0.25%.     

Macroblock-level decoding and deblocking method and its pipeline implementation in H.264 decoder SOC design

INTRODUCTION H.264 (Wiegand et al., 2003) is the latest video coding standard developed by the ISO/IEC Moving Picture Experts Group (MPEG) and the ITU-T Video Coding Experts Group (VCEG). Besides many new technologies which can bring high compression effi-ciency, H.264 also provides other tools to flexibly adapt to different transmission networks, such as flexible macroblock ordering (FMO) (Wenger and Horowitz, 2002a; 2002b) for dealing with data er-rors/losses in error prone…     

Frame loss error concealment for SVC

INTRODUCTION Real-time transmission of video data in network environments, such as wireless and Internet, is a challenging task, as it requires coding efficiency, network friendliness and error robustness. Scalability is a possible solution for network adaptive applica- tions. The Scalable Extension of H.264/AVC (SVC) (JVT of ISO/IEC MPEG & ITU-T VCEG, 2005b) aims at achieving both high compression performance and adaptivity for video delivery over heterogeneous networks. SVC…     

Dominant edge direction based fast intra mode decision in the H.264/AVC encoder

The H.264/AVC video coding standard uses an intra prediction mode with 4×4 and 16×16 blocks for luma and 8×8 blocks for chroma. This standard uses the rate distortion optimization （RDO） method to determine the best coding mode based on the compression performance and video quality. This method offers a large improvement in coding efficiency compared to other compression standards, but the computational complexity is greater due to the various intra prediction modes. This paper proposes a fast intra mode decision algorithm for real-time encoding of H.264/AVC based on the dominant edge direction （DED）. The DED is extracted using pixel value summation and subtraction in the horizontal and vertical directions. By using the DED, three modes instead of nine are chosen for RDO calculation to decide on the best mode in the 4×4 luma block. For the 16×16 luma and the 8 × 8 chroma, only two modes are chosen instead of four. Experimental results show that the entire encoding time saving of the proposed algorithm is about 67% compared to the full intra search method with negligible loss of quality.     

A fast motion estimation algorithm for mobile communications

The limitation of processing power, battery life and memory capacity of portable terminals requires reducing encoding complexity in mobile communications. Motion estimation (ME) is the most computationally intensive module in a typical video codec, which determines not only the encoder's performance but also the reconstructed video quality. In this paper, a fast ME algorithm for H.264/AVC baseline profile coding is proposed based on the analysis of motion vector field and error surface, and the statistical distributions of different type macroblocks (MBs). Simulation results showed that: in comparison with MVFAST, the proposed algorithm can decrease the computational load over 7.2% with no requirement of expanding memory capacity while maintaining the same video quality as MVFAST. Furthermore, its simplicity makes it easy to be implemented on hardware.     

A parallel memory architecture for video coding

To efficiently exploit the performance of single instruction multiple data （SIMD） architectures for video coding, a parallel memory architecture with power-of-two memory modules is proposed. It employs two novel skewing schemes to provide conflict-free access to adjacent elements （8-bit and 16-bit data types） or with power-of-two intervals in both horizontal and vertical directions, which were not possible in previous parallel memory architectures. Area consumptions and delay estimations are given respectively with 4, 8 and 16 memory modules. Under a 0.18-pm CMOS technology, the synthesis results show that the proposed system can achieve 230 MHz clock frequency with 16 memory modules at the cost of 19k gates when read and write latencies are 3 and 2 clock cycles, respectively. We implement the proposed parallel memory architecture on a video signal processor （VSP）. The results show that VSP enhanced with the proposed architecture achieves 1.28× speedups for H.264 real-time decoding.     

浅谈H.264视频编解码标准的先进性

H.264是新一代视频编解码标准,其采用了一系列新的数据压缩、成像和网络传输技术,使编码效率、图像质量与传输效果得到同步提升,体现了视频编解码技术的最新成果和流行趋势.文章从发展历程、技术创新、应用前景等方面论证了H.264的先进性和实用性.     

基于相邻帧相关性的帧间宏块模式选择算法

H．264／AVC在帧间运动估计／补偿编码过程中定义了7种帧间编码模式，并采用率失真优化全遍历策略来选择最优模式，这使计算复杂度急剧增加．针对这一问题，阐明了需对如何准确、快速地对块预测模式做出选择进行相关的研究．并就利用自然图像序列中相邻帧所表现出的强相关性提出了一种快速的帧间编码模式选择算法，通过计算部分编码模式来减少计算的复杂性．实验结果表明，该方法平均节省时间约18％，比特率平均下降约0．02％，而PSNR平均仅仅降低0．02dB．     

H.264/AVC error resilience tools suitable for 3G mobile video services

The emergence of third generation mobile system (3G) makes video transmission in wireless environment possible, and the latest 3GPP/3GPP2 standards require 3G terminals support H.264/AVC. Due to high packet loss rate in wireless environment, error resilience for 3G terminals is necessary. Moreover, because of the hardware restrictions, 3G mobile terminals support only part of H.264/AVC error resilience tool. This paper analyzes various error resilience tools and their functions, and presents 2 error resilience strategies for 3G mobile streaming video services and mobile conversational services. Performances of the proposed error resilience strategies were tested using off-line common test conditions. Experiments showed that the proposed error resilience strategies can yield reasonably satisfactory results.     

Unequal decoding power allocation for efficient video transmission

We present an unequal decoding power allocation （UDPA） approach for minimization of the receiver power consumption subject to a given quality of service （QoS）, by exploiting data partitioning and turbo decoding. We assign unequal decoding power of forward error correction （FEC） to data partitions with different priority by jointly considering the source coding, channel coding and receiver power consumption. The proposed scheme is applied to H.264 video over additive white Gaussion noise （AWGN） channel, and achieves excellent tradeoff between video delivery quality and power consumption, and yields significant power saving compared with the conventional equal decoding power allocation （EDPA） approach in wireless video transmission.     

基于各向同性Sobel算子的H.264/AVC帧内预测模式选择算法

H.264/MPEG-4 AVC定义了9种4×4亮度块的预测模式,在参考模型JM中所采用的RDO计算复杂度较高。本文提出了一种基于各向同性Sobel算子的H.264的快速帧内编码模式选择算法,降低预测的复杂度,提高编码速度。测试表明,此算法能够使编码速度提高将近75%。     

A fast motion estimation algorithm for mobile communications

The limitation of processing power, battery life and memory capacity of portable terminals requires reducing encoding complexity in mobile communications. Motion estimation (ME) is the most computationally intensive module in a typical video codec, which determines not only the encoder’s performance but also the reconstructed video quality. In this paper, a fast ME algorithm for H.264/AVC baseline profile coding is proposed based on the analysis of motion vector field and error surface, and the statistical distributions of different type macroblocks (MBs). Simulation results showed that: in comparison with MVFAST, the proposed algorithm can decrease the computational load over 7.2% with no requirement of expanding memory capacity while maintaining the same video quality as MVFAST. Furthermore, its simplicity makes it easy to be implemented on hardware.     

Perceptual importance analysis for H.264/AVC bit allocation

The existing H.264/AVC rate control schemes rarely include the perceptual considerations. As a result, the improvements in visual quality are hardly comparable to those in peak signal-to-noise ratio （PSNR）. In this paper, we propose a perceptual importance analysis scheme to accurately abstract the spatial and temporal perceptual characteristics of video contents. Then we perform bit allocation at macroblock （MB） level by adopting a perceptual mode decision scheme, which adaptively updates the Lagrangian multiplier for mode decision according to the perceptual importance of each MB. Simulation results show that the proposed scheme can efficiently reduce bit rates without visual quality degradation.     

Fast mode decision for inter macroblocks in H. 264/AVC

In this paper, a fast mode decision algorithm for inter macroblocks in H.264/AVC is proposed, The algorithm is able to classify all modes by both gradient operator and comparison of space-time correlation. Because only part of modes is used to compare with each other the computational complexity can be reduced greatly. The simulation results show that it takes about 75 % of encoding time for other algorithm with similar visual quality.     

基于H.264/AVC帧间宏块预测模式的快速判决方法

1 Introduction The emerging H.264/AVC video coding standard hasbeen developed and standardized collaboratively byboth ITU-T VCEG and ISO/IEC MPEG organizations .As the newest international video coding standard ,H.264/AVC represents a number of advances i…     

一种基于新阈值的视频编码全零块提早判决方法

为了降低计算复杂度和编码时间,提出了一种改进的H.264/AVC全零块提早判决方法.首先,介绍了已有的全零块判决算法,分析了3种变换后将量化为零的频域系数,并在3种不同频域伸缩因子的基础上推导出对应的空域系数.然后,根据这些空域系数,并应用Schwarz不等式原理,设定了3个全零块判决阈值,并根据全零块空域系数的分布情...