Realtime Bilateral Filter on GPU

R96944043 周伯相

http://www.cmlab.csie.ntu.edu.tw/~zho/

Final Project

For the final project, there are three options: research, system and film.

Research: Define a new problem or develop a new approach to solve an existing problem. You can look up recent SIGGRAPH/CVPR/ICCV papers for inspiration. For a new problem, you don't need to solve it completely, but you have to make reasonable progress.

System: Implement an easy-to-use system or library for some existing algorithms. Good candidates are the algorithms presented in recent SIGGRAPH/CVPR/ICCV papers.

Film: Make a short "effects" film. You can use any available tools to make it. At times, you might have to write programs to achieve your goal.

Introduction

I would like to implement a system to enhance image quality by the filters like realtime edge-aware image processing. I present a video player for the realtime bilateral filter on GPU.

Program require: GPU with Shader Model 3.0
Supprt Video format: AVI, Video codec :XVID , Audio codec: PCM, AC3, MP3

You can drag a avi file into the windows to play.

Key Control
G: Gaussian Filtering Mode
B: Bilateral Filtering Mode
O: Original Mode
C: Compare Mode
M: Change Current Mode
A: Video Abstraction
1: 1 pass mode
2: 2 pass mode
Left: Increase Approximation Level
Right: Decrease Approximation Level

Related work

Gaussian filter

Gaussian filter as weighted average

I:Input is a step function and noise
f:Spatial Gaussian
J:Output is blurred

Bilateral filter

The basic idea underlying bilateral filtering is to do in the range of an image what traditional filters do in its domain. Two pixels can be close to one another, that is, occupy nearby spatial location, or they can be similar to one another, that is, have nearby values, possibly in a perceptually meaningful fashion.

Bilateral filter is slow !
Compare to Gaussian filtering, it is much slower because the kernel is not fixed and it may take few seconds on traditional bilateral filtering algorithm on cpu.

Implementation

I use 2 pass of bilateral filtering.

• 1st pass

Result of RenderTarget from 1st pass

fullsize of 1st pass rendertarget	1/2 size of 1st pass rendertarget	1/4 size of 1st pass rendertarget

We can speed up by reducing 1st pass rendertarget size.

• 2nd pass

Use 1st pass rendertarget as a texture to input the result of g(I(b) - I(x)) and computing 7x7 bilateral filtering on GPU.

Result

AVI,Size 640x480, XVID, AC3

Gaussian Filtering 7x7

Bilateral Filtering 7x7

Compare mode
Left side:original video, Right side: Video with Bilateral filter

Full HD Size click to enlarge

Video Abstraction

Performance

• 1 pass mode on shader model 3.0

Hardware	video size	Frame per second
CPU Q6600, GPU nVidia GF7900GT	640x480	> 30
CPU Q6600, GPU nVidia GF7900GT	1280x720	> 30
CPU DuoT7500 GPU nVidia GF8400mG (Notebook)	640x480	7~9

 

 

 

• 2 pass mode on shader model 2.0

Hardware	video size	Approximation	Frame per second
CPU Q6600, GPU nVidia GF7900GT	640x480	No	> 30
CPU Q6600, GPU nVidia GF7900GT	1280x720	No	> 30
CPU DuoT7500 GPU nVidia GF8400mG (Notebook)	640x480	No	> 30
CPU Q6600, GPU nVidia GF7900GT	1080p	No	11~12
CPU Q6600, GPU nVidia GF7900GT	1080p	1:1/4	15~16
CPU DuoT7500 GPU nVidia GF8400mG (Notebook)	1080p	No	5~6
CPU DuoT7500 GPU nVidia GF8400mG (Notebook)	1080p	1:1/4	9~10

 

 

 

 

 

Download program VFX_R96944043_Final.rar

References

• Tomasi, C., Manduchi, R.: Bilateral filtering for gray and color images. In: Proc. of International Conference on Computer Vision, IEEE
• Jiawen Chen Sylvain Paris Fr’edo Durand, Real-time Edge-Aware Image Processing with the Bilateral Grid, ACM SIGGRAPH
• Sylvain Paris and Fr’edo Durand, Fast Approximation of the Bilateral Filter using a Signal Processing Approach.
• Fr’edo Durand and Julie Dorsey, Fast Bilateral Filtering for the Display of High-Dynamic-Range Images
• Holger Winnem‥oller Sven C. Olsen Bruce Gooch, Real-Time Video Abstraction