Fast Image Interpolation via Random Forests

Paper Details:

Jun-Jie Huang, Wan-Chi Siu and Tian-Rui Liu, “Fast Image Interpolation via Random Forests”, paper accepted, to be published

 in IEEE Transactions on Image Processing.

 

Source C++ codes: download here

 

Paper: download here(http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=7117405 )

Correspondence:

 

Professor Wan-Chi Siu

Chair Professor and Centre Director

Centre for Signal Processing

Department of Electronic & Information Engineering

Hong Kong Polytechnic University

Hung Hom, Kowloon, Hong Kong

Email: enwcsiu@polyu.edu.hk

Home Page :  http://www.eie.polyu.edu.hk/~wcsiu/mypage.htm

Fax.:   (852) 2362 8439

Tel.:    (852) 2766 6229

 

Experimental results of 2x magnification:

1. Objective Comparisons:

PSNR (DB), SSIM AND FSIM RESULTS BY DIFFERENT INTERPOLATION METHODS

 

Testing

Images

Bicubic

NEDI

[1]

DFDF

 [2]

RLLR

 [3]

RSAI

[4]

NARM

 [5]

BSAI

 [6]

ICBI

 [7]

FIRF(1,1)

Proposed

FIRF(3,2)

Proposed

Bears

28.47

0.8796

0.9339

28.08

0.8688

0.9253

28.18

0.8687

0.9276

28.62

0.8771

0.9302

28.67

0.8818

0.9377

28.30

0.8799

0.9341

28.45

0.8786

0.9321

27.68

0.8718

0.9320

28.72

0.8815

0.9339

28.77

0.8829

0.9341

Bicycle

20.27

0.7559

0.9236

20.81

0.7761

0.9260

20.39

0.7606

0.9233

20.70

0.7877

0.9286

21.17

0.7904

0.9313

20.46

0.7837

0.9282

20.97

0.7899

0.9296

19.55

0.7380

0.9108

21.51

0.7934

0.9339

21.79

0.8022

0.9365

Boat

27.22

0.8247

0.9317

27.38

0.8293

0.9310

27.43

0.8276

0.9332

27.69

0.8416

0.9372

27.90

0.8421

0.9394

27.88

0.8477

0.9416

27.87

0.8435

0.9388

26.81

0.8164

0.9306

28.01

0.8381

0.9399

28.22

0.8425

0.9408

Butterfly

27.70

0.9241

0.9156

27.35

0.9321

0.9282

28.66

0.9397

0.9451

29.29

0.9480

0.9486

29.22

0.9472

0.9473

30.35

0.9559

0.9571

29.36

0.9488

0.9541

28.59

0.9366

0.9311

30.31

0.9536

0.9599

30.65

0.9565

0.9630

Cameraman

25.37

0.8627

0.9036

25.42

0.8626

0.9059

25.67

0.8670

0.9143

25.75

0.8714

0.9156

26.00

0.8730

0.9163

25.92

0.8760

0.9201

25.96

0.8750

0.9195

25.16

0.8611

0.9099

26.36

0.8764

0.9230

26.57

0.8795

0.9262

Coala

33.22

0.9137

0.9556

32.77

0.9030

0.9492

33.00

0.9040

0.9520

33.62

0.9106

0.9542

33.83

0.9155

0.9577

33.83

0.9138

0.9567

33.62

0.9119

0.9555

33.13

0.9117

0.9573

33.88

0.9168

0.9589

34.02

0.9177

0.9593

Elk

31.59

0.9249

0.9480

32.37

0.9291

0.9530

31.75

0.9230

0.9502

32.87

0.9345

0.9569

33.24

0.9368

0.9581

33.25

0.9356

0.9575

32.86

0.9335

0.9558

31.54

0.9230

0.9493

33.52

0.9366

0.9592

33.88

0.9387

0.9606

Fence

24.54

0.7783

0.8828

22.94

0.7584

0.8823

24.55

0.7758

0.8791

24.00

0.7755

0.8778

24.44

0.7833

0.8803

24.70

0.7924

0.9037

24.51

0.7830

0.8773

23.75

0.7612

0.9017

24.48

0.7814

0.8759

24.60

0.7855

0.8805

Flowers

28.09

0.8150

0.9198

28.25

0.8277

0.9227

28.40

0.8223

0.9241

28.89

0.8369

0.9288

28.94

0.8368

0.9297

28.78

0.8360

0.9293

28.88

0.8351

0.9291

27.96

0.8126

0.9201

28.98

0.8326

0.9301

29.17

0.8371

0.9319

Foreman

35.56

0.9488

0.9652

35.90

0.9532

0.9700

36.81

0.9541

0.9712

37.70

0.9570

0.9740

37.67

0.9577

0.9745

38.60

0.9574

0.9754

37.53

0.9564

0.9727

36.20

0.9493

0.9675

38.13

0.9571

0.9750

38.53

0.9582

0.9764

Girl

33.84

0.8534

0.9416

33.84

0.8572

0.9410

33.80

0.8523

0.9395

34.33

0.8637

0.9441

34.34

0.8633

0.9450

34.28

0.8612

0.9435

34.31

0.8630

0.9437

33.59

0.8467

0.9395

34.07

0.8563

0.9434

34.11

0.8569

0.9434

House

32.17

0.8768

0.9404

31.67

0.8743

0.9434

32.57

0.8775

0.9478

32.93

0.8836

0.9498

32.92

0.8853

0.9505

33.45

0.8834

0.9551

32.95

0.8862

0.9509

31.82

0.8661

0.9401

33.56

0.8855

0.9558

34.06

0.8882

0.9604

Leaves

26.88

0.9363

0.9262

26.23

0.9403

0.9429

27.22

0.9433

0.9478

28.43

0.9562

0.9605

28.66

0.9574

0.9590

29.77

0.9670

0.9670

28.60

0.9587

0.9613

27.37

0.9436

0.9388

29.48

0.9639

0.9655

29.96

0.9678

0.9683

Lena

33.92

0.914

0.9870

33.76

0.9134

0.9868

33.89

0.9122

0.9867

34.47

0.9190

0.9883

34.73

0.9201

0.9887

35.04

0.9238

0.9892

34.61

0.9208

0.9886

33.97

0.9111

0.9860

34.64

0.9172

0.9881

34.76

0.9180

0.9882

Parrot

26.22

0.8852

0.9290

26.10

0.8858

0.9320

26.38

0.8859

0.9334

26.83

0.8952

0.9408

26.73

0.8953

0.9398

26.90

0.8936

0.9413

26.81

0.8933

0.9400

26.31

0.8850

0.9346

27.21

0.8956

0.9425

27.36

0.8970

0.9434

Parthenon

27.08

0.8037

0.8950

26.79

0.7880

0.8910

27.18

0.8032

0.8963

27.14

0.8013

0.8960

27.22

0.8052

0.8978

27.26

0.8059

0.9021

27.34

0.8114

0.8971

26.42

0.7797

0.8930

27.44

0.8110

0.9016

27.56

0.8138

0.9030

Starfish

30.25

0.9168

0.9521

29.36

0.8985

0.9458

30.07

0.9118

0.9543

30.41

0.9140

0.9561

30.78

0.9203

0.9581

31.73

0.9302

0.9646

31.22

0.9258

0.9615

30.19

0.9167

0.9548

31.45

0.9287

0.9633

31.83

0.9324

0.9657

Stream

25.83

0.7974

0.9580

25.60

0.7812

0.9542

25.66

0.7866

0.9561

25.98

0.7948

0.9584

26.03

0.7999

0.9591

25.84

0.8008

0.9591

25.90

0.7966

0.9591

25.32

0.7871

0.9541

26.07

0.8007

0.9598

26.11

0.8018

0.9600

Average

28.79

0.8673

0.9338

28.59

0.8655

0.9350

28.98

0.8675

0.9379

29.43

0.8760

0.9414

29.58

0.8784

0.9426

29.80

0.8802

0.9459

29.54

0.8784

0.9426

28.63

0.8621

0.9362

29.88

0.8792

0.9450

30.11

0.8820

0.9468

 

 

AVERAGE COMPUTATIONAL TIME (MS) OF DIFFERENT INTERPOLATION METHODS

 

Bicubic

NEDI

[1]

DFDF

[2]

RLLR

[3]

RSAI

[4]

NARM

[5]

BSAI

[6]

ICBI

[7]

 

FIRF(1,1)

Proposed

 

FIRF(3,2)

Proposed

Computational Time (ms)

1.28

(C++)

3672.88

(Matlab)

2271.80

(Matlab)

32120.40

(Matlab)

3445.89

(C++)

277206.21

(Matlab)

42.55

(C++)

1510.56

(Matlab)

87.50

(C++)

384.78

(C++)

Approximated C++ Time (ms)

1

367

227

3212

3446

27721

43

151

88

385

 

image001

Comparison of the average computational efficiency of the proposed FIRF(n, k) method (n = 1,…, 5; k = 1, 2) and other image interpolation methods.

 

2. Subjective Comparisons

image003

image004

image005

image006

(a) Original HR Image

(b) Bicubic

(c) NEDI [1]

(d) DFDF [2]

image007

image008

image009

image052

(e) RLLR [3]

(f) RSAI [4]

(g) NARM [5]

(h) Proposed FIRF(3,2)

Fig. 1. Reconstructed HR images of Butterfly by different interpolation methods. (a) Original HR image. (b) Bicubic (PSNR= 27.70dB, SSIM= 0.9241, FSIM= 0.9156). (c) NEDI (PSNR= 27.35dB, SSIM= 0.9321, FSIM= 0.9282). (d) DFDF (PSNR= 28.66dB, SSIM= 0.9397, FSIM= 0.9451). (e) RLLR (PSNR= 29.29dB, SSIM= 0.9480, FSIM= 0.9486). (f) RSAI (PSNR= 29.22dB, SSIM= 0.9472, FSIM= 0.9473). (g) NARM (PSNR= 30.35dB, SSIM= 0.9559, FSIM= 0.9571). (h) Proposed FIRF(3,2) (PSNR= 30.65dB, SSIM= 0.9565, FSIM= 0.9630).

 

image011

image012

image013

image014

(a) Original HR Image

(b) Bicubic

(c) NEDI [1]

(d) DFDF [2]

image015

image016

image017

image053

(e) RLLR [3]

(f) RSAI [4]

(g) NARM [5]

(h) Proposed FIRF(3,2)

Fig. 2. Reconstructed HR images of Leaves by different interpolation methods. (a) Original HR image. (b) Bicubic (PSNR= 26.88dB, SSIM= 0.9363, FSIM= 0.9262). (c) NEDI (PSNR= 26.23dB, SSIM= 0.9403, FSIM= 0.9429). (d) DFDF (PSNR= 27.22dB, SSIM= 0.9433, FSIM= 0.9478). (e) RLLR (PSNR= 28.43dB, SSIM= 0.9562, FSIM= 0.9605). (f) RSAI (PSNR= 28.66dB, SSIM= 0.9574, FSIM= 0.9590). (g) NARM (PSNR= 29.77dB, SSIM= 0.9670, FSIM= 0.9670). (h) Proposed FIRF(3,2) (PSNR= 29.96dB, SSIM= 0.9678, FSIM= 0.9683).

 

image019

image020

image021

image022

(a) Original HR Image

(b) Bicubic

(c) NEDI [1]

(d) DFDF [2]

image023

image024

image025

image054

(e) RLLR [3]

(f) RSAI [4]

(g) NARM [5]

(h) Proposed FIRF(3,2)

Fig. 3. Reconstructed HR images of Cameraman by different interpolation methods. (a) Original HR image. (b) Bicubic (PSNR= 25.37dB, SSIM= 0.8627, FSIM= 0.9036). (c) NEDI (PSNR= 25.42dB, SSIM= 0.8626, FSIM= 0.9059). (d) DFDF (PSNR= 25.67dB, SSIM= 0.8670, FSIM= 0.9143). (e) RLLR (PSNR= 25.75dB, SSIM= 0.8714, FSIM= 0.9156). (f) RSAI (PSNR= 26.00dB, SSIM= 0.8730, FSIM= 0.9163). (g) NARM (PSNR= 25.92dB, SSIM= 0.8760, FSIM= 0.9201). (h) Proposed FIRF(3,2) (PSNR= 26.57dB, SSIM= 0.8795, FSIM= 0.9262).

 

image027

image028

image029

image030

(a) Original HR Image

(b) Bicubic

(c) NEDI [1]

(d) DFDF [2]

image031

image032

image033

image055

(e) RLLR [3]

(f) RSAI [4]

(g) NARM [5]

(h) Proposed FIRF(3,2)

Fig. 4. Reconstructed HR images of Fence by different interpolation methods. (a) Original HR image. (b) Bicubic (PSNR= 24.54dB, SSIM= 0.7783, FSIM= 0.8828). (c) NEDI (PSNR= 22.94dB, SSIM= 0.7584, FSIM= 0.8823). (d) DFDF (PSNR= 24.55dB, SSIM= 0.7758, FSIM= 0.8791). (e) RLLR (PSNR= 24.00dB, SSIM= 0.7755, FSIM= 0.8778). (f) RSAI (PSNR= 24.44dB, SSIM= 0.7833, FSIM= 0.8803). (g) NARM (PSNR= 24.70dB, SSIM= 0.7924, FSIM= 0.9037). (h) Proposed FIRF(3,2) (PSNR= 24.60dB, SSIM= 0.7855, FSIM= 0.8805).

 

image035

image036

image037

image038

(a) Original HR Image

(b) Bicubic

(c) NEDI [1]

(d) DFDF [2]

image039

image040

image041

image056

(e) RLLR [3]

(f) RSAI [4]

(g) NARM [5]

(h) Proposed FIRF(3,2)

Fig. 5. Reconstructed HR images of Parrot by different interpolation methods. (a) Original HR image. (b) Bicubic (PSNR= 26.22dB, SSIM= 0.8852, FSIM= 0.9290). (c) NEDI (PSNR= 26.10dB, SSIM= 0.8858, FSIM= 0.9320). (d) DFDF (PSNR= 26.38dB, SSIM= 0.8859, FSIM= 0.9334). (e) RLLR (PSNR= 26.83dB, SSIM= 0.8952, FSIM= 0.9408). (f) RSAI (PSNR= 26.73dB, SSIM= 0.8953, FSIM= 0.9398). (g) NARM (PSNR= 26.90dB, SSIM= 0.8936, FSIM= 0.9413). (h) Proposed FIRF(3,2) (PSNR= 27.36dB, SSIM= 0.8970, FSIM= 0.9434).

 

image043

image044

image045

image046

(a) Original HR Image

(b) Bicubic

(c) NEDI [1]

(d) DFDF [2]

image047

image048

image049

image057

(e) RLLR [3]

(f) RSAI [4]

(g) NARM [5]

(h) Proposed FIRF(3,2)

Fig. 6. Reconstructed HR images of Boat by different interpolation methods. (a) Original HR image. (b) Bicubic (PSNR= 27.22dB, SSIM= 0.8247, FSIM= 0.9317). (c) NEDI (PSNR= 27.38dB, SSIM= 0.8293, FSIM= 0.9310). (d) DFDF (PSNR= 27.43dB, SSIM= 0.8276, FSIM= 0.9332). (e) RLLR (PSNR= 27.69dB, SSIM= 0.8416, FSIM= 0.9372). (f) RSAI (PSNR= 27.90dB, SSIM= 0.8421, FSIM= 0.9394). (g) NARM (PSNR= 27.88dB, SSIM= 0.8477, FSIM= 0.9416). (h) Proposed FIRF(3,2) (PSNR= 28.22dB, SSIM= 0.8425, FSIM= 0.9408).

 

References:

 

[1]   X. Li and M. T. Orchard, “New edge-directed interpolation,” IEEE Trans. Image Process., vol. 10, no. 10, pp. 1521–1527, Oct. 2001.

[2]   Lei Zhang and X. Wu, “An edge guided image interpolation algorithm via directional filtering and data fusion,” IEEE Trans. on Image Processing, vol. 15, pp. 2226-2238, Aug. 2006.

[3]   X. Liu, D. Zhao, R. Xiong, S. Ma, W. Gao, and H. Sun, “Image interpolation via regularized local linear regression,” IEEE Trans. Image Process., vol. 20, no. 12, pp. 3455–3469, Dec. 2011.

[4]   K.W. Hung and W.C. Siu, “Robust Soft-decision Interpolation using weighted Least Squares,” IEEE Trans. Image Process., vol.21, no.3, pp.1061-1069, March 2012, USA.

[5]   W. Dong, L. Zhang, R. Lukac, and G. Shi, “Sparse representation based image interpolation with nonlocal autoregressive modeling,” IEEE Trans. Image Process., vol. 22, no. 4, pp. 1382–1394, Apr. 2013.

[6]   K.W. Hung and W.C. Siu, “Fast Image Interpolation using the Bilateral Filter,” IET Image Processing, vol. 6, no. 7, pp. 877–90, October 2012, UK.

[7]   A. Giachetti and N. Asuni, “Real-time artifact-free image upscaling,” IEEE Trans. Image Process., vol. 20, no. 10, pp. 2760–2768, Oct. 2011.

 

Our related works:

[1] Jun-Jie Huang and Wan-Chi Siu, "Fast Image Interpolation with Decision Tree", paper accepted to be published in the Proceedings, IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP’2015).

[2] Jun-Jie Huang and Wan-Chi Siu, "Practical Applications of Random Forests for Super-Resolution Imaging", paper accepted to be published in the Proceedings,  IEEE International Symposium on Circuits and Systems, (ISCAS’2015).

[3] Kwok-Wai Hung and Wan-Chi Siu, ‘Fast image interpolation using bilateral filter", IET Image Processing, vol. 6, no. 7, pp. 877-890, October 2012.

[4] Kwok-Wai Hung and Wan-Chi Siu, "Single-Image Super-Resolution Using Iterative Wiener Filter,” Proc. IEEE Int. Conf. Acoustics, Speech, Signal Processing (ICASSP 2012), pp. 1269-1272, 25-30, March, 2012, Kyoto, Japan

[5] He He and Wan-Chi Siu, "Single image super-resolution using Gaussian process regression," Proceedings of 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.449-456, 20-25 June 2011.

[6] Kwok-Wai Hung and Wan-Chi Siu, "Real time interpolation using Bilateral filter for Image zoom or Video up-scaling/transcoding," Proc. IEEE Int. Conf. Consumer Electronics, (ICCE 2012), pp. 67-68, 13-16 January, 2012, Las Vegas, USA.

[7] Kwok-Wai Hung and Wan-Chi Siu, "Fast Video Interpolation/Upsampling Using Linear motion model," Proc. IEEE Int. Conf. Image Processing (ICIP 2011), pp. 1341-1344, 11-14 September, 2011, Brussels, Belgium.

[8] Wing-Shan Tam, Chi-Wah Kok and Wan-Chi Siu, "A Modified Edge Directed Interpolation for Images," pp.13011_1-20, Journal of Electronic Imaging, Vol.19(1), 013011, Jan-March 2010.

[9] Kwok-Wai Hung and Wan-Chi Siu, "Improved Image Interpolation using Bilateral Filter for Weighted Least Square Estimation," Proceedings, pp.3297-3300, IEEE International Conference on Image Processing, (ICIP'2010), 26-29 September, 2010, Hong Kong

[10] Chi-Shing Wong and Wan-Chi Siu, "Further Improved Edge-directed Interpolation and Fast EDI for SDTV to HDTV Conversion," Proceedings, pp.309-313, 18th European Signal Processing Conference (EUSIPCOˇ¦2010), 23-27 August, 2010, Aalborg Denmark.

[11] Chi-Shing Wong and Wan-Chi Siu, "Adaptive Directional Window Selection For Edge-Directed Interpolation," Proceedings, MMC1, Paper No.4, pp.1-6, ICCCN, 2010 Workshop on Multimedia Computing and Communications, 2-5 August, 2010, Zurich, Switzerland.

[12] Kwok-Wai Hung and Wan-Chi Siu, "New Motion Compensation Model via Frequency Classification for Fast Video Super-Resolution," Proceedings, pp.1193-6, (ICIPˇ¦2009) International Conference on Image Processing, 7-11 November, 2009, Cairo Egypt.