Is there a way to speed this realesrgan upscale up?

[codecowboy]

Using the example command from the --help gives a framerate of approx 0.4 fps. A 4k upscale on my M1Pro Mac for 60m of tv footage in Topaz video AI takes approx 12 hours and video2x is estimating ~40 hours. I had assumed upscaling with an Nvidia GPU would be orders of magnitude faster.

Command used:

video2x -d 0 -i input.mkv -o output.mkv -s 4 -p realesrgan --realesrgan-model realesrgan-plus -c libx264rgb -e crf=17 -e preset=veryfast -e tune=film

Here's the output I am getting:

 video2x -d 0 -i input.mkv -o output.mkv -s 4       -p realesrgan --realesrgan-model realesrgan-plus       -c libx264rgb -e crf=17 -e preset=veryfast -e tune=film
[2024-12-13 11:40:44.705] [info] Video2X version 6.2.0
[2024-12-13 11:40:44.705] [info] Processing file: input.mkv
[2024-12-13 11:40:45.623] [info] Using Vulkan device: NVIDIA GeForce RTX 3090 (0x2204)
[2024-12-13 11:40:45.624] [info] Press [space] to pause/resume, [q] to abort.
[libx264 @ 0x7f3a081b2b00] using SAR=1/1
[libx264 @ 0x7f3a081b2b00] DPB size (16 frames, 921600 mbs) > level limit (12 frames, 696320 mbs)
[libx264 @ 0x7f3a081b2b00] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0x7f3a081b2b00] profile High, level 6.2, 4:2:0, 8-bit
[libx264 @ 0x7f3a081b2b00] 264 - core 163 r3060 5db6aa6 - H.264/MPEG-4 AVC codec - Copyleft 2003-2021 - http://www.videolan.org/x264.html - options: cabac=1 ref=16 deblock=1:-1:-1 analyse=0x3:0x133 me=umh subme=10 psy=1 psy_rd=1.00:0.15 mixed_ref=1 me_range=24 chroma_me=1 trellis=2 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-3 threads=12 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=8 b_pyramid=2 b_adapt=2 b_bias=0 direct=3 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=60 rc=crf mbtree=1 crf=17.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
[0 NVIDIA GeForce RTX 3090]  queueC=2[8]  queueG=0[16]  queueT=1[2]
[0 NVIDIA GeForce RTX 3090]  bugsbn1=0  bugbilz=0  bugcopc=0  bugihfa=0
[0 NVIDIA GeForce RTX 3090]  fp16-p/s/a=1/1/1  int8-p/s/a=1/1/1
[0 NVIDIA GeForce RTX 3090]  subgroup=32  basic=1  vote=1  ballot=1  shuffle=1
[1 llvmpipe (LLVM 15.0.7, 256 bits)]  queueC=0[1]  queueG=0[1]  queueT=0[1]
[1 llvmpipe (LLVM 15.0.7, 256 bits)]  bugsbn1=0  bugbilz=0  bugcopc=0  bugihfa=0
[1 llvmpipe (LLVM 15.0.7, 256 bits)]  fp16-p/s/a=1/1/1  int8-p/s/a=1/1/1
[1 llvmpipe (LLVM 15.0.7, 256 bits)]  subgroup=8  basic=1  vote=1  ballot=1  shuffle=1
[0 NVIDIA GeForce RTX 3090]  queueC=2[8]  queueG=0[16]  queueT=1[2]
[0 NVIDIA GeForce RTX 3090]  bugsbn1=0  bugbilz=0  bugcopc=0  bugihfa=0
[0 NVIDIA GeForce RTX 3090]  fp16-p/s/u/a=1/1/1/1  int8-p/s/u/a=1/1/1/1
[0 NVIDIA GeForce RTX 3090]  subgroup=32  basic/vote/ballot/shuffle=1/1/1/1
[0 NVIDIA GeForce RTX 3090]  fp16-8x8x16/16x8x8/16x8x16/16x16x16=0/1/1/1
[1 llvmpipe (LLVM 15.0.7, 256 bits)]  queueC=0[1]  queueG=0[1]  queueT=0[1]
[1 llvmpipe (LLVM 15.0.7, 256 bits)]  bugsbn1=0  bugbilz=0  bugcopc=0  bugihfa=0
[1 llvmpipe (LLVM 15.0.7, 256 bits)]  fp16-p/s/u/a=1/1/1/1  int8-p/s/u/a=1/1/1/1
[1 llvmpipe (LLVM 15.0.7, 256 bits)]  subgroup=8  basic/vote/ballot/shuffle=1/1/1/1
[1 llvmpipe (LLVM 15.0.7, 256 bits)]  fp16-8x8x16/16x8x8/16x8x16/16x16x16=0/0/0/0
[2024-12-13 11:40:48.537] [warning] Estimating the total number of frames using duration * fps
frame=38/72609 (0.05%); fps=0.46; elapsed=00:01:23; remaining=44:01:50

I'm using Nvidia's beta 1.4 Vulkan driver on Ubuntu 22.04 and --list-devices shows the GPU as device 0. I got what I could from the --help info and tried a preset of 'veryfast' but the framerate is still very low. Video2x version is the latest (6.2). Are there some other command line args I can try which will speed this up? Thanks!

[k4yt3x]

That example is not meant to be the optimal setting. It just shows you how to apply custom options. -c libx264rgb and -e tune=film are unnecessary.

realesrgan-plus is for live action footage and it's inherently slow. Check your GPU usage. It is likely already working at 100%. If so, I don't think it can be made much faster. preset is for the encoder, and it will affect your output file size, not so much the quality. You only need to adjust it towards the faster side if your CPU is the bottleneck, which isn't the case here.