Sol 4

A weekend is a weekend, even if its on Mars.
I’ve been experimenting with the real-time applications of image upscaling and super-resolution, including image sharpening and motion vector upscaling and reconstruction. Sounds fancy right?

I’ve been gaming. Yea, I’ve been gaming 😅.

DALL.E 3 generated image

Nvidia DLSS (Deep Learning Super Sampling) and AMD FSR 2.0 (FidelityFX Super-Resolution) are at the cutting edge of image upscaling for high-fidelity gaming experiences. Since screen sizes have been increasing ad infinitum, these techniques help maintain great gaming performance and visuals without breaking the bank.

Newer generation Nvidia Gaming GPUs, such as the RTX series, come with tensor cores onboard. Tensor cores are cores which only perform 4x4 matrix multiplications. Each tensor core can perform 1 matrix multiply-accumulate operation per GPU clock cycle. This operation performs matrix multiplications in FP16 and then accumulates the multiplication products in FP32 precision. This leads to a significant boost in computing speed, without significant loss of accuracy.

Using this improvement in hardware, along with deep learning techniques for super resolution, DLSS upsamples and performs anti-aliasing to output frames at a higher resolution. These networks can be trained using Perceptual or Adversarial losses to enable the network to translate frames from one domain to the other, while maintaining the general style and aesthetic of the original frame.

The working of AMD FSR is still a bit of a mystery to me at the moment.

Also, I wrote a blog on setting up an Apache Spark cluster with Jupyterlab and Llama-3.1 8B services.