GPU RGBA -> NV12 / I420 / YV12, one texture holding the framestore. More...

Inheritance diagram for score::gfx::Yuv420PackedEncoder:

Detailed Description

GPU RGBA -> NV12 / I420 / YV12, one texture holding the framestore.

The plane encoders give two or three plane textures; a consumer that hands a device one pointer then has to concatenate them, and pays a GPU->CPU round trip per plane to get there. This produces the framestore directly, so one readback is the whole thing. At 2160p that is 2.4 ms a frame against 10.3.

Plane geometry, which is what chroma_byte() has to navigate:

NV12: Y is w bytes over h rows; interleaved CbCr is also w bytes (w/2 sites x 2 components) over h/2 rows. Uniform rows. I420: Y is w bytes over h rows; Cb then Cr are w/2 bytes over h/2 rows. YV12: I420 with Cr before Cb.

The target is RGBA8 of (w/4) x (3h/2), four framestore bytes per texel, and R8 of w x (3h/2) when the width is not a multiple of four – 722 and 1922 are even, so 4:2:0 can express them, but an RGBA8 target cannot be w/4 texels wide. One byte per texel runs a fragment, a fetch and a colour conversion per byte, which measured slower than the plane encoders it replaces, so it is the fallback rather than the rule.

Chroma is a 2x2 box, centre-sited. Both plausible improvements were tried and measured worse against a real NDI receiver; they remain in chroma_at() behind SCORE_GFX_CHROMA_SITING so the measurement can be repeated.

Siting. MPEG-2/H.264 put the sample on the even luma column. Sending colour bars through the SDK and locating each chroma edge on the way back: centre displaced it by -0.001 luma columns, left by +0.165. The receiver assumes centre.

Pre-filtering. [1 3 3 1]/8 each way band-limits before decimating. On red/blue stripes round-tripped through the SDK, mean |RGB error| by stripe period in luma columns:

period       4      6      8     12     16     32
box       5.48  33.06   8.46   5.23   3.15   2.67
[1 3 3 1] 49.63  47.04  27.02  15.74  12.95   8.02

Worse everywhere, including at Nyquist: the receiver upsamples with something box-like and does not reconstruct a band-limited signal, so the blur costs more than the aliasing it removes.

Requires even width and height.

Public Types
enum class	Siting { Centre = 0 , Left = 1 , CentreWide = 2 }
	Where a chroma sample sits horizontally relative to its luma pair. More...

enum class	Layout { NV12 , I420 , YV12 }

Public Member Functions
	Yuv420PackedEncoder (Layout layout, Siting siting=Siting::Centre) noexcept

int	framestoreRows () const noexcept
	Rows in the whole framestore: the picture, plus half of it again.

void	init (QRhi &rhi, const RenderState &state, QRhiTexture *inputRGBA, int width, int height, const QString &colorConversion) override

void	exec (QRhi &rhi, QRhiCommandBuffer &cb) override

int	planeCount () const override
	Number of readback planes (1 for UYVY, 2 for NV12, 3 for I420).

const QRhiReadbackResult &	readback (int) const override
	Get the readback result for a given plane. Valid after endOffscreenFrame.

QRhiTexture *	outputTexture () const noexcept override

void	setReadbackEnabled (bool e) noexcept override

void	release () override
	Release all GPU resources.

Static Public Member Functions
static std::unique_ptr< Yuv420PackedEncoder >	nv12 ()

static std::unique_ptr< Yuv420PackedEncoder >	i420 ()

static std::unique_ptr< Yuv420PackedEncoder >	yv12 ()

Public Attributes
Layout	m_layout {Layout::NV12}

Siting	m_siting {Siting::Centre}

QRhiTexture *	m_outTexture {}

QRhiTextureRenderTarget *	m_renderTarget {}

QRhiRenderPassDescriptor *	m_rpDesc {}

QRhiSampler *	m_sampler {}

QRhiShaderResourceBindings *	m_srb {}

QRhiGraphicsPipeline *	m_pipeline {}

QRhiReadbackResult	m_readback {}

int	m_width {}

int	m_height {}

int	m_bytesPerTexel {1}

bool	m_readbackEnabled {true}

Static Public Attributes
static constexpr const char *	chroma_nv12

static constexpr const char *	chroma_planar

static constexpr const char *	frag

Static Public Attributes inherited from score::gfx::GPUVideoEncoder
static constexpr const char *	vertex_shader

static constexpr const char *	rgb_to_yuv_glsl

static constexpr const char *	y_flip_glsl
	Fragment shader Y-flip: on GL, flip v_texcoord.y. On Metal/HLSL, no flip.

Member Enumeration Documentation

◆ Layout

enum class score::gfx::Yuv420PackedEncoder::Layout

strong

Enumerator
NV12	Y, then Cb/Cr interleaved.
I420	Y, then Cb, then Cr.
YV12	Y, then Cr, then Cb.

◆ Siting

enum class score::gfx::Yuv420PackedEncoder::Siting

strong

Where a chroma sample sits horizontally relative to its luma pair.

Enumerator
Centre	between the two columns: a 2x2 box, what swscale emits
Left	on the even column, [1 2 1]/4: what MPEG-2/H.264 specify
CentreWide	centre, but [1 3 3 1]/8 each way instead of a box

Member Function Documentation

◆ exec()

void score::gfx::Yuv420PackedEncoder::exec	(	QRhi &	rhi,
		QRhiCommandBuffer &	cb
	)

inlineoverridevirtual

Execute the conversion pass. Call inside beginOffscreenFrame/endOffscreenFrame. Renders the conversion shader and schedules GPU->CPU readback(s).

Implements score::gfx::GPUVideoEncoder.

◆ init()

void score::gfx::Yuv420PackedEncoder::init	(	QRhi &	rhi,
		const RenderState &	state,
		QRhiTexture *	inputRGBA,
		int	width,
		int	height,
		const QString &	colorConversion
	)

inlineoverridevirtual

Create render targets, textures, samplers, and the graphics pipeline.

Parameters

rhi	The QRhi instance.
state	The render state (for shader compilation).
inputRGBA	The scene RGBA texture to read from.
width	Source width in pixels.
height	Source height in pixels.
colorConversion	GLSL code from colorMatrixOut() defining convert_from_rgb(). If empty, defaults to BT.709 full range.

Implements score::gfx::GPUVideoEncoder.

◆ outputTexture()

QRhiTexture * score::gfx::Yuv420PackedEncoder::outputTexture ( ) const

inlineoverridevirtualnoexcept

The single output texture (RGBA8, format-dependent dimensions) for single-plane encoders, so vendor-neutral code (CpuStagedVideoOutput's DVP download) can copy it straight to sysmem instead of using readback(). Default nullptr (multi-plane encoders) => callers must use readback().

Reimplemented from score::gfx::GPUVideoEncoder.

◆ planeCount()

int score::gfx::Yuv420PackedEncoder::planeCount ( ) const

inlineoverridevirtual

Number of readback planes (1 for UYVY, 2 for NV12, 3 for I420).

Implements score::gfx::GPUVideoEncoder.

◆ readback()

const QRhiReadbackResult & score::gfx::Yuv420PackedEncoder::readback ( int plane ) const

inlineoverridevirtual

Get the readback result for a given plane. Valid after endOffscreenFrame.

Implements score::gfx::GPUVideoEncoder.

◆ release()

void score::gfx::Yuv420PackedEncoder::release ( )

inlineoverridevirtual

Release all GPU resources.

Implements score::gfx::GPUVideoEncoder.

◆ setReadbackEnabled()

void score::gfx::Yuv420PackedEncoder::setReadbackEnabled ( bool )

inlineoverridevirtualnoexcept

Enable/disable the GPU->CPU readback scheduled in exec(). Default enabled (readback runs). Disable when a GPU-direct download reads outputTexture() directly, so exec() doesn't also pay the readback (no double transfer).

Reimplemented from score::gfx::GPUVideoEncoder.

Member Data Documentation

◆ chroma_nv12

constexpr const char* score::gfx::Yuv420PackedEncoder::chroma_nv12

staticconstexpr

Initial value:

= R"_(
    float chroma_byte(int b, int cr, ivec2 sz) {
      vec2 uv = chroma_at(b >> 1, cr, sz);
      return ((b & 1) == 0) ? uv.x : uv.y;
    }
  )_"

One byte of the chroma region, for a semi-planar layout: site b/2 of row cr, Cb on even bytes and Cr on odd ones.

◆ chroma_planar

constexpr const char* score::gfx::Yuv420PackedEncoder::chroma_planar

staticconstexpr

Initial value:

= R"_(
    float chroma_byte(int b, int cr, ivec2 sz) {
      int cw = sz.x >> 1;      // bytes in one chroma plane row
      int ch = sz.y >> 1;      // rows in one chroma plane
      // A target row is w bytes and a chroma plane row is w/2, so a target row
      // holds EXACTLY two chroma rows. That makes the mapping shifts and
      // compares -- no integer division, which is worth avoiding here because
      // it would run once per chroma byte of every frame.
      int sel = (b < cw) ? 0 : 1;   // "half" is a GLSL reserved word
      int col = b - sel * cw;
      int gr = (cr << 1) + sel;   // chroma row counted across both planes
      bool first = gr < ch;
      vec2 uv = chroma_at(col, first ? gr : gr - ch, sz);
      return first ? uv.%1 : uv.%2;
    }
  )_"

And for the fully planar ones. %1 / %2 are the components of the first and second plane: (x, y) is Cb-then-Cr (I420), (y, x) is Cr-then-Cb (YV12).

The documentation for this struct was generated from the following file:

YUV420Packed.hpp

Detailed Description

Public Types

Public Member Functions

Static Public Member Functions

Public Attributes

Static Public Attributes

Member Enumeration Documentation

◆ Layout

◆ Siting

Member Function Documentation

◆ exec()

◆ init()

◆ outputTexture()

◆ planeCount()

◆ readback()

◆ release()

◆ setReadbackEnabled()

Member Data Documentation

◆ chroma_nv12

◆ chroma_planar