OpenMediaLib User and Development Guide

• OpenMediaLib User Development Guide
  • Introduction
    • Terminology
  • High Level Use
  • Reverse Polish Notation
  • Applying RPN to Video/Audio
  • Clip Modifications
  • Compositing
  • Playlists
  • Stack Manipulations
  • Advanced Stack Usage
  • Aspect Ratio Considerations
  • The Encoding Filter Graph
  • Compositing Revisited
  • Really, Really Advanced Stack Usage
  • General Audio Issues
    • The Sample Calculator
    • Audio Resampling
    • Audio Mixing
  • Python
    • Python Callbacks and Filter Implementations
  • Interpolation
  • Threading

• Putting it All Together

• Appendices
  • Appendix A Input Operations
  • Appendix B Filter Operations
  • Appendix C Stores

Terminology

image – an object which conforms to the OIL image_type interface. Typically, images provided by OML are in the native colour space of the source material (codec or usage dependent).

colour space – in OIL parlance, a colour space stipulates the make up of the image, both in terms of the sample space used (RGB, YUV/YCbCr etc.) and the packing of the samples (hence, a colour space is RGB, BGR, RGBA, YUV420P etc.).

audio – an object which conforms to the OML audio_type interface. At the point of writing, all audio must be coerced to the 16 bit signed integers at the input phase. It is described by the frequency (being the number of samples per channel per second), the number of channels and the number of samples in each channel.

frame – an object which provides an image and/or audio. When both are provided, the number of audio samples in successive frame should correlate to a pattern dictated by the frame rate (see General Audio Issues below).

frames per second (fps) or frame rate – all frames must report a valid fps expressed as integer numerator:denominator pairs (for example, PAL 25 fps should be reported as 25:1 and NTSC should report 30000:1001) and provide a frame accurate seek interface where applicable.

variable frame rates – some video formats provide support for variable frame rates – in those cases, images are not delivered at fixed durations. It is incumbent on the input implementation to map these to a fixed frame rate (typically by employing a duplicate and drop mechanism for images).

sample – depends on the context – it can refer to either discrete audio or image measurements, for example, the value of a colour component at a particular offset in an image.

sample aspect ratio (sar) – typically, images coming from video sources do not consist of 'square' pixels – often, they are rectangular, which means that the display resolution has a different width to the resolution provided. As with frame rates, the sample aspect ratio is expressed as a numerator:denominator pair.

aspect ratio – the aspect ratio is typically determined by dividing the width by the height in the square pixel scenario (1:1 being the common sar for still images).

input – an object which conforms to the OML input interface – its primary purpose is to provide frames.

pull/push – the general model for the input and store interaction is defined as pull and push respectively – applications pull frames from inputs, and push to the store.

RPN – Reverse Polish Notation – this is described in more detail below.