OpenMediaLib User and Development Guide

• OpenMediaLib User Development Guide
  • Introduction
    • Terminology
  • High Level Use
  • Reverse Polish Notation
  • Applying RPN to Video/Audio
  • Clip Modifications
  • Compositing
  • Playlists
  • Stack Manipulations
  • Advanced Stack Usage
  • Aspect Ratio Considerations
  • The Encoding Filter Graph
  • Compositing Revisited
  • Really, Really Advanced Stack Usage
  • General Audio Issues
    • The Sample Calculator
    • Audio Resampling
    • Audio Mixing
  • Python
    • Python Callbacks and Filter Implementations
  • Interpolation
  • Threading

• Putting it All Together

• Appendices
  • Appendix A Input Operations
  • Appendix B Filter Operations
  • Appendix C Stores

Applying RPN to Video/Audio

Arithmetic operations and filter graphs are very similar. To illustrate this, we will start with a fairly simple graph which simply applies a filter to a video:

filter:something
  <input>

The first line here is the node which the application reads from – its first (and in this case, only) slot is connected to the <input> node.

It should be clear that this is similar to our sine case in the previous section and thus, our RPN specification is simply:

<input> filter:something

This isn't particularly interesting in itself, but if we take a binary operator like a 'composite' (which is analogous with a +, * or / operator), we end up with a more interesting graph:

filter:composite
  <background>
  <foreground>

which maps (rather unsurprisingly) to:

<background> <foreground> filter:composite

Or in combination with a filter on the foreground:

<composite>
  <background>
  <filter>
    <foreground>

we get:

<background> <foreground> <filter> filter:composite

Note that the composite operation is not commutative:

<input1> <input2> filter:composite != <input2> <input1> filter:composite

These examples should show that RPN does indeed provide an unambiguous grammar for filter graph construction, but it would be understandable if there is still a 'so what?' question hanging – so far, all we have focused on is ensuring that RPN is non-restrictive, but we haven't touched on the advantages (other than the unambiguous and analogous aspects).

The advantages are subtle – for example, in order to construct our graph, we simply need push and pop methods:

stack.push( background_info )
stack.push( foreground_info )
stack.push( filter_info )
stack.push( composite_info )
return stack.pop( )

NB: the pushes on to the stack can be strings which describe the operator required (ie: a filename or the name of filter [preceded with 'filter:']) or a pre-constructed OML operator or a partially constructed filter graph or a modifier for the current operator [strings which take the form of name=value] or a callback specification.

This is comparable to a more specifically crafted bit of code:

composite = create( composite_info )
foreground = create( foreground_info )
background = create( background_info )
filter = create( filter_info )
composite.connect( background, 0 )
composite.connect( filter, 1 )
filter.connect( foreground, 0 )
return composite

Obviously, there are more lines of code to carry out the same task, and that complexity will increase as the number of filters and inputs are increased. To be fair, you can reduce the lines of code, but you would not be able to reduce the number of function calls or variable assignment (at least, not by much, and not without making the code even more brittle and dense).

Whilst the loc count is debatable, the larger issue lies in the fact that the code itself is brittle – changing a connection in a non-trivial graph is a non-trivial operation (as an exercise, try extending both to include a second filter between the foreground and the composite and compare the lines of code added and modified – it should be immediately obvious which one 'wins').

The following sections detail some of the filters which are currently available.