Standard version: 1.5
(september 23, 2016): This is to be merged with Multi-View
A recent trend is to embed multiple image types in the same image stream, for example depth information as well as colour data. The current version 1.4 OFX Image Effect API only presents a single colour image buffer from any clip, either input or output. This prevents hosts with rich image streams from passing such data to plug-ins that could take advantage of it.
The Foundry has implemented an unpublished extension to the OFX API in its Nuke compositing application that allows support for deep image buffers. They are proposing cleaned up versions of these extensions to the OFX 1.5 Committee for adoption in the Image Effect API.
To do this we need to add a new concept to OFX, a plane, which plane represents a single typed image data buffer, for example colour, depth, motion vectors and so on.
The basic idea behind this extensions is that it is orthogonal to time, so when an effect fetches image data from the host it would specify the frame time and the image plane to fetch, eg: the source clip's the colour plane of frame 10.
The extensions will need supporting machinery, so that plug-ins advertise what planes/views they process and need to render an output and so on.
It is proposed that the OFX API be updated to include deep image buffers for version 1.5.0.
A plane is a single image buffer of a specific type. The API defines the types of planes available via #defining a set of string literals, for example…
The data types of the corresponding planes are sometimes strongly defined, for example depth would always be a one component floating point image, colour images could still be of variant data type and 3 or 4 components.
The full set of image planes has yet to be determined, but the initial types should include….
Note that some planes come in pairs, for example a forwards motion vector plane cannot exists without a backwards motion vector plane.
Not all hosts will support all image plane types. To manage this the multidimensional string property kOfxImageEffectPropPlanesAvailable is set on host descriptors.
This property is the set of planes that a host may present on a clip. All hosts must advertise kOfxImagePlaneColour, the other planes are optional.
Plugins need to indicate to the host what planes it will process on output, it to, uses the kOfxImageEffectPropPlanesAvailable property on it's descriptor.
This is the set of planes a plugin instance may be able to render. The actual planes rendered by an instance are indicated by the new Get Clip Planes action (see below).
If a host does not support the planes a plugin renders, a host should ignore that plugin.
A plugin may break the coherence between planes available on its input clips. This is typically because it is doing a spatial or temporal transformation on the subset of planes being processed. In such a case, it should indicate to the host that the non-processed planes should not be passed through.
To do this a new plugin descriptor property is required. kOfxImageEffectPropPassThroughPlanes. This is a one dimensional integer property, if set to zero it indicates that non processed planes should not be passed through to output by the host, if set to not zero, then the host should pass through un-rendered planes to the node's output.
A host advertises the planes available to be fetched from a specific clip instance by the property kOfxImageEffectPropPlanesAvailable on the clip's corresponding property handle.
When fetching an image from a clip instance, a plugin will need to specify the plane as well as the frame. To do this the signature of the clipGetImagePlane function in the OfxImageEffectSuite needs to be modified in the next version of that suite so that it looks like…
OfxStatus ( * clipGetImagePlane ) ( OfxImageClipHandle clip , OfxTime time , const char * plane , OfxRectD * region , OfxPropertySetHandle * imageHandle ) ;
The new plane argument to the function being one of the predefined string literals that define a plane.
A plugin's descriptor indicates the planes it may render on output via the kOfxImageEffectPropPlanesAvailable property. Depending on instance parameter values and available input planes, an instance needs to indicate what planes it will actually render prior to rendering.
A plug-in instance will also need to tell the host what planes it will fetch from each input clip to produce it's given output.
Finally, if the plane has enabled passthrough of non rendered planes, it needs to indicate indicate what frame time at what clip the host should get those planes from.
To do this a new action is required, kOfxImageEffectActionGetClipPlanes. This action has a single 'inarg', being the time that a render will subsequently be called for.
It has the following 'outargs'….
When a host calls the render action on a plug-in instance, it needs to indicate which set of planes it wants the plugin to fill in. This will be a subset of the planes that the plugin has specified in the Get Clip Planes action. To do this, the render actions 'inArgs' needs an extra property, being the kOfxImageEffectPropPlanesAvailable property specified above.