"

- To me it is not interesting to let the getClipPreferences live any longer because if we use it then for time invariant metadatas, its going to have a misleading name and documentation. 

If a host is going to implement the metadata suite anyway, I don't see why they couldn't call the getTimeInvariant action instead of getClipPreferences for plug-ins that support it (it's just a if/else case)."

What you are saying I think is: If an host says it supports MetaData action and plugin says it wants to use it. Then instead of kOfxImageEffectActionGetClipPreferences action the MetaData action would be the one called by host.  For that I think you are correct that is possibly not too destructive.  

And then what I am saying I think is propHandle for OfxImageEffectSuiteV1  for clipGetPropertySet and clipGetHandle is the worry.  There is also  kOfxImageEffectPropClipPreferencesSlaveParam. Net fair amount of code not just an if/else for plugins. I would like we get the same clip property set from the same interface, but perhaps a transition solution would be to simply add the functional equivalent of *extra prop to bridge this? Maybe something like you get the property set from either clipGetPropertySet or clipGetHandle and you pass it back to MetaData suite to get the extra goodness. ?

Then once such mechanism is in place, something like  OfxImageEffectSuiteV2 is a lot less a worry, and that could receive an handle that is directly in that form ("PropV2", assuming the Clip Preferences props are declared to be (by clip accessed) metadata). 

Pierre, 

- To me it is not interesting to let the getClipPreferences live any longer because if we use it then for time invariant metadatas, its going to have a misleading name and documentation. 

If a host is going to implement the metadata suite anyway, I don't see why they couldn't call the getTimeInvariant action instead of getClipPreferences for plug-ins that support it (it's just a if/else case). 

I agree the name all in all is just the name, but the important part is that each party very well understand that there are 2 types of metadata: time variant and time invariant and they serve a very different purpose. It is also important to mention the instanceChanged call recursively downstream for metadata slave (previously clip preferences slave) parameters because I think it is omitted from the spec and that would lean towards a more unified behavior for all hosts. 

We could introduce a FPS metadata in seconds aswell and the change of action should enable hosts to implement that correctly.

Regarding recursive PropertySet, to me it could wait a PropertySuiteV3 because it's just a very different implementation for the host and requires major changes, whereas the proposal that I made below does not require the host to implement much.

Who is recursive ProperySets going to benifit ? If it's for a user interface point of view, we could specify groups in metadata names by introducing a separator "/", e.g: "ASC/CDL/Power/R" 

This is just a matter of formating property names, and it doesn't require extra work for any host implementations.

Lastly, to me it is very important that we introduce binary properties as mentionned in my proposal because it allows plug-ins to communicate arbitrary data types, if not more complicated data structures such as Audio or anything else other than images:

A plug-in could create the metadata "Audio" and another plug-in could receive it in it's render action. 

This is a generic way to pass arbitrary data. This is less optimized than the whole getRegionsOfInterest/getFramesNeeded pipeline because the plug-in will receive the "full" metadata (it can be a very big chunk of data) whereas for images it can just ask for a chunk of it, but it at least enables to pass arbitrary data.

Just another comment Alexandre, to simply set that Clip Preferences is Clip Property static (time invariant) meta-data is very good way to close the loop... On a longer term horizon, I can see how once v1.5 is in place, that then imageEffectV2 becomes more evolutionary safe if v1.5 Clip Preferences become the default and defined clip meta-data properties and easier for an host to continue supporting imageEffectV1 compiled plugins...

Question:  Would creating a timeLineSuiteV2 be a way to create an explicit behavior expectation for OFX Frame time to and from Clock Time (seconds)? 

In theory one should be able to use FPS to convert OFX Frame time into seconds but in practice support for FPS is all over the place. 

Thanks Alexandre,

A few more notes to be clear we are on same page 

1) OK not writable in render thread. 

2) I am thinking we need a registration process for effects at Param Creation stage OR after for effects supporting meta-data creation, no?  Actually to be better maybe something else later. By analogy, one issue with Nuke implementation of the Saturation effect example in Programming guide is it displays all the image channels supported by Nuke - as opposed to the one that actually exist from the input clip POV. For example if someone had a 3D tracker, and an option to create a Camera as Meta-Data. As creating the camera might involve a few UI steps etc - we probably want to make meta-data shown as available when there is actually some metadata as opposed to potential support if it existed for it.  

3) I understand that this model could make Clip Preferences redundant but prefer to avoid discussion about imageEffectV2 etc at this stage

4) Although in the end it's just name - value pairs, there can be an implicit data hierarchy (to keep same example target, it has a 3 gamma values (Power.R, Power.G, Power.B):   ASC_CDL,  power, RGB  (even if flattened as a simple key name to ASC_CDL_power_R etc or even OFX_CDL_Power_R if a provided meta data)...

5) Also in terms of objectives I understand in life a number of things could support that abstraction (point cloud per frame, associated audio track files, etc) however as a v1.5 objective maybe the objective is to simply deal with data streams that would typically be reasonable to save in a parameter stream (even if the implementation does not necessarily specify it must).

6) Time model:

From an API perspective, we have two established models:  Clip Preferences (clip properties) and Parameters. Clips at least in specs have FPS value (where 1/FPS = a fraction of second in theory though as many don't report that in a fashion that enables to convert OFX frame time to second) and Parameters work in OFX frame time, but have an additional time variant ability which is to be able to find next key-frame time. 

SO, as another example from there:  Let's say I have drone video and there is an associated GPS log file that is 15 Hz (for this discussion it does not matter where that log is: JSON/XMP/as a track in the MP4, as a separate CSV file...).  The video is captured at 50P.  As part of process of ingesting meta-data goodness to forward to effects is this to be converted in frame time once in OFX world?  (Also I understand a simple application is to tag TC to an element but then there is no such thing as timecode in SMPTE sense for a 200 Hz stream of data...). 

So another simple example: Let's say I shoot a 45 minutes video continuous, and while shooting I have the option of clicking a button once in a while to mark good frame.... So I clicked the button 7 times over 45 minutes, and this would be available as meta data:  In this case I would probably want to open the meta-data, collect the 7 values, and write them as key-frames in my effect as marker and then close the meta-data.  (This is a bit what lifetime means in Alexadre write-up - so the host then does not have to keep open all that for the whole session). 

So transposing to "color" meta-data.As meta-data it would be associated to a clip (at least the output clip). However clip properties are not dynamic (which is OK). So if some day one provided say a 3x3 AWB data stream (or something else perhaps today collectable using EXIF for timelapse image sequences) it would then need for the effect to have capacity to use the parameter channels collection mechanism I believe. There is still an association to clip for accessing the data from the effects POV. 

Pierre, before getting into the specifics (and I agree too that this should be implemented using the property suite) we need to clarify what metadatas are:

- A metadata is a key/value pair, where the value must be encodable in the current property set supported formats (int, double, string)

- A metadata key is a unique identifier: 2 meta-datas cannot have the same key

- A string metadata should be able to encode binary data

- There can be temporal metadata (e.g: timecode) and time invariant metadata (e.g: pixel aspect ratio)

- We already have a kOfxImageEffectActionGetClipPreferences action which returns some metadatas, though they are all time invariant

- Metadatas need to be fetched on a clip instance, much like we do today with the clipGetImage function

- We need to think about metadatas lifetime: we do not want each effect to have a copy of the input clips metadata.
Therefore, the host needs to be able to cache temporarily one metadata out of the others,
and the host needs to be able to ask for a single meta-data by giving its key, or fetching it all at once by not specying any key:

If a plug-in just needs to inquire about the "timecode" in a source clip,

the host should only need to only fetch the "timecode" metadata from upstream effects and not everything else.

The way I see it:

- A property on an imageEffect descriptor and host descriptor indicating whether a host/plug-in supports the suite below:

kOfxImageEffectPropSupportsMetaData int x1 property
Valid values: 0 if the host/plug-in does not support it, 1 otherwise.
Default value: 0

- A new suite OfxMetaDataSuiteV1 or OfxImageEffectSuiteV2, including the following functions:

/* @brief Get the time-invariant metadata property set corresponding to the given key.
@param inArgs The current action inArgs that were passed by the host to avoid lost
of context by the host.
@param clip The clip on which to get the metadata from
@param key The name of the metadata. If NULL, the host must return a property set
containing all metadatas available on the clip.

@returns kOfxStatOK - The metadata property set was successfully fetched and returned
in the handle
kOfxStatFailed - The metadata property set could not be fetched because
no metadata matches the corresponding key
kOfxStatErrBadHandle - The inArgs handle or clip handle was invalid
kOfxStatErrMemory - The host had not enough memory to complete the operation
plug-in should abort whatever it was doing
*/
OfxStatus clipGetMetadata(OfxPropertySetHandle inArgs,
OfxImageClipHandle clip,
const char* key,
OfxPropertySetHandle* propertySetHandle);

/* @brief Get the time-variant metadata property set corresponding to the given key.
@param inArgs The current action inArgs that were passed by the host to avoid lost
of context by the host.
@param clip The clip on which to get the metadata from
@param time The time at which to sample the metadata
@param view The view on which to sample the metadata (to support the multi-view suite)
@param key The name of the metadata. If NULL, the host must return a property set
containing all metadatas available on the clip.

@returns kOfxStatOK - The metadata property set was successfully fetched and returned
in the handle
kOfxStatFailed - The metadata property set could not be fetched because
no metadata matches the corresponding key
kOfxStatErrBadHandle - The inArgs handle or clip handle was invalid
kOfxStatErrMemory - The host had not enough memory to complete the operation
plug-in should abort whatever it was doing
*/
OfxStatus clipGetMetadataAtTime(OfxPropertySetHandle inArgs,
OfxImageClipHandle clip,
OfxTime time,
int view,
const char* key,
OfxPropertySetHandle* propertySetHandle);

To me it should make more sense to include this in ImageEffectSuiteV2, but we would also need
to rev-up all other function, passing them a OfxPropertySetHandle inArgs so we can deal with
host lost of context in the meantime.

- kOfxImageClipPropTimeInvariantMetadataPresent string xN property
on a clip instance indicating a list of time invariant metadata keys available on the given clip.

The host should in-turn return the time invariant metadata keys from the corresponding effect
by calling the kOfxImageEffectActionGetTimeInvariantMetadata action

- kOfxImageClipPropTimeVariantMetadataPresent string xN property
on a clip instance indicating a list of time/view variant metadata keys available on the given clip.

The host should in-turn return the time variant meta-data key from the corresponding effect
by calling the kOfxImageEffectActionGetTimeVariantMetadata action

- The kOfxImageEffectActionGetTimeInvariantMetadata action:

* inArgs:
— kOfxImageEffectPropMetadataKey string x1 property, indicating the key of the metadata to return.
If empty, the plug-in should return all available time invariant metadata.
Note that the plug-in may disregard the key and return all available metadatas anyway:
it is just here so it can be optimized by the host when it only needs to fetch a single metadata.

* outArgs:
A property set where the plug-in can create properties of type int, double or string:
these properties are the metadatas encoded by the plug-in.

A string property may contain binary data, so it should be handled correctly by the host and the plug-in.
The property suite has been modified accordingly.

The effect may only return time invariant metadata from this action.

- The kOfxImageEffectPropPassThroughMetadata property of type string x N is already created
in the outArgs and the plug-in may indicate into it the kOfxImageEffectPropMetadataKey of all
metadata inherited from a source clip.
The source clip itself is mentioned in the following property:

- The kOfxImageEffectPropMetadataPassThroughClip property of type string x1 already created in the outArgs:
the plug-in may indicate the name of the clip from which to pass-through properties
indicated in the kOfxImageEffectPropInheritedMetadata property.

- The kOfxImageEffectActionGetTimeVariantMetadata action:

It is the same as the action above, except that the effect may only return time/view variant
metadatas from this action, such as a timecode.

The inArgs take these additional properties…

— kOfxPropTime double x1, The time at which to sample the metadata

— To support Nuke multi-view extension, we also add the kFnOfxImageEffectPropView int x1
property indicating the view on which to sample the metadata.

Maybe the 2 actions above could be a single action, however there needs to be a way to indicate
to the host which metadata are time invariant and which are not.
This would require the outArgs property set to contain 2 sub property sets, which is not feasible currently.
Since both actions serve a very different purpose, I think this is best to isolate them.

Note that these 2 actions can be called on a render thread as well as the main thread.

A plug-in may want to control its metadatas with parameters but it wouldn't be able to
update them directly from the get metadata action since it may be called on a render thread.

To enable this, a new property is added on a plug-in parameter descriptor:

- kOfxParamPropMetadataSlave int x1
Valid values - 0 or 1
Default value - 0
Indicates whether a change to this parameter involves a change to the metadatas.
If set to 1, the host should not only call the kOfxActionInstanceChanged action but also
call this action with a type kOfxTypeClip recursively downstream on all effects.

This enables a plug-in to update its parameters when a metadata is changed upstream.

In order to implement the 2 actions above we need to add some modifications to the property suite,
as mentioned by Pierre:

- Add the following functions to the PropertySuite:

/* @brief Creates a int property with the given dimension.
The property will have by default the values pointed to by the defaultValue array.
*/
OfxStatus createProperty(OfxPropertySetHandle properties, const char* property, int dimension, int* defaultValue)

OfxStatus createProperty(OfxPropertySetHandle properties, const char* property, int dimension, double* defaultValue)

// Same as above, except that it also takes a nBytes array of size 'dimension' indicating for each string its size in bytes.
OfxStatus createProperty(OfxPropertySetHandle properties, const char* property, int dimension, int *nBytes, const char** defaultValue)

I don’t think we need to add a version for pointer properties since it then brings issues
for the lifetime of the pointer itself.

To properly handle binary metadata, we need to add the following functions:

/* @brief Set a single value in a string property being plain binary data.
@param properties is the handle of the thing holding the property
@param property is the string labelling the property
@param index is for multidimenstional properties and is dimension of the one we are setting
@param nBytes The number of bytes to copy from the "value" buffer.
@param value is the buffer we are setting. The host should make a copy of this buffer.

*/
OfxStatus propSetBytes(OfxPropertySetHandle properties, const char* property, int index, int nBytes, const char* value);

/* @brief Set a single value in a string property being plain binary data.
@param properties is the handle of the thing holding the property
@param property is the string labelling the property
@param index is for multidimenstional properties and is dimension of the one we are setting
@param nBytes The number of bytes to copy onto the "value" buffer.
@param value is the buffer we are getting. The buffer should already be allocated of nBytes.
The host should make a copy to this buffer.

*/
OfxStatus propGetBytes(OfxPropertySetHandle properties, const char* property, int index, int nBytes, char** value);

/* @brief Getting the number of bytes in a single dimension of a string property.
@param properties is the handle of the thing holding the property
@param property is the string labelling the property
@param index is for multidimenstional properties and is dimension of the one we are setting
@param nBytes The number of bytes in the buffer.
This enables the plug-in to call propGetBytes with an appropriate number of bytes.
*/
OfxStatus propGetNumBytes(OfxPropertySetHandle properties, const char* property, int index, int *nBytes);

/* @brief Set all values at once in a string property being plain binary data.
@param properties is the handle of the thing holding the property
@param property is the string labelling the property
@param count is for multidimensional properties and is number of dimensions of the property
@param nBytes The number of bytes to copy from the "value" buffer. This is an array of size "count" containing
for each dimension the size in bytes of the corresponding buffer in "value".
@param value is an array of size "count" containing for each dimension
the buffer we are setting. The host should make a copy of this buffer.

*/
OfxStatus propSetBytesN(OfxPropertySetHandle properties, const char* property, int count, int *nBytes, const char** value);

/* @brief Set all values at once in a string property being plain binary data.
@param properties is the handle of the thing holding the property
@param property is the string labelling the property
@param count is for multidimensional properties and is number of dimensions of the property
@param nBytes The number of bytes to copy to the "value" buffer. This is an array of size "count" containing
for each dimension the size in bytes of the corresponding buffer in "value".
@param value is an array of size "count" containing for each dimension
the buffer we are setting. Each buffer is already allocated of at least nBytes.
The host should make a copy to this buffer.

*/
OfxStatus propGetBytesN(OfxPropertySetHandle properties, const char* property, int count, int *nBytes, char*** value);

/* @brief Getting the number of bytes in a single dimension of a string property.
@param properties is the handle of the thing holding the property
@param property is the string labelling the property
@param count is for multidimensional properties and is number of dimensions of the property
@param nBytes The number of bytes for each dimension.
This enables the plug-in to call propGetBytes with an appropriate number of bytes.
*/
OfxStatus propGetNumBytesN(OfxPropertySetHandle properties, const char* property, int count, int **nBytes);

To enable the plug-in to iterate over available metadatas, we need to introduce these 2 functions:

/** @brief Get the number of properties in a property set.
@param numProperties In output, this is the number of properties in the set.
@returns - kOfxStatFailed The host did not manage to figure out the number of properties
in the set or does not want the plug-in to be able to iterate over the properties.
- kOfxStatOk The host successfully returned the number of properties in the property set.
- kOfxStatErrBadHandle The properties handle is invalid
*/
OfxStatus propGetNumProperties(OfxPropertySetHandle properties, int* numProperties);

/** @brief Get the name of a property in a property set
@param index The index of the property to fetch in the set. This must be a number ranging
from 0 to the number of properties returned by the propGetNumProperties function.
@param property In output, a UTF-8 encoded c-string containing the name of the property.
This name can be used to call any other function of the property suite. The string is
valid until the action ends.
@returns - kOfxStatFailed The host did not manage to figure out the property name
in the set or does not want the plug-in to be able to iterate over the properties.
- kOfxStatOk The host successfully returned the property name
- kOfxStatErrBadIndex The index is invalid
- kOfxStatErrBadHandle The properties handle is invalid
*/
OfxStatus propGetPropertyName(OfxPropertySetHandle properties, int index, char** property);

It is clear that with these additions, the kOfxImageEffectActionGetClipPreferences action is made obsolete.

Plug-ins marked with kOfxImageEffectPropSupportsMetadata=1 do not have to implement the
kOfxImageEffectActionGetClipPreferences action for a host that is also marked kOfxImageEffectPropSupportsMetadata=1
and may specify the properties in outArgs of the kOfxImageEffectActionGetClipPreferences in outArgs of the
kOfxImageEffectActionGetTimeInvariantMetadata action instead.
These are the following properties:
- kOfxImageEffectPropFrameRate
- kOfxImageEffectPropFieldOrder
- kOfxImageEffectPropPreMultiplication
- kOfxImageEffectPropContinuousSamples
- kOfxImageEffectPropFrameVarying.

If they are unspecified, they will be assigned their default values as specified by the kOfxImageEffectActionGetClipPreferences spec.

If the host is kOfxImageEffectPropSupportsMetadata=1 it should still call kOfxImageEffectActionGetClipPreferences
for plug-ins that are kOfxImageEffectPropSupportsMetadata=0

This suite also fixes the following proposal http://openeffects.org/standard_changes/238
and proposes an implementation to fix it.


Standard metadata:
-------------------

I did not specify any standard metadata, however we could specify some common metadata
in the same way that we did for the kOfxImageEffectActionGetClipPreferences action.

This would allow plug-ins to communicate non-yet standard piece of data, without forcing
a host to change its implementation.


Caching:
--------

The host may cache the properties of a plug-in, by copying the content of the properties
to a local cache.
Binary meta-data must then ensure not to contain pointers to external data structures outside
of that buffer region.

This draft also provides a standard way to cache piece of data and proposes an implementation
for http://openeffects.org/standard_changes/caching

Here's some additional thoughts -

I looked a bit more (inertial sensors, motion control system, drone API, stand-alone attachement) and one thing that should be part of design is:

1) Such data channel have a sampling rate that might be different than FPS (e.g. gyro might be 150 Hz and GPS 15 Hz and whatever the video capture FPS is).
It's almost like a data channel that is dynamic should have a similar structure to a clip?

Since time in OFX is typically in double frame time, does that imply channel sampling rate to be normalized to frame time (e.g. next value is at +0.00333).

2) Data channel like aux Image Channels tend to be grouped ( e.g. 3 Rotations) - the same channel  grouping for this and Image Channels might mimic each others.

Naming is a challenge as we probably all name similar things slightly different. We might consider part of a string to be following some predefined names so we have at effects level opportunity to filter list of available channels???