Creating Frame Processor Plugins (C++)
C++ Frame Processor Plugins​
C++ based Frame Processor Plugins are built with JSI and can be directly called from a Frame Processor as native JSI functions can be shared across contexts. The benefits of C++ based Frame Processor Plugins lie in the nature of C++ based code:
- Better Performance due to less marshalling required and lower level access
- Shared platform code - e.g. OpenCV Frame Processor Plugins only need to be implemented once
For example:
// 1. Include headers
#include "frameprocessors/FrameHostObject.h"
// 2. Create C++ func
auto myPlugin = [=](jsi::Runtime& runtime,
const jsi::Value& thisArg,
const jsi::Value* args,
size_t count) -> jsi::Value {
auto frame = args[0].asObject(runtime).asHostObject<FrameHostObject>(runtime);
// Unwrap the Frame, then do your Frame Processing here, and return any JSI value as a result.
// For example, you can run very efficient OpenCV tasks here.
return jsi::Value(42);
};
// 3. Wrap C++ func in jsi::Function
auto jsiFunc = jsi::Function::createFromHostFunction(runtime,
jsi::PropNameID::forUtf8(runtime,
"myCppPlugin"),
1,
myPlugin);
// 4. Add it to global so it can be called from JS
runtime.global().setProperty(runtime, "myCppPlugin", jsiFunc);
Then to call that function:
const frameProcessor = useFrameProcessor((frame) => {
'worklet'
const result = global.myCppPlugin(frame)
console.log(`C++ result: ${result}`) // <-- 42
}, [])
To include VisionCamera's C++ library in your plugin's C++ code, you need to link it and include the headers.
- iOS
- Android
VisionCamera is built using CocoaPods, which allow you to include local dependencies.
To add VisionCamera to your package's .podspec
, just add the dependency:
s.dependency "VisionCamera"
Then include it in your C++ code:
#include "FrameProcessors/FrameHostObject.h"
VisionCamera is built using prefabs, which allow you to easily extend the library with your own native code.
To link VisionCamera in your package's CMakeLists.txt
, just add the library:
find_package(react-native-vision-camera REQUIRED CONFIG)
# ...
target_link_libraries(
${PACKAGE_NAME}
# ... other libs
react-native-vision-camera::VisionCamera
)
Then include it in your C++ code:
#include "frame-processor/FrameHostObject.h"