add part of opencv
|
After Width: | Height: | Size: 15 KiB |
BIN
Lib/opencv/sources/doc/tutorials/dnn/dnn_android/11_demo.jpg
Normal file
|
After Width: | Height: | Size: 118 KiB |
|
After Width: | Height: | Size: 41 KiB |
|
After Width: | Height: | Size: 55 KiB |
|
After Width: | Height: | Size: 34 KiB |
|
After Width: | Height: | Size: 37 KiB |
BIN
Lib/opencv/sources/doc/tutorials/dnn/dnn_android/5_setup.png
Normal file
|
After Width: | Height: | Size: 5.6 KiB |
|
After Width: | Height: | Size: 9.5 KiB |
|
After Width: | Height: | Size: 28 KiB |
|
After Width: | Height: | Size: 52 KiB |
|
After Width: | Height: | Size: 56 KiB |
@@ -0,0 +1,97 @@
|
||||
# How to run deep networks on Android device {#tutorial_dnn_android}
|
||||
|
||||
## Introduction
|
||||
In this tutorial you'll know how to run deep learning networks on Android device
|
||||
using OpenCV deep learning module.
|
||||
|
||||
Tutorial was written for the following versions of corresponding software:
|
||||
- Android Studio 2.3.3
|
||||
- OpenCV 3.3.0+
|
||||
|
||||
## Requirements
|
||||
|
||||
- Download and install Android Studio from https://developer.android.com/studio.
|
||||
|
||||
- Get the latest pre-built OpenCV for Android release from https://github.com/opencv/opencv/releases and unpack it (for example, `opencv-4.2.0-android-sdk.zip`).
|
||||
|
||||
- Download MobileNet object detection model from https://github.com/chuanqi305/MobileNet-SSD. We need a configuration file `MobileNetSSD_deploy.prototxt` and weights `MobileNetSSD_deploy.caffemodel`.
|
||||
|
||||
## Create an empty Android Studio project
|
||||
- Open Android Studio. Start a new project. Let's call it `opencv_mobilenet`.
|
||||

|
||||
|
||||
- Keep default target settings.
|
||||

|
||||
|
||||
- Use "Empty Activity" template. Name activity as `MainActivity` with a
|
||||
corresponding layout `activity_main`.
|
||||

|
||||
|
||||

|
||||
|
||||
- Wait until a project was created. Go to `Run->Edit Configurations`.
|
||||
Choose `USB Device` as target device for runs.
|
||||

|
||||
Plug in your device and run the project. It should be installed and launched
|
||||
successfully before we'll go next.
|
||||
@note Read @ref tutorial_android_dev_intro in case of problems.
|
||||
|
||||

|
||||
|
||||
## Add OpenCV dependency
|
||||
|
||||
- Go to `File->New->Import module` and provide a path to `unpacked_OpenCV_package/sdk/java`. The name of module detects automatically.
|
||||
Disable all features that Android Studio will suggest you on the next window.
|
||||

|
||||
|
||||

|
||||
|
||||
- Open two files:
|
||||
|
||||
1. `AndroidStudioProjects/opencv_mobilenet/app/build.gradle`
|
||||
|
||||
2. `AndroidStudioProjects/opencv_mobilenet/openCVLibrary330/build.gradle`
|
||||
|
||||
Copy both `compileSdkVersion` and `buildToolsVersion` from the first file to
|
||||
the second one.
|
||||
|
||||
`compileSdkVersion 14` -> `compileSdkVersion 26`
|
||||
|
||||
`buildToolsVersion "25.0.0"` -> `buildToolsVersion "26.0.1"`
|
||||
|
||||
- Make the project. There is no errors should be at this point.
|
||||
|
||||
- Go to `File->Project Structure`. Add OpenCV module dependency.
|
||||

|
||||
|
||||

|
||||
|
||||
- Install once an appropriate OpenCV manager from `unpacked_OpenCV_package/apk`
|
||||
to target device.
|
||||
@code
|
||||
adb install OpenCV_3.3.0_Manager_3.30_armeabi-v7a.apk
|
||||
@endcode
|
||||
|
||||
- Congratulations! We're ready now to make a sample using OpenCV.
|
||||
|
||||
## Make a sample
|
||||
Our sample will takes pictures from a camera, forwards it into a deep network and
|
||||
receives a set of rectangles, class identifiers and confidence values in `[0, 1]`
|
||||
range.
|
||||
|
||||
- First of all, we need to add a necessary widget which displays processed
|
||||
frames. Modify `app/src/main/res/layout/activity_main.xml`:
|
||||
@include android/mobilenet-objdetect/res/layout/activity_main.xml
|
||||
|
||||
- Put downloaded `MobileNetSSD_deploy.prototxt` and `MobileNetSSD_deploy.caffemodel`
|
||||
into `app/build/intermediates/assets/debug` folder.
|
||||
|
||||
- Modify `/app/src/main/AndroidManifest.xml` to enable full-screen mode, set up
|
||||
a correct screen orientation and allow to use a camera.
|
||||
@include android/mobilenet-objdetect/AndroidManifest.xml
|
||||
|
||||
- Replace content of `app/src/main/java/org/opencv/samples/opencv_mobilenet/MainActivity.java`:
|
||||
@include android/mobilenet-objdetect/src/org/opencv/samples/opencv_mobilenet/MainActivity.java
|
||||
|
||||
- Launch an application and make a fun!
|
||||

|
||||
@@ -0,0 +1,226 @@
|
||||
# Custom deep learning layers support {#tutorial_dnn_custom_layers}
|
||||
|
||||
## Introduction
|
||||
Deep learning is a fast growing area. The new approaches to build neural networks
|
||||
usually introduce new types of layers. They could be modifications of existing
|
||||
ones or implement outstanding researching ideas.
|
||||
|
||||
OpenCV gives an opportunity to import and run networks from different deep learning
|
||||
frameworks. There are a number of the most popular layers. However you can face
|
||||
a problem that your network cannot be imported using OpenCV because of unimplemented layers.
|
||||
|
||||
The first solution is to create a feature request at https://github.com/opencv/opencv/issues
|
||||
mentioning details such a source of model and type of new layer. A new layer could
|
||||
be implemented if OpenCV community shares this need.
|
||||
|
||||
The second way is to define a **custom layer** so OpenCV's deep learning engine
|
||||
will know how to use it. This tutorial is dedicated to show you a process of deep
|
||||
learning models import customization.
|
||||
|
||||
## Define a custom layer in C++
|
||||
Deep learning layer is a building block of network's pipeline.
|
||||
It has connections to **input blobs** and produces results to **output blobs**.
|
||||
There are trained **weights** and **hyper-parameters**.
|
||||
Layers' names, types, weights and hyper-parameters are stored in files are generated by
|
||||
native frameworks during training. If OpenCV mets unknown layer type it throws an
|
||||
exception trying to read a model:
|
||||
|
||||
```
|
||||
Unspecified error: Can't create layer "layer_name" of type "MyType" in function getLayerInstance
|
||||
```
|
||||
|
||||
To import the model correctly you have to derive a class from cv::dnn::Layer with
|
||||
the following methods:
|
||||
|
||||
@snippet dnn/custom_layers.hpp A custom layer interface
|
||||
|
||||
And register it before the import:
|
||||
|
||||
@snippet dnn/custom_layers.hpp Register a custom layer
|
||||
|
||||
@note `MyType` is a type of unimplemented layer from the thrown exception.
|
||||
|
||||
Let's see what all the methods do:
|
||||
|
||||
- Constructor
|
||||
|
||||
@snippet dnn/custom_layers.hpp MyLayer::MyLayer
|
||||
|
||||
Retrieves hyper-parameters from cv::dnn::LayerParams. If your layer has trainable
|
||||
weights they will be already stored in the Layer's member cv::dnn::Layer::blobs.
|
||||
|
||||
- A static method `create`
|
||||
|
||||
@snippet dnn/custom_layers.hpp MyLayer::create
|
||||
|
||||
This method should create an instance of you layer and return cv::Ptr with it.
|
||||
|
||||
- Output blobs' shape computation
|
||||
|
||||
@snippet dnn/custom_layers.hpp MyLayer::getMemoryShapes
|
||||
|
||||
Returns layer's output shapes depends on input shapes. You may request an extra
|
||||
memory using `internals`.
|
||||
|
||||
- Run a layer
|
||||
|
||||
@snippet dnn/custom_layers.hpp MyLayer::forward
|
||||
|
||||
Implement a layer's logic here. Compute outputs for given inputs.
|
||||
|
||||
@note OpenCV manages memory allocated for layers. In the most cases the same memory
|
||||
can be reused between layers. So your `forward` implementation should not rely that
|
||||
the second invocation of `forward` will has the same data at `outputs` and `internals`.
|
||||
|
||||
- Optional `finalize` method
|
||||
|
||||
@snippet dnn/custom_layers.hpp MyLayer::finalize
|
||||
|
||||
The chain of methods are the following: OpenCV deep learning engine calls `create`
|
||||
method once then it calls `getMemoryShapes` for an every created layer then you
|
||||
can make some preparations depends on known input dimensions at cv::dnn::Layer::finalize.
|
||||
After network was initialized only `forward` method is called for an every network's input.
|
||||
|
||||
@note Varying input blobs' sizes such height or width or batch size you make OpenCV
|
||||
reallocate all the internal memory. That leads efficiency gaps. Try to initialize
|
||||
and deploy models using a fixed batch size and image's dimensions.
|
||||
|
||||
## Example: custom layer from Caffe
|
||||
Let's create a custom layer `Interp` from https://github.com/cdmh/deeplab-public.
|
||||
It's just a simple resize that takes an input blob of size `N x C x Hi x Wi` and returns
|
||||
an output blob of size `N x C x Ho x Wo` where `N` is a batch size, `C` is a number of channels,
|
||||
`Hi x Wi` and `Ho x Wo` are input and output `height x width` correspondingly.
|
||||
This layer has no trainable weights but it has hyper-parameters to specify an output size.
|
||||
|
||||
In example,
|
||||
~~~~~~~~~~~~~
|
||||
layer {
|
||||
name: "output"
|
||||
type: "Interp"
|
||||
bottom: "input"
|
||||
top: "output"
|
||||
interp_param {
|
||||
height: 9
|
||||
width: 8
|
||||
}
|
||||
}
|
||||
~~~~~~~~~~~~~
|
||||
|
||||
This way our implementation can look like:
|
||||
|
||||
@snippet dnn/custom_layers.hpp InterpLayer
|
||||
|
||||
Next we need to register a new layer type and try to import the model.
|
||||
|
||||
@snippet dnn/custom_layers.hpp Register InterpLayer
|
||||
|
||||
## Example: custom layer from TensorFlow
|
||||
This is an example of how to import a network with [tf.image.resize_bilinear](https://www.tensorflow.org/versions/master/api_docs/python/tf/image/resize_bilinear)
|
||||
operation. This is also a resize but with an implementation different from OpenCV's or `Interp` above.
|
||||
|
||||
Let's create a single layer network:
|
||||
~~~~~~~~~~~~~{.py}
|
||||
inp = tf.placeholder(tf.float32, [2, 3, 4, 5], 'input')
|
||||
resized = tf.image.resize_bilinear(inp, size=[9, 8], name='resize_bilinear')
|
||||
~~~~~~~~~~~~~
|
||||
OpenCV sees that TensorFlow's graph in the following way:
|
||||
|
||||
```
|
||||
node {
|
||||
name: "input"
|
||||
op: "Placeholder"
|
||||
attr {
|
||||
key: "dtype"
|
||||
value {
|
||||
type: DT_FLOAT
|
||||
}
|
||||
}
|
||||
}
|
||||
node {
|
||||
name: "resize_bilinear/size"
|
||||
op: "Const"
|
||||
attr {
|
||||
key: "dtype"
|
||||
value {
|
||||
type: DT_INT32
|
||||
}
|
||||
}
|
||||
attr {
|
||||
key: "value"
|
||||
value {
|
||||
tensor {
|
||||
dtype: DT_INT32
|
||||
tensor_shape {
|
||||
dim {
|
||||
size: 2
|
||||
}
|
||||
}
|
||||
tensor_content: "\t\000\000\000\010\000\000\000"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
node {
|
||||
name: "resize_bilinear"
|
||||
op: "ResizeBilinear"
|
||||
input: "input:0"
|
||||
input: "resize_bilinear/size"
|
||||
attr {
|
||||
key: "T"
|
||||
value {
|
||||
type: DT_FLOAT
|
||||
}
|
||||
}
|
||||
attr {
|
||||
key: "align_corners"
|
||||
value {
|
||||
b: false
|
||||
}
|
||||
}
|
||||
}
|
||||
library {
|
||||
}
|
||||
```
|
||||
Custom layers import from TensorFlow is designed to put all layer's `attr` into
|
||||
cv::dnn::LayerParams but input `Const` blobs into cv::dnn::Layer::blobs.
|
||||
In our case resize's output shape will be stored in layer's `blobs[0]`.
|
||||
|
||||
@snippet dnn/custom_layers.hpp ResizeBilinearLayer
|
||||
|
||||
Next we register a layer and try to import the model.
|
||||
|
||||
@snippet dnn/custom_layers.hpp Register ResizeBilinearLayer
|
||||
|
||||
## Define a custom layer in Python
|
||||
The following example shows how to customize OpenCV's layers in Python.
|
||||
|
||||
Let's consider [Holistically-Nested Edge Detection](https://arxiv.org/abs/1504.06375)
|
||||
deep learning model. That was trained with one and only difference comparing to
|
||||
a current version of [Caffe framework](http://caffe.berkeleyvision.org/). `Crop`
|
||||
layers that receive two input blobs and crop the first one to match spatial dimensions
|
||||
of the second one used to crop from the center. Nowadays Caffe's layer does it
|
||||
from the top-left corner. So using the latest version of Caffe or OpenCV you'll
|
||||
get shifted results with filled borders.
|
||||
|
||||
Next we're going to replace OpenCV's `Crop` layer that makes top-left cropping by
|
||||
a centric one.
|
||||
|
||||
- Create a class with `getMemoryShapes` and `forward` methods
|
||||
|
||||
@snippet dnn/edge_detection.py CropLayer
|
||||
|
||||
@note Both methods should return lists.
|
||||
|
||||
- Register a new layer.
|
||||
|
||||
@snippet dnn/edge_detection.py Register
|
||||
|
||||
That's it! We've replaced an implemented OpenCV's layer to a custom one.
|
||||
You may find a full script in the [source code](https://github.com/opencv/opencv/tree/master/samples/dnn/edge_detection.py).
|
||||
|
||||
<table border="0">
|
||||
<tr>
|
||||
<td></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
</table>
|
||||
@@ -0,0 +1,65 @@
|
||||
Load Caffe framework models {#tutorial_dnn_googlenet}
|
||||
===========================
|
||||
|
||||
Introduction
|
||||
------------
|
||||
|
||||
In this tutorial you will learn how to use opencv_dnn module for image classification by using
|
||||
GoogLeNet trained network from [Caffe model zoo](http://caffe.berkeleyvision.org/model_zoo.html).
|
||||
|
||||
We will demonstrate results of this example on the following picture.
|
||||

|
||||
|
||||
Source Code
|
||||
-----------
|
||||
|
||||
We will be using snippets from the example application, that can be downloaded [here](https://github.com/opencv/opencv/blob/master/samples/dnn/classification.cpp).
|
||||
|
||||
@include dnn/classification.cpp
|
||||
|
||||
Explanation
|
||||
-----------
|
||||
|
||||
-# Firstly, download GoogLeNet model files:
|
||||
[bvlc_googlenet.prototxt ](https://github.com/opencv/opencv_extra/blob/master/testdata/dnn/bvlc_googlenet.prototxt) and
|
||||
[bvlc_googlenet.caffemodel](http://dl.caffe.berkeleyvision.org/bvlc_googlenet.caffemodel)
|
||||
|
||||
Also you need file with names of [ILSVRC2012](http://image-net.org/challenges/LSVRC/2012/browse-synsets) classes:
|
||||
[classification_classes_ILSVRC2012.txt](https://github.com/opencv/opencv/blob/master/samples/data/dnn/classification_classes_ILSVRC2012.txt).
|
||||
|
||||
Put these files into working dir of this program example.
|
||||
|
||||
-# Read and initialize network using path to .prototxt and .caffemodel files
|
||||
@snippet dnn/classification.cpp Read and initialize network
|
||||
|
||||
You can skip an argument `framework` if one of the files `model` or `config` has an
|
||||
extension `.caffemodel` or `.prototxt`.
|
||||
This way function cv::dnn::readNet can automatically detects a model's format.
|
||||
|
||||
-# Read input image and convert to the blob, acceptable by GoogleNet
|
||||
@snippet dnn/classification.cpp Open a video file or an image file or a camera stream
|
||||
|
||||
cv::VideoCapture can load both images and videos.
|
||||
|
||||
@snippet dnn/classification.cpp Create a 4D blob from a frame
|
||||
We convert the image to a 4-dimensional blob (so-called batch) with `1x3x224x224` shape
|
||||
after applying necessary pre-processing like resizing and mean subtraction
|
||||
`(-104, -117, -123)` for each blue, green and red channels correspondingly using cv::dnn::blobFromImage function.
|
||||
|
||||
-# Pass the blob to the network
|
||||
@snippet dnn/classification.cpp Set input blob
|
||||
|
||||
-# Make forward pass
|
||||
@snippet dnn/classification.cpp Make forward pass
|
||||
During the forward pass output of each network layer is computed, but in this example we need output from the last layer only.
|
||||
|
||||
-# Determine the best class
|
||||
@snippet dnn/classification.cpp Get a class with a highest score
|
||||
We put the output of network, which contain probabilities for each of 1000 ILSVRC2012 image classes, to the `prob` blob.
|
||||
And find the index of element with maximal value in this one. This index corresponds to the class of the image.
|
||||
|
||||
-# Run an example from command line
|
||||
@code
|
||||
./example_dnn_classification --model=bvlc_googlenet.caffemodel --config=bvlc_googlenet.prototxt --width=224 --height=224 --classes=classification_classes_ILSVRC2012.txt --input=space_shuttle.jpg --mean="104 117 123"
|
||||
@endcode
|
||||
For our image we get prediction of class `space shuttle` with more than 99% sureness.
|
||||
@@ -0,0 +1,78 @@
|
||||
# How to enable Halide backend for improve efficiency {#tutorial_dnn_halide}
|
||||
|
||||
## Introduction
|
||||
This tutorial guidelines how to run your models in OpenCV deep learning module
|
||||
using Halide language backend. Halide is an open-source project that let us
|
||||
write image processing algorithms in well-readable format, schedule computations
|
||||
according to specific device and evaluate it with a quite good efficiency.
|
||||
|
||||
An official website of the Halide project: http://halide-lang.org/.
|
||||
|
||||
An up to date efficiency comparison: https://github.com/opencv/opencv/wiki/DNN-Efficiency
|
||||
|
||||
## Requirements
|
||||
### LLVM compiler
|
||||
|
||||
@note LLVM compilation might take a long time.
|
||||
|
||||
- Download LLVM source code from http://releases.llvm.org/4.0.0/llvm-4.0.0.src.tar.xz.
|
||||
Unpack it. Let **llvm_root** is a root directory of source code.
|
||||
|
||||
- Create directory **llvm_root**/tools/clang
|
||||
|
||||
- Download Clang with the same version as LLVM. In our case it will be from
|
||||
http://releases.llvm.org/4.0.0/cfe-4.0.0.src.tar.xz. Unpack it into
|
||||
**llvm_root**/tools/clang. Note that it should be a root for Clang source code.
|
||||
|
||||
- Build LLVM on Linux
|
||||
@code
|
||||
cd llvm_root
|
||||
mkdir build && cd build
|
||||
cmake -DLLVM_ENABLE_TERMINFO=OFF -DLLVM_TARGETS_TO_BUILD="X86" -DLLVM_ENABLE_ASSERTIONS=ON -DCMAKE_BUILD_TYPE=Release ..
|
||||
make -j4
|
||||
@endcode
|
||||
|
||||
- Build LLVM on Windows (Developer Command Prompt)
|
||||
@code
|
||||
mkdir \\path-to-llvm-build\\ && cd \\path-to-llvm-build\\
|
||||
cmake.exe -DLLVM_ENABLE_TERMINFO=OFF -DLLVM_TARGETS_TO_BUILD=X86 -DLLVM_ENABLE_ASSERTIONS=ON -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=\\path-to-llvm-install\\ -G "Visual Studio 14 Win64" \\path-to-llvm-src\\
|
||||
MSBuild.exe /m:4 /t:Build /p:Configuration=Release .\\INSTALL.vcxproj
|
||||
@endcode
|
||||
|
||||
@note `\\path-to-llvm-build\\` and `\\path-to-llvm-install\\` are different directories.
|
||||
|
||||
### Halide language.
|
||||
|
||||
- Download source code from GitHub repository, https://github.com/halide/Halide
|
||||
or using git. The root directory will be a **halide_root**.
|
||||
@code
|
||||
git clone https://github.com/halide/Halide.git
|
||||
@endcode
|
||||
|
||||
- Build Halide on Linux
|
||||
@code
|
||||
cd halide_root
|
||||
mkdir build && cd build
|
||||
cmake -DLLVM_DIR=llvm_root/build/lib/cmake/llvm -DCMAKE_BUILD_TYPE=Release -DLLVM_VERSION=40 -DWITH_TESTS=OFF -DWITH_APPS=OFF -DWITH_TUTORIALS=OFF ..
|
||||
make -j4
|
||||
@endcode
|
||||
|
||||
- Build Halide on Windows (Developer Command Prompt)
|
||||
@code
|
||||
cd halide_root
|
||||
mkdir build && cd build
|
||||
cmake.exe -DLLVM_DIR=\\path-to-llvm-install\\lib\\cmake\\llvm -DLLVM_VERSION=40 -DWITH_TESTS=OFF -DWITH_APPS=OFF -DWITH_TUTORIALS=OFF -DCMAKE_BUILD_TYPE=Release -G "Visual Studio 14 Win64" ..
|
||||
MSBuild.exe /m:4 /t:Build /p:Configuration=Release .\\ALL_BUILD.vcxproj
|
||||
@endcode
|
||||
|
||||
## Build OpenCV with Halide backend
|
||||
When you build OpenCV add the following configuration flags:
|
||||
|
||||
- `WITH_HALIDE` - enable Halide linkage
|
||||
|
||||
- `HALIDE_ROOT_DIR` - path to Halide build directory
|
||||
|
||||
## Set Halide as a preferable backend
|
||||
@code
|
||||
net.setPreferableBackend(DNN_BACKEND_HALIDE);
|
||||
@endcode
|
||||
@@ -0,0 +1,82 @@
|
||||
# How to schedule your network for Halide backend {#tutorial_dnn_halide_scheduling}
|
||||
|
||||
## Introduction
|
||||
Halide code is the same for every device we use. But for achieving the satisfied
|
||||
efficiency we should schedule computations properly. In this tutorial we describe
|
||||
the ways to schedule your networks using Halide backend in OpenCV deep learning module.
|
||||
|
||||
For better understanding of Halide scheduling you might want to read tutorials @ http://halide-lang.org/tutorials.
|
||||
|
||||
If it's your first meeting with Halide in OpenCV, we recommend to start from @ref tutorial_dnn_halide.
|
||||
|
||||
## Configuration files
|
||||
You can schedule computations of Halide pipeline by writing textual configuration files.
|
||||
It means that you can easily vectorize, parallelize and manage loops order of
|
||||
layers computation. Pass path to file with scheduling directives for specific
|
||||
device into ```cv::dnn::Net::setHalideScheduler``` before the first ```cv::dnn::Net::forward``` call.
|
||||
|
||||
Scheduling configuration files represented as YAML files where each node is a
|
||||
scheduled function or a scheduling directive.
|
||||
@code
|
||||
relu1:
|
||||
reorder: [x, c, y]
|
||||
split: { y: 2, c: 8 }
|
||||
parallel: [yo, co]
|
||||
unroll: yi
|
||||
vectorize: { x: 4 }
|
||||
conv1_constant_exterior:
|
||||
compute_at: { relu1: yi }
|
||||
@endcode
|
||||
|
||||
Considered use variables `n` for batch dimension, `c` for channels,
|
||||
`y` for rows and `x` for columns. For variables after split are used names
|
||||
with the same prefix but `o` and `i` suffixes for outer and inner variables
|
||||
correspondingly. In example, for variable `x` in range `[0, 10)` directive
|
||||
`split: { x: 2 }` gives new ones `xo` in range `[0, 5)` and `xi` in range `[0, 2)`.
|
||||
Variable name `x` is no longer available in the same scheduling node.
|
||||
|
||||
You can find scheduling examples at [opencv_extra/testdata/dnn](https://github.com/opencv/opencv_extra/tree/master/testdata/dnn)
|
||||
and use it for schedule your networks.
|
||||
|
||||
## Layers fusing
|
||||
Thanks to layers fusing we can schedule only the top layers of fused sets.
|
||||
Because for every output value we use the fused formula.
|
||||
In example, if you have three layers Convolution + Scale + ReLU one by one,
|
||||
@code
|
||||
conv(x, y, c, n) = sum(...) + bias(c);
|
||||
scale(x, y, c, n) = conv(x, y, c, n) * weights(c);
|
||||
relu(x, y, c, n) = max(scale(x, y, c, n), 0);
|
||||
@endcode
|
||||
|
||||
fused function is something like
|
||||
@code
|
||||
relu(x, y, c, n) = max((sum(...) + bias(c)) * weights(c), 0);
|
||||
@endcode
|
||||
|
||||
So only function called `relu` require scheduling.
|
||||
|
||||
## Scheduling patterns
|
||||
Sometimes networks built using blocked structure that means some layer are
|
||||
identical or quite similar. If you want to apply the same scheduling for
|
||||
different layers accurate to tiling or vectorization factors, define scheduling
|
||||
patterns in section `patterns` at the beginning of scheduling file.
|
||||
Also, your patters may use some parametric variables.
|
||||
@code
|
||||
# At the beginning of the file
|
||||
patterns:
|
||||
fully_connected:
|
||||
split: { c: c_split }
|
||||
fuse: { src: [x, y, co], dst: block }
|
||||
parallel: block
|
||||
vectorize: { ci: c_split }
|
||||
# Somewhere below
|
||||
fc8:
|
||||
pattern: fully_connected
|
||||
params: { c_split: 8 }
|
||||
@endcode
|
||||
|
||||
## Automatic scheduling
|
||||
You can let DNN to schedule layers automatically. Just skip call of ```cv::dnn::Net::setHalideScheduler```. Sometimes it might be even more efficient than manual scheduling.
|
||||
But if specific layers require be scheduled manually, you would be able to
|
||||
mix both manual and automatic scheduling ways. Write scheduling file
|
||||
and skip layers that you want to be scheduled automatically.
|
||||
@@ -0,0 +1,44 @@
|
||||
# How to run deep networks in browser {#tutorial_dnn_javascript}
|
||||
|
||||
## Introduction
|
||||
This tutorial will show us how to run deep learning models using OpenCV.js right
|
||||
in a browser. Tutorial refers a sample of face detection and face recognition
|
||||
models pipeline.
|
||||
|
||||
## Face detection
|
||||
Face detection network gets BGR image as input and produces set of bounding boxes
|
||||
that might contain faces. All that we need is just select the boxes with a strong
|
||||
confidence.
|
||||
|
||||
## Face recognition
|
||||
Network is called OpenFace (project https://github.com/cmusatyalab/openface).
|
||||
Face recognition model receives RGB face image of size `96x96`. Then it returns
|
||||
`128`-dimensional unit vector that represents input face as a point on the unit
|
||||
multidimensional sphere. So difference between two faces is an angle between two
|
||||
output vectors.
|
||||
|
||||
## Sample
|
||||
All the sample is an HTML page that has JavaScript code to use OpenCV.js functionality.
|
||||
You may see an insertion of this page below. Press `Start` button to begin a demo.
|
||||
Press `Add a person` to name a person that is recognized as an unknown one.
|
||||
Next we'll discuss main parts of the code.
|
||||
|
||||
@htmlinclude js_face_recognition.html
|
||||
|
||||
-# Run face detection network to detect faces on input image.
|
||||
@snippet dnn/js_face_recognition.html Run face detection model
|
||||
You may play with input blob sizes to balance detection quality and efficiency.
|
||||
The bigger input blob the smaller faces may be detected.
|
||||
|
||||
-# Run face recognition network to receive `128`-dimensional unit feature vector by input face image.
|
||||
@snippet dnn/js_face_recognition.html Get 128 floating points feature vector
|
||||
|
||||
-# Perform a recognition.
|
||||
@snippet dnn/js_face_recognition.html Recognize
|
||||
Match a new feature vector with registered ones. Return a name of the best matched person.
|
||||
|
||||
-# The main loop.
|
||||
@snippet dnn/js_face_recognition.html Define frames processing
|
||||
A main loop of our application receives a frames from a camera and makes a recognition
|
||||
of an every detected face on the frame. We start this function ones when OpenCV.js was
|
||||
initialized and deep learning models were downloaded.
|
||||
@@ -0,0 +1,44 @@
|
||||
YOLO DNNs {#tutorial_dnn_yolo}
|
||||
===============================
|
||||
|
||||
Introduction
|
||||
------------
|
||||
|
||||
In this text you will learn how to use opencv_dnn module using yolo_object_detection (Sample of using OpenCV dnn module in real time with device capture, video and image).
|
||||
|
||||
We will demonstrate results of this example on the following picture.
|
||||

|
||||
|
||||
Examples
|
||||
--------
|
||||
|
||||
VIDEO DEMO:
|
||||
@youtube{NHtRlndE2cg}
|
||||
|
||||
Source Code
|
||||
-----------
|
||||
|
||||
Use a universal sample for object detection models written
|
||||
[in C++](https://github.com/opencv/opencv/blob/master/samples/dnn/object_detection.cpp) and
|
||||
[in Python](https://github.com/opencv/opencv/blob/master/samples/dnn/object_detection.py) languages
|
||||
|
||||
Usage examples
|
||||
--------------
|
||||
|
||||
Execute in webcam:
|
||||
|
||||
@code{.bash}
|
||||
|
||||
$ example_dnn_object_detection --config=[PATH-TO-DARKNET]/cfg/yolo.cfg --model=[PATH-TO-DARKNET]/yolo.weights --classes=object_detection_classes_pascal_voc.txt --width=416 --height=416 --scale=0.00392 --rgb
|
||||
|
||||
@endcode
|
||||
|
||||
Execute with image or video file:
|
||||
|
||||
@code{.bash}
|
||||
|
||||
$ example_dnn_object_detection --config=[PATH-TO-DARKNET]/cfg/yolo.cfg --model=[PATH-TO-DARKNET]/yolo.weights --classes=object_detection_classes_pascal_voc.txt --width=416 --height=416 --scale=0.00392 --input=[PATH-TO-IMAGE-OR-VIDEO-FILE] --rgb
|
||||
|
||||
@endcode
|
||||
|
||||
Questions and suggestions email to: Alessandro de Oliveira Faria cabelo@opensuse.org or OpenCV Team.
|
||||
BIN
Lib/opencv/sources/doc/tutorials/dnn/dnn_yolo/images/yolo.jpg
Normal file
|
After Width: | Height: | Size: 210 KiB |
BIN
Lib/opencv/sources/doc/tutorials/dnn/images/lena_hed.jpg
Normal file
|
After Width: | Height: | Size: 38 KiB |
BIN
Lib/opencv/sources/doc/tutorials/dnn/images/space_shuttle.jpg
Normal file
|
After Width: | Height: | Size: 27 KiB |
@@ -0,0 +1,58 @@
|
||||
Deep Neural Networks (dnn module) {#tutorial_table_of_content_dnn}
|
||||
=====================================
|
||||
|
||||
- @subpage tutorial_dnn_googlenet
|
||||
|
||||
*Compatibility:* \> OpenCV 3.3
|
||||
|
||||
*Author:* Vitaliy Lyudvichenko
|
||||
|
||||
In this tutorial you will learn how to use opencv_dnn module for image classification by using GoogLeNet trained network from Caffe model zoo.
|
||||
|
||||
- @subpage tutorial_dnn_halide
|
||||
|
||||
*Compatibility:* \> OpenCV 3.3
|
||||
|
||||
*Author:* Dmitry Kurtaev
|
||||
|
||||
This tutorial guidelines how to run your models in OpenCV deep learning module using Halide language backend.
|
||||
|
||||
- @subpage tutorial_dnn_halide_scheduling
|
||||
|
||||
*Compatibility:* \> OpenCV 3.3
|
||||
|
||||
*Author:* Dmitry Kurtaev
|
||||
|
||||
In this tutorial we describe the ways to schedule your networks using Halide backend in OpenCV deep learning module.
|
||||
|
||||
- @subpage tutorial_dnn_android
|
||||
|
||||
*Compatibility:* \> OpenCV 3.3
|
||||
|
||||
*Author:* Dmitry Kurtaev
|
||||
|
||||
This tutorial will show you how to run deep learning model using OpenCV on Android device.
|
||||
|
||||
- @subpage tutorial_dnn_yolo
|
||||
|
||||
*Compatibility:* \> OpenCV 3.3.1
|
||||
|
||||
*Author:* Alessandro de Oliveira Faria
|
||||
|
||||
In this tutorial you will learn how to use opencv_dnn module using yolo_object_detection with device capture, video file or image.
|
||||
|
||||
- @subpage tutorial_dnn_javascript
|
||||
|
||||
*Compatibility:* \> OpenCV 3.3.1
|
||||
|
||||
*Author:* Dmitry Kurtaev
|
||||
|
||||
In this tutorial we'll run deep learning models in browser using OpenCV.js.
|
||||
|
||||
- @subpage tutorial_dnn_custom_layers
|
||||
|
||||
*Compatibility:* \> OpenCV 3.4.1
|
||||
|
||||
*Author:* Dmitry Kurtaev
|
||||
|
||||
How to define custom layers to import networks.
|
||||