SentiSight Embedded SDK

Contact us for price

Object recognition for mobile computer vision applications

SKU: NTRSDKSE102 Category:

Overview

Overview

 

SentiSight Embedded is designed for developers who want to use computer vision-based object recognition in their applications for smartphones, tablets and other mobile devices. SentiSight is designed to be as universal as possible and is able to perform fully automatic or manual object learning. It enables searching for learned objects in images or videos from built-in cameras with PC-like accuracy.
Available as a software development kit that provides for the development of object recognition applications for the devices that are running Android OS.

The SentiSight Embedded 1.3 technology has these capabilities for visual-based object learning and recognition on mobile and embedded devices:

  • Accurate Object Detection

The SentiSight algorithm is able to find out:

  • whether a particular object is present in a scene;
  • wherethe object is located within the scene;
  • how manyinstances of the object occur in the scene.

 

  • Two Algorithms for Object Recognition

Depending on the object type, one of these algorithms (or both) may be used for successful recognition:

    • The blob-based algorithm uses small details of an object as distinctive features that are extracted into an object model and are used later to recognize the object. This algorithm offers performance but is not suitable for solid-colored, reflecting or transparent (glass, etc.) objects.
    • Shape recognition

The shape-based algorithm is useful for objects which do not have any distinctive details but have stable external edges (boundaries) and/or internal edges. This algorithm performs at slower speeds but allows for the recognition of most objects not identified by the blob-based algorithm.

  • Color Usage Mode

The blob-based and shape-based algorithms may be configured to detect object colors and use this information for improving recognition accuracy. This mode enables SentiSight-based applications to distinguish similar objects that only differ in color.

  • Object’s Image Quality Determination

A quality threshold can be used during object learning to ensure that only the best quality object model will be stored into database.

  • Simultaneous Multiple Object Recognition

The SentiSight Embedded is able to detect and recognize several 2D and 3D objects simultaneously.

  • Object Evaluation

The algorithm makes estimates based on the region an object occupies in a scene, providing additional information about the size, orientation and scale of the recognized object.

Highlights

Highlights

 

  • Innovative algorithm, that is tolerant to appearance, object scale, rotation and pose.
  • PC-level accuracy of object detection and processing with mobile devices.
  • Smatphone built-in cameras are suitable for obtaining object images.
  • Compatability and interoperability with PC-side SentiSight-based products.
  • Reasonable prices, flexible licensing and free customer support.

Use Cases

Use Cases

 

  • Recognition of documents, stamps, labels, packaging and other items for sorting, usage monitoring and similar applications;
  • Object counting and inspection;
  • Augmented and extended reality applications for toys, games, devices and Web applications such as: smart toys for children that recognize cards, images, pictograms, etc.; recognition of places based on photographs; recognition of products such as beverages, foods and other consumer goods;
  • Law enforcement applications for identification, such as tattoo recognition.

SDK Contents

SDK Contents

 

SentiSight Embedded 1.3 SDK is based on SentiSight Embedded object recognition technology and is intended for computer vision systems developers and integrators. The SDK allows rapid development of object recognition applications for smart phones, tablets and other devices that are running Android OS. The integrator has complete control over SDK data input/output and can call the functions from the SentiSight Embedded algorithm. SentiSight Embedded SDK is able to capture images and video streams from cameras that are built-in in Android devices.

The SentiSight Embedded 1.3 SDK contains:

  • SentiSight Embedded 1.3 Component– 1 single computer license
  • Javalanguage programming samples and tutorials
  • SentiSight Embedded 1.3 SDK documentation

See the licensing model for more information on specific license types.

SentiSight Embedded Component

The SentiSight Embedded component is able to enroll new objects by creating object models from videos or still images and saving the models to database. Also the component can recognize objects by comparing the object models with the actual objects found in images or video frames.
The recognized objects are tracked on the subsequent frames until they disappear.
One SentiSight Embedded Component license is included with SentiSight Embedded SDK. More licenses for this component can be purchased any time by SentiSight Embedded SDK customers.

System Requirements

System Requirements

 

  • A smartphone or tablet that is runningAndroid 4.0 (API level 14) OS or newer.
    • API level 19 is the recommendedtarget for code compilation.
    • If you have a custom Android-based device or development board,contact us to find out if it is supported.
  • ARM-based processor:
    • Hardware floating-point support is required.
    • 1 GHz processor recommendedfor object processing in less than 1 second. Slower processors may be also used, but the object processing will take longer time.
  • At least20 MB of free RAM should be available for the application. Additional RAM is required for applications that perform 1-to-many identification, as all object models need to be stored in RAM for matching. For example, 100 object models require about 50 MB of additional RAM when 640 x 480 images and blob recognition algorithm used.
  • Free storagespace (built-in flash or external memory card):
    • 30 MB required for SentiSight Embedded components deployment for each separate application.
    • Additional space would be required if an application needs to store original object images. SentiSight Embedded does not require the original object image to be stored for the matching; only the object models need to be stored.
  • Camera. SentiSight Embedded is able to work with all cameras that are supported by Android OS. At least3 MegaPixel (640 x 480 pixels) camera is required for the SentiSight Embedded algorithm. Integrators may also use image files or receive image data from external devices like stand-alone or network cameras.
  • Network connection. A SentiSight Embedded-based application may require network connection foractivating the SentiSight Embedded component licenses. See the list of available activation options in the licensing model for more information. Also, network connection may be required for client/server
  • PC-side development environment requirements:
    • Java SE JDK 6 (or higher)
    • Eclipse Indigo (3.7) IDE
    • Android development environment (at least API level 14 required)
    • Apache Maven 3.1.xor newer
    • Internet connection for activating SentiSight Embedded component licenses

Technical Specifications

Technical Specifications

 

A Java application based on SentiSight Embedded 1.3 technology is able to process a image with an object in less than 1 second. Object recognition algorithms can be run on more than one thread of multi-core processors.
These conditions may alter the performance of the algorithms:

  • Rotation and translation.The algorithm is generally rotation- and translation-invariant in a plane perpendicular to the camera. The algorithm is also invariant for rotations of up to 10-15 degrees beyond a plane perpendicular to the camera. Different views of an object may be added to a model to handle larger rotations.
  • Resolution and scale changes.Scale (size of the object within the image) difference between an object’s model and the object itself can be up to 2-3 times. Objects should contain enough detail, and be large enough to be recognizable.
  • The algorithm is robust to occlusions as big as 50 % of the object if enough unique edges remain visible.
  • Lighting conditions(illumination, shadows and reflectance).
    • Planar objects will have issues with reflectance.
    • 3D objects will have issues with variable lighting conditions – consistent lighting greatly reduces potential problems.
  • In general, transparent objects are difficult to recognize.
  • The algorithm recognizes only rigid/stable objects. At minimum, a significant piece of the object should be unchanging.

Object model size depends on how feature-rich is an object, and thus is individual for each object. All object models should be loaded into RAM before identification, thus the maximum object model database size is limited by the amount of available RAM.

The SentiSight Embedded 1.3 algorithm was tested with a subset of Amsterdam Library of Object Images (ALOI).

  • The subset containedobjects 1-100 from ALOI.
  • Images with object viewpoint variations (ALOI-VIEWcollection) were used. 36 images per object were used.

The average object model size when testing with the 768 x 576 pixels images (the original resolution) was:

  • 7 Megabytes when blob-based algorithm was used.
  • 2 Megabytes when shape-based algorithm was used.

When the images were resized to 320 x 240 pixels, the average object model size was:

  • 2 Megabytes when blob-based algorithm was used.
  • 5 Megabytes when shape-based algorithm was used.

At 0.1 % False Acceptance Rate (FAR), the recognition rate is from 70 % – 99 %, depending on object structural appearance, transparency, etc. For objects with well-defined internal structure, the recognition rate is 98 % – 99 % at 0.1 % FAR.

Licensing Model

Licensing Model

The following licensing model is intended for end-user product developers. Integrators who want to develop and sell a SentiSight Embedded based development tool (with API, programming possibilities, programming samples, etc.), must obtain permission from Neurotechnology and sign a special VAR agreement. For more information please contact us.

Product Development

An integrator should obtain a SentiSight Embedded 1.3 SDK (EUR 339) to develop a product based on SentiSight Embedded technology. The SDK needs to be purchased just once and may be used by all the developers within the integrator’s company.
SentiSight Embedded 1.3 SDK includes SentiSight Embedded component. A license for an individual SentiSight Embedded component is required for each CPU that runs the component (a processor can have any number of cores).
1 single computer license for SentiSight Embedded component is included with SentiSight Embedded 1.3 SDK.
Components are copy-protected – a license is required for a component to run. License activation options are listed below on this page.
Additional component licenses may be obtained by SentiSight Embedded SDK customers as required by their development process.

Product Deployment

To deploy a product developed with SentiSight Embedded SDK, an integrator needs to obtain only the additional licenses required for the particular SentiSight Embedded components that will run on each CPU of their customer’s devices or computers. The available SentiSight Embedded components and license types for product deployment are the same as for product development.
Each SentiSight Embedded component running on a device or computer belonging to the integrator’s customer requires a license. License activation options are listed below on this page.
Prices for SentiSight Embedded 1.3 SDK and additional SentiSight Embedded component licenses can be found here.

Licensing Agreement

The Licensing Agreement contains all licensing terms and conditions.
Note that you unambiguously accept this agreement by placing an order using Neurotechnology online ordering service or by email or other means of communications. Please read the agreement before making an order.

Single Computer Licenses

A single computer license allows the installation and running of a SentiSight Embedded component installation on one CPU (a processor can have any number of cores).
Each single computer license requires activation for a SentiSight Embedded component to run. The available activation options are listed below on this page.
Additional single computer licenses for SentiSight Embedded components may be obtained at any time by SentiSight Embedded SDK customers.

License Activation Options

  • Single computer licenses for SentiSight Embedded components are supplied in three ways:
  • Internet activation. A special license file is stored on a device or computer; the license file allows to run particular SentiSight Embedded components on that device or computer after checking the license over the Internet. Internet connection should be available periodically for a short amount of time. A single computer license can be transferred to another device or computer by moving the license file there and waiting until the previous activation expires.
  • Serial numbers are used to activate licenses for particular SentiSight Embedded components. The activation is done via the Internet or by email. After activation the network connection is not required for single computer license usage.
    Note: activation by serial number is not suitable for virtual environments.
  • Licenses may be stored in a volume license manager dongle. License activation using volume license manager may be performed without connection to the Internet.

Volume License Manager

Volume license manager is used on site by integrators or end users to manage single computer licenses for SentiSight Embedded components via a LAN or the Internet. It consists of license management software and a dongle, used to store the purchased licenses.
The license manager allows the management of installation licenses for SentiSight Embedded components across multiple devices or computers in the network. The number of managed licenses is limited by the number of licenses in the license manager. Once issued, the license is assigned to a specific device or computer on the network.
The volume license manager requires a PC with Microsoft Windows or Linux to run the license management software; the dongle would be connected to the PC.
Additional SentiSight Embedded component licenses for the license manager may be purchased at any time. Neurotechnology will generate an update code and send it to you. Simply enter the code into the license manager to add the purchased licenses.

Enterprise License

The SentiSight Embedded enterprise license allows an unlimited use of SentiSight Embedded components in end-user products for a specific territory, market segment or project. Specific restrictions would be included in the licensing agreement.

The enterprise license price depends on the application size and the number of potential users of the application within the designated territory, market segment or project.

Related Products