Simple TFLite Running on Android Phone

Checking State from Image with TensorFlow is very easy.

Background Story

One day at office time, i am staring at local TV Channel playing on our App. My job is develop OTT App that provides lot of awesome content and some Local TV Channels. While i am stare at the app i saw something. Something that give me a Wonderfull idea.

I saw an opportunity to adding ads. On local TV Channel when playing ads, i can replace the content with our ads. How we know when is TV Channel playing ads? there is a pattern

The Pattern

Channel TV Playing Content :

Channel TV Playing Ads :

The Goal

My Goal

Replace content/stream with our ads, when TV Channel playing Ads.

This Blog Goal

Our App can checking automatically, TV Channel playing Ads or content.

Execution

Train TFLite Model

A TensorFlow model is a data structure that contains the logic and knowledge of a machine learning network trained to solve a particular problem. There are many ways to obtain a TensorFlow model, from using pre-trained models to training your own.

Why TFLite Model? Mobile app support TF Lite for model use for TensorFlow. TFLite is the smaller model and already quantize. So this model is faster that TensorFlow Model with small accuration reduction.

Firebase Machine Learning

The easiest way to train TFLite Model is using Firebase Machine Learning. You can just upload some pictures and labelling, and train.. very easy. Tutorial Train with Firebase

To Train Your Model it self follow this link

Step by Step Train Model using Tensorflow (CNN)

Use The Model

I use Firebase MLKit to use The Model in my Android App Project.

• Open you Android App Project.
• If you haven’t already, add Firebase to your Android project.
• prepare your TFLite model files. Or download my TFLite Model, manifest file also the dict file if you want to use mine.
• Put 3 files into you assets Folder

• Add the dependencies for the ML Kit Android libraries to your module (app-level) Gradle file (usually app/build.gradle):

apply plugin: 'com.android.application'
apply plugin: 'com.google.gms.google-services'

dependencies {
  // ...

  implementation 'com.google.firebase:firebase-ml-model-interpreter:22.0.3'
}

• Then, add the following to your app’s build.gradle file to ensure Gradle doesn’t compress the models when building the app:

android {
    // ...
    aaptOptions {
        noCompress "tflite"  // Your model's file extension: "tflite", "lite", etc.
    }
}

• Load the model

Specify the you model file name. Mine is “model.tflite” . And create labeler

val localModel = FirebaseCustomLocalModel.Builder()
    .setAssetFilePath("model.tflite")
    .build()
val labelerOptions = FirebaseVisionOnDeviceAutoMLImageLabelerOptions.Builder(localModel)
    .setConfidenceThreshold(ICON_CONFIDENCE_THRESHOLD)
    .build()
val labeler = FirebaseVision.getInstance().getOnDeviceAutoMLImageLabeler(labelerOptions)

Checking TV Channel Icon State

Finally.. i can check the TV Channel Icon State using Tensor flow.

The final step is : use following code to check the image. Just give the bitmap object from the image you want to test.

For testing images that you must to convert to bitmap you can download from this link.

val image = FirebaseVisionImage.fromBitmap(bitmap)
try {
    labeler.processImage(image)
        .addOnSuccessListener { labels ->
           for (label in labels) {
                Log.d(TAG, "label : ${label.text}, confidence : ${label.confidence}")
            }
        }
} catch (e: FirebaseMLException) {
    e.printStackTrace()
}

label.text -> represent the name of label result.

label.confidence -> represent the value of label result. the bigger value you got, the more valid result you got.

If you want to other images for testing, you can get from this link. Watch and capture the logo.