MRZ Recognition on Android Using Dynamsoft Label Recognizer SDK

Xiao Ling - Aug 2 '21 - - Dev Community

Machine Readable Zone (MRZ) recognition is a critical feature in applications that require scanning and processing identity documents, such as passports and visas. On Android devices, implementing accurate MRZ recognition can significantly enhance the efficiency and user experience of such apps. In this article, we will explore how to use the Dynamsoft Label Recognizer SDK to integrate MRZ recognition into your Android application. The SDK provides powerful OCR capabilities specifically tailored for reading MRZ data, allowing you to capture and decode information quickly and reliably, making your app more robust and user-friendly.

MRZ Recognition on Android

Prerequisites

Sample Passport Image with MRZ

Below is a sample MRZ image for testing:

passport mrz image

Implementing MRZ Recognition on Android

Step 1: Configure Your Android Project with Dynamsoft Label Recognizer SDK

  1. In your project's root build.gradle file, add the Dynamsoft Maven repository:

    allprojects {
    repositories {
        google()
        mavenCentral()
        maven { url "https://download2.dynamsoft.com/maven/aar" }
    }
}

  2. In the module's build.gradle file, include the necessary dependencies:

    implementation "com.dynamsoft:dynamsoftbarcodereaderbundle:10.2.1100"
implementation "com.dynamsoft:dynamsoftcodeparser:2.2.11"
implementation "com.dynamsoft:dynamsoftcodeparserdedicator:1.2.20"

    Explanation

    • dynamsoftbarcodereaderbundle: Provides camera controls and OCR recognition capabilities.
    • dynamsoftcodeparser and dynamsoftcodeparserdedicator: These libraries are used to parse MRZ text and extract relevant data.

Step 2: Add MRZ Model Files to the Assets Folder

Ensure the MRZ recognition model files (MRZ.data, Confusable.data, and MRZScanner.json) are added to your project’s assets folder:

MRZ model files

These files are used for recognizing MRZ text, with the JSON file specifying the algorithm settings.

Here is a snippet of the MRZScanner.json configuration:

{
  "CaptureVisionTemplates": [
    {
      "Name": "ReadMRZ",
      "OutputOriginalImage": 0,
      "ImageROIProcessingNameArray": [
        "roi-mrz"
      ],
      "SemanticProcessingNameArray":  ["sp-mrz"],
      "Timeout": 2000
    }
  ],
  "TargetROIDefOptions": [
    {
      "Name": "roi-mrz",
      "TaskSettingNameArray": [
        "task-mrz"
      ]
    }
  ],
  "TextLineSpecificationOptions": [
    {
      "Name": "tls-mrz-passport",
      "BaseTextLineSpecificationName": "tls-base",
      "StringLengthRange": [ 44, 44 ],
      "OutputResults": 1,
      "ExpectedGroupsCount": 1,
      "ConcatResults": 1,
      "ConcatSeparator": "",
      "SubGroups": [
        {
          "StringRegExPattern": "(P[A-Z<][A-Z<]{3}[A-Z<]{39}){(44)}",
          "StringLengthRange": [ 44, 44 ],
          "BaseTextLineSpecificationName": "tls-base"
        },
        {
          "StringRegExPattern": "([A-Z0-9<]{9}[0-9][A-Z<]{3}[0-9]{2}[(01-12)][(01-31)][0-9][MF<][0-9]{2}[(01-12)][(01-31)][0-9][A-Z0-9<]{14}[0-9<][0-9]){(44)}",
          "StringLengthRange": [ 44, 44 ],
          "BaseTextLineSpecificationName": "tls-base"
        }
      ]
    },
    {
      "Name": "tls-mrz-visa-td3",
      "BaseTextLineSpecificationName": "tls-base",
      "StringLengthRange": [ 44, 44 ],
      "OutputResults": 1,
      "ExpectedGroupsCount": 1,
      "ConcatResults": 1,
      "ConcatSeparator": "",
      "SubGroups": [
        {
          "StringRegExPattern": "(V[A-Z<][A-Z<]{3}[A-Z<]{39}){(44)}",
          "StringLengthRange": [ 44, 44 ],
          "BaseTextLineSpecificationName": "tls-base"
        },
        {
          "StringRegExPattern": "([A-Z0-9<]{9}[0-9][A-Z<]{3}[0-9]{2}[(01-12)][(01-31)][0-9][MF<][0-9]{2}[(01-12)][(01-31)][0-9][A-Z0-9<]{14}[A-Z0-9<]{2}){(44)}",
          "StringLengthRange": [ 44, 44 ],
          "BaseTextLineSpecificationName": "tls-base"
        }
      ]
    },
    {
      "Name": "tls-mrz-visa-td2",
      "BaseTextLineSpecificationName": "tls-base",
      "StringLengthRange": [ 36, 36 ],
      "OutputResults": 1,
      "ExpectedGroupsCount": 1,
      "ConcatResults": 1,
      "ConcatSeparator": "",
      "SubGroups": [
        {
          "StringRegExPattern": "(V[A-Z<][A-Z<]{3}[A-Z<]{31}){(36)}",
          "StringLengthRange": [ 36, 36 ],
          "BaseTextLineSpecificationName": "tls-base"
        },
        {
          "StringRegExPattern": "([A-Z0-9<]{9}[0-9][A-Z<]{3}[0-9]{2}[(01-12)][(01-31)][0-9][MF<][0-9]{2}[(01-12)][(01-31)][0-9][A-Z0-9<]{8}){(36)}",
          "StringLengthRange": [ 36, 36 ],
          "BaseTextLineSpecificationName": "tls-base"
        }
      ]
    },
    {
      "Name": "tls-mrz-id-td2",
      "BaseTextLineSpecificationName": "tls-base",
      "StringLengthRange": [ 36, 36 ],
      "OutputResults": 1,
      "ExpectedGroupsCount": 1,
      "ConcatResults": 1,
      "ConcatSeparator": "",
      "SubGroups": [
        {
          "StringRegExPattern": "([ACI][A-Z<][A-Z<]{3}[A-Z<]{31}){(36)}",
          "StringLengthRange": [ 36, 36 ],
          "BaseTextLineSpecificationName": "tls-base"
        },
        {
          "StringRegExPattern": "([A-Z0-9<]{9}[0-9][A-Z<]{3}[0-9]{2}[(01-12)][(01-31)][0-9][MF<][0-9]{2}[(01-12)][(01-31)][0-9][A-Z0-9<]{8}){(36)}",
          "StringLengthRange": [ 36, 36 ],
          "BaseTextLineSpecificationName": "tls-base"
        }
      ]
    },
    {
      "Name": "tls-mrz-id-td1",
      "BaseTextLineSpecificationName": "tls-base",
      "StringLengthRange": [ 30, 30 ],
      "OutputResults": 1,
      "ExpectedGroupsCount": 1,
      "ConcatResults": 1,
      "ConcatSeparator": "",
      "SubGroups": [
        {
          "StringRegExPattern": "([ACI][A-Z<][A-Z<]{3}[A-Z0-9<]{9}[0-9][A-Z0-9<]{15}){(30)}",
          "StringLengthRange": [ 30, 30 ],
          "BaseTextLineSpecificationName": "tls-base"
        },
        {
          "StringRegExPattern": "([0-9]{2}[(01-12)][(01-31)][0-9][MF<][0-9]{2}[(01-12)][(01-31)][0-9][A-Z<]{3}[A-Z0-9<]{11}[0-9]){(30)}",
          "StringLengthRange": [ 30, 30 ],
          "BaseTextLineSpecificationName": "tls-base"
        },
        {
          "StringRegExPattern": "([A-Z<]{30}){(30)}",
          "StringLengthRange": [ 30, 30 ],
          "BaseTextLineSpecificationName": "tls-base"
        }
      ]
    },
    {
      "Name": "tls-base",
      "CharacterModelName": "MRZ",
      "CharHeightRange": [ 5, 1000, 1 ],
      "BinarizationModes": [
        {
          "BlockSizeX": 30,
          "BlockSizeY": 30,
          "Mode": "BM_LOCAL_BLOCK",
          "EnableFillBinaryVacancy": 0,
          "ThresholdCompensation": 15
        }
      ],
      "ConfusableCharactersCorrection": {
        "ConfusableCharacters": [
          [ "0", "O" ],
          [ "1", "I" ],
          [ "5", "S" ]
        ],
        "FontNameArray": [ "OCR_B" ]
      }
    }
  ],
  "LabelRecognizerTaskSettingOptions": [
    {
      "Name": "task-mrz",
      "ConfusableCharactersPath": "ConfusableChars.data",
      "TextLineSpecificationNameArray": [ "tls-mrz-passport", "tls-mrz-visa-td3", "tls-mrz-id-td1", "tls-mrz-id-td2", "tls-mrz-visa-td2" ],
      "SectionImageParameterArray": [
        {
          "Section": "ST_REGION_PREDETECTION",
          "ImageParameterName": "ip-mrz"
        },
        {
          "Section": "ST_TEXT_LINE_LOCALIZATION",
          "ImageParameterName": "ip-mrz"
        },
        {
          "Section": "ST_TEXT_LINE_RECOGNITION",
          "ImageParameterName": "ip-mrz"
        }
      ]
    }
  ],
  "CharacterModelOptions": [
    {
      "DirectoryPath": "",
      "Name": "MRZ"
    }
  ],
  "ImageParameterOptions": [
    {
      "Name": "ip-mrz",
      "TextureDetectionModes": [
        {
          "Mode": "TDM_GENERAL_WIDTH_CONCENTRATION",
          "Sensitivity": 8
        }
      ],
      "BinarizationModes": [
        {
          "EnableFillBinaryVacancy": 0,
          "ThresholdCompensation": 21,
          "Mode": "BM_LOCAL_BLOCK"
        }
      ],
      "TextDetectionMode": {
        "Mode": "TTDM_LINE",
        "CharHeightRange": [ 5, 1000, 1 ],
        "Direction": "HORIZONTAL",
        "Sensitivity": 7
      }
    }
  ],
  "SemanticProcessingOptions": [
    {
      "Name": "sp-mrz",
      "ReferenceObjectFilter": {
        "ReferenceTargetROIDefNameArray": [
          "roi-mrz"
        ]
      },
      "TaskSettingNameArray": [
        "dcp-mrz"
      ]
    }
  ],
  "CodeParserTaskSettingOptions": [
    {
      "Name": "dcp-mrz",
      "CodeSpecifications": [ "MRTD_TD3_PASSPORT", "MRTD_TD2_VISA", "MRTD_TD3_VISA", "MRTD_TD1_ID", "MRTD_TD2_ID" ]
    }
  ]
}
Enter fullscreen mode Exit fullscreen mode

This setup supports various MRZ types, including MRTD_TD3_PASSPORT, MRTD_TD2_VISA, MRTD_TD3_VISA, MRTD_TD1_ID and MRTD_TD2_ID.

Step 3: Initialize Dynamsoft Label Recognizer

In your MainActivity.java file, set up the license key for the SDK as shown below:

protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        if (savedInstanceState == null) {
            LicenseManager.initLicense("LICENSE-KEY", this, (isSuccessful, error) -> {
                if (!isSuccessful) {
                    error.printStackTrace();
                    runOnUiThread(() -> ((TextView) findViewById(R.id.tv_license_error)).setText("License initialization failed: "+error.getMessage()));
                }
            });
        }
        ...
}
Enter fullscreen mode Exit fullscreen mode

This setup initializes the SDK and checks for any licensing errors, ensuring the app is ready for MRZ recognition.

Step 4: Create a Camera Preview for MRZ Recognition

  1. Add a new layout file named activity_scan.xml in the layout folder:

    <?xml version="1.0" encoding="utf-8"?>
<RelativeLayout xmlns:android="http://schemas.android.com/apk/res/android"
    android:layout_width="match_parent"
    android:layout_height="match_parent">

    <com.dynamsoft.dce.CameraView
        android:id="@+id/dce_camera_view"
        android:layout_width="match_parent"
        android:layout_height="match_parent" />

    <TextView
        android:layout_above="@+id/tv_result"
        android:id="@+id/tv_message"
        android:layout_marginHorizontal="24dp"
        android:layout_centerHorizontal="true"
        android:layout_marginBottom="80dp"
        android:textColor="@color/red00"
        android:textSize="16sp"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"/>

    <TextView
        android:layout_alignParentBottom="true"
        android:layout_centerHorizontal="true"
        android:layout_marginBottom="70dp"
        android:id="@+id/tv_result"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content" />
</RelativeLayout>

    Explanation

    • CameraView: Displays the camera preview for capturing MRZ data.
    • TextView: Displays messages such as license information and detection errors.

  2. Inflate the layout and configure the camera preview in the MainActivity.java file:

    @Override
protected void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    setContentView(R.layout.activity_scan);
    PermissionUtil.requestCameraPermission(this);
    LicenseManager.initLicense("LICENSE-KEY",
            this,
            (isSuccess, error) -> {
                if (!isSuccess) {
                    runOnUiThread(() -> {
                        ((TextView)findViewById(R.id.tv_message))
                                .setText("License initialization failed: "+error.getMessage());
                    });
                    error.printStackTrace();
                }
            });
    mCameraView = findViewById(R.id.dce_camera_view);
    mTextResult = findViewById(R.id.tv_result);
    mCamera = new CameraEnhancer(mCameraView, this);

    try {
        mCamera.enableEnhancedFeatures(EnumEnhancerFeatures.EF_FRAME_FILTER);
    } catch (CameraEnhancerException e) {
        throw new RuntimeException(e);
    }

    mRouter = new CaptureVisionRouter(this);
    MultiFrameResultCrossFilter filter = new MultiFrameResultCrossFilter();
    filter.enableResultCrossVerification(EnumCapturedResultItemType.CRIT_TEXT_LINE, true);
    mRouter.addResultFilter(filter);
    try {
        mRouter.initSettingsFromFile("MRZScanner.json");
        mRouter.setInput(mCamera);
    } catch (CaptureVisionRouterException e) {
        throw new RuntimeException(e);
    }
}

@Override
protected void onResume() {
    super.onResume();
    try {
        mCamera.open();
    } catch (CameraEnhancerException e) {
        e.printStackTrace();
    }
    mRouter.startCapturing("ReadMRZ", new CompletionListener() {
        @Override
        public void onSuccess() {
        }

        @Override
        public void onFailure(int errorCode, String errorString) {
            runOnUiThread(() -> showDialog("Error", String.format(Locale.getDefault(),
                    "ErrorCode: %d %nErrorMessage: %s", errorCode, errorString)));
        }
    });
}

@Override
protected void onPause() {
    super.onPause();
    succeed = false;
    try {
        mCamera.close();

    } catch (CameraEnhancerException e) {
        e.printStackTrace();
    }
    mRouter.stopCapturing();
}

@Override
protected void onStop() {
    mCameraView.getDrawingLayer(DrawingLayer.DLR_LAYER_ID).clearDrawingItems();
    super.onStop();
}

  3. Register the callbacks for handling recognized MRZ data:

    mRouter.addResultReceiver(new CapturedResultReceiver() {
        @Override
        // Implement this method to receive RecognizedTextLinesResult.
        public void onRecognizedTextLinesReceived(@NonNull RecognizedTextLinesResult result) {
            onLabelTextReceived(result);
        }

        @Override
        public void onParsedResultsReceived(@NonNull ParsedResult result) {
            if (!succeed) {
                onParsedResultReceived(result);
            }
        }
    });

    The ParsedResult object contains the MRZ data extracted from the captured image, formatted as key-value pairs.

    MRZ data

  4. Bind the parsed MRZ data to an Intent and start a new activity to display the results:

    private void onParsedResultReceived(ParsedResult result) {
    if (result.getItems() == null) {
        return;
    }
    if (result.getItems().length == 0) {
        runOnUiThread(() -> {
            if (!mText.isEmpty()) {
                String errorMsg = "error: Failed to parse the content. The MRZ text is " + mText;
                mTextResult.setText(errorMsg);
            }
        });
    } else {
        HashMap<String, String> labelMap = assembleMap(result.getItems()[0]);
        if (labelMap != null && !labelMap.isEmpty()) {
            succeed = true;
            Intent intent = new Intent(this, ResultActivity.class);
            intent.putExtra("labelMap", labelMap);
            startActivity(intent);
            runOnUiThread(() -> {
                mTextResult.setText("");
            });

        } else {
            runOnUiThread(() -> {
                if (!mText.isEmpty()) {
                    String errorMsg = "error: Failed to parse the content. The MRZ text is " + mText;
                    mTextResult.setText(errorMsg);
                }
            });
        }

    }
}

Step 5: Display the MRZ Data

  1. Create a layout file named activity_result.xml in the layout folder:

    <?xml version="1.0" encoding="utf-8"?>
<RelativeLayout xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:app="http://schemas.android.com/apk/res-auto"
    xmlns:tools="http://schemas.android.com/tools"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    android:background="@color/dy_black_3B"
    tools:context=".ResultActivity">

    <RelativeLayout
        android:id="@+id/rl_title"
        android:layout_width="match_parent"
        android:layout_height="48dp"
        android:background="@color/dy_black_2B">

        <ImageView
            android:id="@+id/iv_back"
            android:layout_width="wrap_content"
            android:layout_height="wrap_content"
            android:layout_centerVertical="true"
            android:layout_marginStart="16dp"
            android:background="@color/transparent"
            android:src="@drawable/ic_arrow_left" />

        <TextView
            android:layout_width="wrap_content"
            android:layout_height="wrap_content"
            android:layout_centerVertical="true"
            android:layout_toEndOf="@id/iv_back"
            android:text="Back"
            android:textColor="@color/white"
            android:textSize="17sp" />

        <TextView
            android:layout_width="wrap_content"
            android:layout_height="wrap_content"
            android:layout_centerInParent="true"
            android:layout_gravity="center_horizontal"
            android:text="Result"
            android:textColor="@color/white"
            android:textSize="20sp"
            android:textStyle="bold" />

    </RelativeLayout>

    <ScrollView
        android:layout_below="@id/rl_title"
        android:layout_width="match_parent"
        android:layout_height="wrap_content">

        <LinearLayout
            android:id="@+id/ll_content"
            android:layout_width="match_parent"
            android:layout_height="wrap_content"
            android:orientation="vertical"
            android:padding="20dp">
        </LinearLayout>
    </ScrollView>
</RelativeLayout>

  2. Inflate the layout and display the MRZ data in the ResultActivity.java file:

    package com.dynamsoft.mrzscanner;

import androidx.annotation.NonNull;
import androidx.appcompat.app.AppCompatActivity;
import androidx.core.content.ContextCompat;

import android.graphics.Color;
import android.os.Bundle;
import android.view.View;
import android.view.ViewGroup;
import android.widget.LinearLayout;
import android.widget.TextView;

import java.util.HashMap;

public class ResultActivity extends AppCompatActivity {
    private LinearLayout content;

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_result);
        findViewById(R.id.iv_back).setOnClickListener((v) -> {
            finish();
        });
        content = findViewById(R.id.ll_content);
        HashMap<String, String> properties = (HashMap<String, String>) getIntent().
                getSerializableExtra("labelMap");
        if (properties != null) {
            fillViews(properties);
        }

    }

    @NonNull
    private View childView(String label, String labelText) {
        LinearLayout layout = new LinearLayout(this);
        LinearLayout.LayoutParams params = new LinearLayout.LayoutParams(ViewGroup.LayoutParams.MATCH_PARENT,
                ViewGroup.LayoutParams.WRAP_CONTENT);
        params.setMargins(0, 30, 0, 0);
        layout.setLayoutParams(params);
        layout.setOrientation(LinearLayout.VERTICAL);
        TextView labelView = new TextView(this);
        labelView.setPadding(0, 30, 0, 0);
        labelView.setTextColor(ContextCompat.getColor(this, R.color.dy_grey_AA));
        labelView.setTextSize(16);
        labelView.setText(label);
        TextView textView = new TextView(this);
        textView.setTextSize(16);
        textView.setTextColor(Color.WHITE);
        textView.setText(labelText);
        layout.addView(labelView);
        layout.addView(textView);
        return layout;
    }

    private void fillViews(HashMap<String, String> properties) {
        content.addView(childView("Document Type:", properties.get("Document Type")));
        content.addView(childView("Document Number:", properties.get("Document Number")));
        content.addView(childView("Name:", properties.get("Name")));
        content.addView(childView("Issuing State:", properties.get("Issuing State")));
        content.addView(childView("Nationality:", properties.get("Nationality")));
        content.addView(childView("Date of Birth(YYYY-MM-DD):", properties.get("Date of Birth(YYYY-MM-DD)")));
        content.addView(childView("Sex:", Character.toUpperCase(properties.get("Sex").charAt(0)) + properties.get("Sex").substring(1)));
        content.addView(childView("Date of Expiry(YYYY-MM-DD):", properties.get("Date of Expiry(YYYY-MM-DD)")));
    }
}

    Android OCR recognition for passport mrz

Source Code

https://github.com/yushulx/android-camera-barcode-mrz-document-scanner/tree/main/examples/10.x/mrz

Image
How to Install a UPVC Door Panels Cat Flap
Image
Zanurz się w ciszy: Innowacyjne sposoby na wyciszenie mieszkań
Image
10 . Pinterest Account To Be Following About Electric Patio Heater Free Standing
Image
TypeScript (and JSDoc) vs JavaScript

typescript, javascript, node
Image
The Most Underrated Companies To Follow In The Repairing Bifold Doors Industry
Image
Ten Things You've Learned In Kindergarden That Will Aid You In Obtaining Sectional With Chaise
Image
Cat Flap Cover For Winter
Image
How To Determine If You're In The Right Position To Go After Double Chaise Sofa
Image
Cat Flap Installation Cost Near Me
Image
Essentials Hoodie The Perfect Blend of Comfort and Style

fashion, webdev, beginners, react
Image
5 Bi Fold Repairs Lessons From The Pros
Image
VRPG day 1
Image
The Secret to Fast Typing on the Game Boy: Here’s How I Did It!

algorithms, c, gameboy, gbdk
Image
Ten Reasons To Hate People Who Can't Be Disproved Repairing Bifold Doors
Image
10 Basics Regarding Bi Fold Door Repair You Didn't Learn At School
Image
10 Things You've Learned About Preschool That Will Help You With Replace Bifold Doors
Image
20 Trailblazers Leading The Way In Bifold Door Repair
Image
My First Time Participating in Hacktoberfest: A Journey into Open Source

devchallenge, hacktoberfestchallenge, hacktoberfest, opensource
Image
True color with an explainer on bits and bytes

beginners, programming
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Terabox Video Player