How to Build Desktop .NET MRZ Scanner for Passport and ID Card on Windows and Linux

Xiao Ling - Oct 12 '22 - - Dev Community

This article aims to help C# developers to build desktop .NET MRZ (Machine Readable Zone) scanner applications with Dynamsoft C++ Label Recognizer SDK. We firstly build a .NET class library for detecting MRZ zone and extracting MRZ text from passport, ID card, and travel documents. Then we demonstrate how to build console and GUI applications with the .NET class library on Windows and Linux.

NuGet Package

https://www.nuget.org/packages/MrzScannerSDK

License Key

Get a 30-day FREE trial license.

Building .NET MRZ Scanner Library

In this section, we will create a .NET class library project containing MRZ detection and extraction functions.

NuGet Package with .NET DLL, C++ Runtime Libraries and MRZ Detection model

We run the command dotnet new classlib -o MrzScannerSDK to create a .NET class library project, and then open the MrzScannerSDK.csproj file to configure the build.

To generate the NuGet package automatically when running dotnet build, add the following line to the MrzScannerSDK.csproj file:

<GeneratePackageOnBuild>true</GeneratePackageOnBuild>
Enter fullscreen mode Exit fullscreen mode

The C++ OCR SDK contains some C++ runtime libraries (*.dll, *.so) and MRZ model files. We need to copy them to the output directory and package them into the NuGet package.

<ItemGroup>
<None CopyToOutputDirectory="Always" Link="\%(Filename)%(Extension)" Include="lib/win/DynamicPdfx64.dll" Pack="true" PackagePath="runtimes/win-x64/native/DynamicPdfx64.dll" />
<None CopyToOutputDirectory="Always" Link="\%(Filename)%(Extension)" Include="lib/win/DynamsoftLabelRecognizerx64.dll" Pack="true" PackagePath="runtimes/win-x64/native/DynamsoftLabelRecognizerx64.dll"  />
<None CopyToOutputDirectory="Always" Link="\%(Filename)%(Extension)" Include="lib/win/DynamsoftLicenseClientx64.dll" Pack="true" PackagePath="runtimes/win-x64/native/DynamsoftLicenseClientx64.dll" />
<None CopyToOutputDirectory="Always" Link="\%(Filename)%(Extension)" Include="lib/win/vcomp140.dll" Pack="true" PackagePath="runtimes/win-x64/native/vcomp140.dll" />

<None CopyToOutputDirectory="Always" Link="\%(Filename)%(Extension)" Include="lib/linux/libDynamicPdf.so" Pack="true" PackagePath="runtimes/linux-x64/native/libDynamicPdf.so" />
<None CopyToOutputDirectory="Always" Link="\%(Filename)%(Extension)" Include="lib/linux/libDynamsoftLabelRecognizer.so" Pack="true" PackagePath="runtimes/linux-x64/native/libDynamsoftLabelRecognizer.so" />
<None CopyToOutputDirectory="Always" Link="\%(Filename)%(Extension)" Include="lib/linux/libDynamsoftLicenseClient.so" Pack="true" PackagePath="runtimes/linux-x64/native/libDynamsoftLicenseClient.so" />
</ItemGroup>

<ItemGroup>
<None Update="README.md">
    <Pack>True</Pack>
    <PackagePath>\</PackagePath>
</None>
<None Update="MRZ.json">
    <CopyToOutputDirectory>Always</CopyToOutputDirectory>
    <Pack>True</Pack>
    <PackagePath>\</PackagePath>
</None>
<None Update="model/**/*.*">
    <CopyToOutputDirectory>Always</CopyToOutputDirectory>
    <Pack>True</Pack>
    <PackagePath>\</PackagePath>
</None>
</ItemGroup>
Enter fullscreen mode Exit fullscreen mode

You can try to build the project. The structure of the NuGet package should be like this:

nuget package with *.dll, *.so and mrz model

To use the model, we need to modify the parameter DirectoryPath dynamically in the MRZ.json file. The detailed implementation will be introduced in the next paragraph.

Encapsulating C++ MRZ Detection Functions in .NET Class Library

We use DllImport to import the primary C++ MRZ detection functions. The DllImport attribute specifies the name of the DLL, the name of the function, and the calling convention.

  • DLR_InitLicense: set the license key

    [DllImport("DynamsoftLabelRecognizer")]
static extern int DLR_InitLicense(string license, [Out] byte[] errorMsg, int errorMsgSize);

  • DLR_CreateInstance: create an instance of the label recognizer

    [DllImport("DynamsoftLabelRecognizer")]
static extern IntPtr DLR_CreateInstance();

  • DLR_RecognizeByFile: recognize the MRZ zone from the image file

    [DllImport("DynamsoftLabelRecognizer")]
static extern int DLR_RecognizeByFile(IntPtr handler, string sourceFilePath, string templateName);

  • DLR_RecognizeByBuffer: recognize the MRZ zone from the image buffer

    [DllImport("DynamsoftLabelRecognizer")]
static extern int DLR_RecognizeByBuffer(IntPtr handler, IntPtr sourceImage, string templateName);

  • DLR_AppendSettingsFromFile: load the MRZ detection model

    [DllImport("DynamsoftLabelRecognizer")]
static extern int DLR_AppendSettingsFromFile(IntPtr handler, string filename, [Out] byte[] errorMsg, int errorMsgSize);

  • DLR_GetAllResults: get the MRZ text

    [DllImport("DynamsoftLabelRecognizer")]
static extern int DLR_GetAllResults(IntPtr hBarcode, ref IntPtr pDLR_ResultArray);

  • DLR_FreeResults: free the memory of the MRZ text

    [DllImport("DynamsoftLabelRecognizer")]
static extern int DLR_FreeResults(ref IntPtr pDLR_ResultArray);

Meanwhile, we use StructLayout to define the C++ struct in C#. For example:

[StructLayout(LayoutKind.Sequential, Pack = 1)]
internal struct DLR_LineResult
{
    public string lineSpecificationName;

    public string text;

    public string characterModelName;

    public Quadrilateral location;

    public int confidence;

    public int characterResultsCount;

    public IntPtr characterResults;

    [MarshalAs(UnmanagedType.ByValArray, SizeConst = 64)]
    public char[] reserved;
}

[StructLayout(LayoutKind.Sequential, Pack = 1)]
internal struct DLR_Result
{
    public string referenceRegionName;

    public string textAreaName;

    public Quadrilateral location;

    public int confidence;

    public int lineResultsCount;

    public IntPtr lineResults;

    public int pageNumber;

    [MarshalAs(UnmanagedType.ByValArray, SizeConst = 60)]
    public char[] reserved;
}
Enter fullscreen mode Exit fullscreen mode

The next step is to encapsulate the C++ functions in C#. For example, a function for loading the MRZ detection model:


public int LoadModel()
{
    if (handler == IntPtr.Zero) return -1;

    string dir = Directory.GetCurrentDirectory();
    string[] files = Directory.GetDirectories(dir, "model", SearchOption.AllDirectories);
    string modelPath = files[0];
    string config = Path.Join(modelPath.Split("model")[0], "MRZ.json");

    string contents = File.ReadAllText(config);

    JsonNode configNode = JsonNode.Parse(contents)!;

    if ((string)(configNode!["CharacterModelArray"]![0]!["DirectoryPath"]!) == "model")
    {
        configNode["CharacterModelArray"]![0]!["DirectoryPath"] = modelPath;

        var options = new JsonSerializerOptions { WriteIndented = true };

        contents = configNode.ToJsonString(options);

        File.WriteAllText(config, contents);
    }

    byte[] errorMsg = new byte[512];
    int ret = DLR_AppendSettingsFromFile(handler, config, errorMsg, 512);
    return ret;
}
Enter fullscreen mode Exit fullscreen mode

As we mentioned above, the MRZ.json file is a JSON-formatted configuration file. The DirectoryPath parameter specifies the path of the MRZ detection model. We need to modify the DirectoryPath parameter dynamically to make it point to the absolute path of the MRZ detection model. JsonNode is a class used to deserialize string to JSON object and serialize JSON object to string. It simplifies the process of modifying the DirectoryPath parameter.

According to the C++ methods DLR_RecognizeByFile() and DLR_RecognizeByBuffer(), we create two corresponding C# methods DetectFile() and DetectBuffer(). The DetectBuffer() method is not as easy as DetectFile() because it requires the image buffer to be converted to an ImageData object before calling the DLR_RecognizeByBuffer() function:

public Result[]? DetectBuffer(IntPtr pBufferBytes, int width, int height, int stride, ImagePixelFormat format)
{
    if (handler == IntPtr.Zero) return null;

    IntPtr pResultArray = IntPtr.Zero;

    ImageData imageData = new ImageData();
    imageData.width = width;
    imageData.height = height;
    imageData.stride = stride;
    imageData.format = format;
    imageData.bytesLength = stride * height;
    imageData.bytes = pBufferBytes;

    IntPtr pimageData = Marshal.AllocHGlobal(Marshal.SizeOf(imageData));
    Marshal.StructureToPtr(imageData, pimageData, false);
    int ret = DLR_RecognizeByBuffer(handler, pimageData, "locr");
    Marshal.FreeHGlobal(pimageData);

    return GetResults();
}
Enter fullscreen mode Exit fullscreen mode

The ImageData object is defined as follows:

[StructLayout(LayoutKind.Sequential, Pack = 1)]
internal struct ImageData
{
    public int bytesLength;

    public IntPtr bytes;

    public int width;

    public int height;

    public int stride;

    public ImagePixelFormat format;
}
Enter fullscreen mode Exit fullscreen mode

After calling MRZ detection functions, we can get the MRZ text by calling the GetResults() method:

private Result[]? GetResults()
{
    IntPtr pDLR_ResultArray = IntPtr.Zero;
    DLR_GetAllResults(handler, ref pDLR_ResultArray);

    if (pDLR_ResultArray != IntPtr.Zero)
    {
        List<Result> resultArray = new List<Result>();
        DLR_ResultArray? results = (DLR_ResultArray?)Marshal.PtrToStructure(pDLR_ResultArray, typeof(DLR_ResultArray));
        if (results != null)
        {
            int count = results.Value.resultsCount;

            if (count > 0)
            {
                IntPtr[] mrzResults = new IntPtr[count];
                Marshal.Copy(results.Value.results, mrzResults, 0, count);

                for (int i = 0; i < count; i++)
                {
                    DLR_Result result = (DLR_Result)Marshal.PtrToStructure(mrzResults[i], typeof(DLR_Result))!;
                    int lineResultsCount = result.lineResultsCount;
                    IntPtr lineResults = result.lineResults;
                    IntPtr[] lines = new IntPtr[lineResultsCount];
                    Marshal.Copy(lineResults, lines, 0, lineResultsCount);

                    for (int j = 0; j < lineResultsCount; j++)
                    {
                        Result mrzResult = new Result();
                        DLR_LineResult lineResult = (DLR_LineResult)Marshal.PtrToStructure(lines[j], typeof(DLR_LineResult))!;
                        mrzResult.Text = lineResult.text;
                        mrzResult.Confidence = lineResult.confidence;
                        DM_Point[] points = lineResult.location.points;
                        mrzResult.Points = new int[8];
                        for (int k = 0; k < 4; k++)
                        {
                            mrzResult.Points[k * 2] = points[k].coordinate[0];
                            mrzResult.Points[k * 2 + 1] = points[k].coordinate[1];
                        }

                        resultArray.Add(mrzResult);
                    }
                }
            }
        }

        DLR_FreeResults(ref pDLR_ResultArray);

        return resultArray.ToArray();
    }

    return null;
}
Enter fullscreen mode Exit fullscreen mode

One more step is to extract the information from the MRZ text. We write the parsing logic based on MRZ specification. The Regex class is helpful for matching and finding the information we need.

public static JsonNode? Parse(string[] lines)
{
    JsonNode mrzInfo = new JsonObject();

    if (lines.Length == 0)
    {
        return null;
    }

    if (lines.Length == 2)
    {
        string line1 = lines[0];
        string line2 = lines[1];

        var type = line1.Substring(0, 1);
        if (!new Regex(@"[I|P|V]").IsMatch(type)) return null;
        if (type == "P")
        {
            mrzInfo["type"] = "PASSPORT (TD-3)";
        }
        else if (type == "V")
        {
            if (line1.Length == 44)
            {
                mrzInfo["type"] = "VISA (MRV-A)";
            }
            else if (line1.Length == 36)
            {
                mrzInfo["type"] = "VISA (MRV-B)";
            }
        }
        else if (type == "I")
        {
            mrzInfo["type"] = "ID CARD (TD-2)";
        }

        // Get issuing State infomation
        var nation = line1.Substring(2, 5);
        if (new Regex(@"[0-9]").IsMatch(nation)) return null;
        if (nation[nation.Length - 1] == '<') {
            nation = nation.Substring(0, 2);
        }
        mrzInfo["nationality"] = nation;
        ...
    }
    else if (lines.Length == 3)
    {
        string line1 = lines[0];
        string line2 = lines[1];
        string line3 = lines[2];
        var type = line1.Substring(0, 1);
        if (!new Regex(@"[I|P|V]").IsMatch(type)) return null;
        mrzInfo["type"] = "ID CARD (TD-1)";
        // Get nationality infomation
        var nation = line2.Substring(15, 3);
        if (new Regex(@"[0-9]").IsMatch(nation)) return null;
        nation = nation.Replace("<", "");
        mrzInfo["nationality"] = nation;
        // Get surname information
        ...
    }

    return mrzInfo;
}
Enter fullscreen mode Exit fullscreen mode

As all C# methods are ready, we can build the package to *.nupkg file.

dotnet build --configuration Release
Enter fullscreen mode Exit fullscreen mode

The package can be installed in Windows and Linux .NET project.

Desktop Console and GUI Applications for Scanning MRZ

The following paragraphs will introduce how to build a desktop console and GUI applications for scanning MRZ.

Desktop Console Application

  1. Create a new .NET console project in terminal.

    dotnet new console -o app

  2. Set the license key acquired from Dynamsoft customer portal.

    MrzScanner.InitLicense("DLS2eyJoYW5kc2hha2VDb2RlIjoiMjAwMDAxLTE2NDk4Mjk3OTI2MzUiLCJvcmdhbml6YXRpb25JRCI6IjIwMDAwMSIsInNlc3Npb25QYXNzd29yZCI6IndTcGR6Vm05WDJrcEQ5YUoifQ==");

  3. Create an instance of MRZ scanner.

    MrzScanner scanner = MrzScanner.Create();

  4. Load the MRZ detection model.

    scanner.LoadModel();

  5. Detect MRZ from an image file.

    MrzScanner.Result[]? results = scanner.DetectFile("<image-file>");

  6. Output the results.

    if (results != null)
{
    foreach (MrzScanner.Result result in results)
    {
        Console.WriteLine(result.Text);
        Console.WriteLine(result.Points[0] + ", " +result.Points[1] + ", " + result.Points[2] + ", " + result.Points[3] + ", " + result.Points[4] + ", " + result.Points[5] + ", " + result.Points[6] + ", " + result.Points[7]);
    }
}

WinForms Application

Using WinForms can create a fancy GUI application. Here is the UI designed with Visual Studio Toolbox.

WinForm UI design

  • The menu allows users to enter a valid license key.
  • The status bar shows the license activation status.
  • The left picture box shows the original image and the right picture box shows the image with detected MRZ contours.
  • The Load File button allows users to load an image file.
  • The Camera Scan button allows users to scan MRZ from a camera.
  • The first rich text box shows the parsed MRZ information and the second rich text box shows the image file list.

OpenCvSharp4 is required for camera streaming and image decoding. Thus, add the following NuGet packages to the project.

<ItemGroup>
<PackageReference Include="OpenCvSharp4" Version="4.6.0.20220608" />
</ItemGroup>
<ItemGroup>
<PackageReference Include="OpenCvSharp4.Extensions" Version="4.5.5.20211231" />
<PackageReference Include="OpenCvSharp4.runtime.win" Version="4.6.0.20220608" />
</ItemGroup>
Enter fullscreen mode Exit fullscreen mode

To detect MRZ from an image file, we use OpenFileDialog to load images:

private void button1_Click(object sender, EventArgs e)
{
    StopScan();
    using (OpenFileDialog dlg = new OpenFileDialog())
    {
        dlg.Title = "Open Image";
        dlg.Filter = "Image files (*.bmp, *.jpg, *.png) | *.bmp; *.jpg; *.png";

        if (dlg.ShowDialog() == DialogResult.OK)
        {
            listBox1.Items.Add(dlg.FileName);
            MrzDetection(dlg.FileName);
        }
    }
}
Enter fullscreen mode Exit fullscreen mode

Use OpenCV to decode the image to Mat and then use MrzScanner to detect MRZ.

private void MrzDetection(string filename)
{
    try
    {
        _mat = Cv2.ImRead(filename, ImreadModes.Color);
        Mat copy = new Mat(_mat.Rows, _mat.Cols, MatType.CV_8UC3);
        _mat.CopyTo(copy);
        pictureBox1.Image = BitmapConverter.ToBitmap(copy);
        pictureBox2.Image = DecodeMat(copy);
    }
    catch (Exception ex)
    {
        MessageBox.Show(ex.Message);
    }
}

private Bitmap DecodeMat(Mat mat)
{
    _results = scanner.DetectBuffer(mat.Data, mat.Cols, mat.Rows, (int)mat.Step(), MrzScanner.ImagePixelFormat.IPF_RGB_888);
    if (_results != null)
    {
        string[] lines = new string[_results.Length];
        var index = 0;
        foreach (Result result in _results)
        {
            lines[index++] = result.Text;
            richTextBox1.Text += result.Text + Environment.NewLine;
            if (result.Points != null)
            {
                Point[] points = new Point[4];
                for (int i = 0; i < 4; i++)
                {
                    points[i] = new Point(result.Points[i * 2], result.Points[i * 2 + 1]);
                }
                Cv2.DrawContours(mat, new Point[][] { points }, 0, Scalar.Red, 2);
            }
        }

        JsonNode? info = Parse(lines);
        if (info != null) richTextBox1.Text = info.ToString();
    }

    Bitmap bitmap = BitmapConverter.ToBitmap(mat);
    return bitmap;
}
Enter fullscreen mode Exit fullscreen mode

To detect MRZ from a camera, we start a thread to keep capturing images from the camera and then use MrzScanner to detect MRZ.

private void button2_Click(object sender, EventArgs e)
{
    if (!capture.IsOpened())
    {
        MessageBox.Show("Failed to open video stream or file");
        return;
    }

    if (button2.Text == "Camera Scan")
    {
        StartScan();
    }
    else
    {
        StopScan();
    }
}

private void StartScan()
{
    button2.Text = "Stop";
    isCapturing = true;
    thread = new Thread(new ThreadStart(FrameCallback));
    thread.Start();
}

private void StopScan()
{
    button2.Text = "Camera Scan";
    isCapturing = false;
    if (thread != null) thread.Join();
}

private void FrameCallback()
{
    while (isCapturing)
    {
        capture.Read(_mat);
        Mat copy = new Mat(_mat.Rows, _mat.Cols, MatType.CV_8UC3);
        _mat.CopyTo(copy);
        pictureBox1.Image = DecodeMat(copy);
    }
}
Enter fullscreen mode Exit fullscreen mode

Finally, run the desktop .NET MRZ scanner:

dotnet run
Enter fullscreen mode Exit fullscreen mode

Winform .NET MRZ scanner

Source Code

https://github.com/yushulx/dotnet-mrz-sdk

Image
Waktu Yang Sesuai Buat Stop Main Judi Online
Image
Beyond the Beat: The Essential Role of Silence in Music by Daniel Siegel Loanso
Image
Polser
Image
The Ultimate Guide to Physical Therapy Services in Las Vegas

ptlasvegas
Image
Situs Judi Online Terpilih Dengan Payment Sangat Ringan Serta Terjaga
Image
How Asbestos Lawsuit Settlement Amounts Has Transformed My Life The Better
Image
permainan slots online microgaming yang penting dicoba
Image
11 Ways To Completely Sabotage Your Asbestos Exposure Compensation
Image
"5 Reasons to Upgrade Your Kitchen with a Premium End Grain Chopping Board"
Image
NetMeds Customer Care Helpline Number -(+.9836891069))//Call ((+91-/7735273071)/// call now

webdev, javascript, beginners, programming
Image
10 Things You Learned In Kindergarden That Will Help You Get Mesothelioma Asbestos Claim
Image
How is AI Being Used in Sports Analytics Today?

ai, analytics
Image
Teknik Taruhan Judi Online Besar Kecil Dadu
Image
PharmEasy Customer Care Helpline Number -(+.9836891069))//Call ((+91-/7735273071)/// call now

webdev, javascript, beginners, programming
Image
Data Types C++
Image
DumpsBoss DP-203 Dumps PDF Your Guide to Lifelong Learning
Image
Database Migration Service in AWS

devops, technology, software, trending
Image
TrueMeds Customer Care Helpline Number -(+.9836891069))//Call ((+91-/7735273071)/// call now

webdev, javascript, beginners, programming
Image
Buy Verified Cash App Accounts
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Terabox Video Player