How to Build a Windows Desktop App for Document, Barcode, and MRZ Detection with C# and .NET WinForms

Xiao Ling - Jul 31 '23 - - Dev Community

Dynamsoft's Capture Vision SDKs consist of the Dynamsoft Document Normalizer, Dynamsoft Barcode Reader, and Dynamsoft Label Recognizer. All of these SDKs are cross-platform, supporting Windows, Linux, Android, iOS and web. Each vision SDK can be used independently or in combination with others. In this article, we'll show you how to build a Windows desktop app that integrates Dynamsoft's vision APIs for document rectification, barcode scanning, and MRZ detection using C# and .NET WinForms.

Development Environment

  • Visual Studio 2022
  • Visual Studio Code
  • .NET 6.0 SDK or later

NuGet Packages

OpenCV

We use OpenCV to access the camera and display the video stream.

Dynamsoft Vision SDKs

  • BarcodeQRCodeSDK: Used to detect 1D and 2D barcodes.
  • DocumentScannerSDK: Used to detect document edges and rectify documents.
  • MrzScannerSDK: Used to detect MRZ (Machine Readable Zone) on passport, Visa, ID card and other travel documents.

To utilize the Dynamsoft Vision SDKs, you need a valid license. A free trial license is available from the Dynamsoft Customer Portal. You have the option to request individual licenses for each of the three SDKs, or a unified license that covers all SDKs.

Create a Windows Forms Project

In Visual Studio, create a new Windows Forms project for .NET, not for the .NET Framework.

Create a WinForms project

In the Solution Explorer, right-click the project and select Manage NuGet Packages. Search for the above NuGet packages and install them.

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <OutputType>WinExe</OutputType>
    <TargetFramework>net6.0-windows</TargetFramework>
    <Nullable>enable</Nullable>
    <UseWindowsForms>true</UseWindowsForms>
    <ImplicitUsings>enable</ImplicitUsings>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="BarcodeQRCodeSDK " Version="2.3.4" />
    <PackageReference Include="DocumentScannerSDK " Version="1.1.0" />
    <PackageReference Include="MrzScannerSDK" Version="1.2.0" />
    <PackageReference Include="OpenCvSharp4" Version="4.6.0.20220608" />
  </ItemGroup>
  <ItemGroup>
    <PackageReference Include="OpenCvSharp4.Extensions" Version="4.5.5.20211231" />
    <PackageReference Include="OpenCvSharp4.runtime.win" Version="4.6.0.20220608" />
  </ItemGroup>
</Project>
Enter fullscreen mode Exit fullscreen mode

Windows Form UI Design

The UI contains some basic controls, such as ToolStripStatusLabel, ToolStripMenuItem, PictureBox, Button, CheckBox, and RichTextBox.

Windows capture vision UI design

  • There are two PictureBox controls. One is used to display the input image or video stream. The other is used to display the captured image with the detected results.
  • The ToolStripMenuItem allows you to enter the license key and SDK-relevant templates.
  • The CheckBox is used to enable or disable the corresponding SDK. You will see the corresponding vision effect in the PictureBox control.
  • The Button provides the operations of loading an image, toggling the camera, as well as saving the captured image.
  • The ToolStripStatusLabel makes you know the status of the SDKs.
  • The RichTextBox displays the detected results.

How to Initialize Dynamsoft Vision SDKs

Press F7 to view the code behind the form. Within the constructor, initialize the SDKs and set the license keys.

using Dynamsoft;
using static Dynamsoft.MrzScanner;
using static Dynamsoft.DocumentScanner;
using static Dynamsoft.BarcodeQRCodeReader;

using MrzResult = Dynamsoft.MrzScanner.Result;
using DocResult = Dynamsoft.DocumentScanner.Result;
using BarcodeResult = Dynamsoft.BarcodeQRCodeReader.Result;

public Form1()
{
    InitializeComponent();
    FormClosing += new FormClosingEventHandler(Form1_Closing);
    string license = "DLS2eyJoYW5kc2hha2VDb2RlIjoiMjAwMDAxLTE2NDk4Mjk3OTI2MzUiLCJvcmdhbml6YXRpb25JRCI6IjIwMDAwMSIsInNlc3Npb25QYXNzd29yZCI6IndTcGR6Vm05WDJrcEQ5YUoifQ==";

    ActivateLicense(license);

    // Initialize camera
    capture = new VideoCapture(0);
    isCapturing = false;

    // Initialize MRZ scanner
    mrzScanner = MrzScanner.Create();
    mrzScanner.LoadModel();

    // Initialize document scanner
    documentScanner = DocumentScanner.Create();
    documentScanner.SetParameters(DocumentScanner.Templates.color);

    // Initialize barcode scanner
    barcodeScanner = BarcodeQRCodeReader.Create();
}

private void ActivateLicense(string license)
{
    int ret = MrzScanner.InitLicense(license);
    ret = DocumentScanner.InitLicense(license);
    BarcodeQRCodeReader.InitLicense(license);
    if (ret != 0)
    {
        toolStripStatusLabel1.Text = "License is invalid.";
    }
    else
    {
        toolStripStatusLabel1.Text = "License is activated successfully.";
    }
}
Enter fullscreen mode Exit fullscreen mode

The MRZ model can be found in the NuGet package.

MRZ model

While developing the .NET application, the model will be automatically sourced from the NuGet package path. However, for the distribution of the application, it is necessary to copy the model to the directory where the executable file resides and explicitly specify the path to the model.

string? assemblyPath = System.IO.Path.GetDirectoryName(
        System.Reflection.Assembly.GetExecutingAssembly().Location
    );

string modelPath = assemblyPath == null ? "" : Path.Join(assemblyPath, "model");

mrzScanner.LoadModel(modelPath);
Enter fullscreen mode Exit fullscreen mode

Menu Items and Input Dialogs

When you click on the menu item, an input box will appear, allowing you to enter the license key or template string. This box can be created by customizing a form:

public static string InputBox(string title, string promptText, string value)
{
    Form form = new Form();
    TextBox textBox = new TextBox();
    Button buttonOk = new Button();
    Button buttonCancel = new Button();

    form.Text = title;
    textBox.Text = value;

    buttonOk.Text = "OK";
    buttonCancel.Text = "Cancel";
    buttonOk.DialogResult = DialogResult.OK;
    buttonCancel.DialogResult = DialogResult.Cancel;

    textBox.SetBounds(12, 36, 372, 20);
    buttonOk.SetBounds(60, 72, 80, 30);
    buttonCancel.SetBounds(260, 72, 80, 30);

    form.ClientSize = new System.Drawing.Size(400, 120);
    form.Controls.AddRange(new Control[] { textBox, buttonOk, buttonCancel });
    form.FormBorderStyle = FormBorderStyle.FixedDialog;
    form.StartPosition = FormStartPosition.CenterScreen;
    form.MinimizeBox = false;
    form.MaximizeBox = false;
    form.AcceptButton = buttonOk;
    form.CancelButton = buttonCancel;

    DialogResult dialogResult = form.ShowDialog();
    return textBox.Text;
}
Enter fullscreen mode Exit fullscreen mode

Input box

The click event handler of the menu item is as follows:

enterLicenseKeyToolStripMenuItem.Click += enterLicenseKeyToolStripMenuItem_Click;
dDNToolStripMenuItem.Click += dDNToolStripMenuItem_Click;
dBRToolStripMenuItem.Click += dBRToolStripMenuItem_Click;

private void enterLicenseKeyToolStripMenuItem_Click(object sender, EventArgs e)
{
    string license = InputBox("Enter License Key", "", "");
    if (license != null && license != "")
    {
        ActivateLicense(license);
    }
}

private void dDNToolStripMenuItem_Click(object sender, EventArgs e)
{
    string template = InputBox("Set DDN Template", "", "");
    if (template != null && template != "")
    {
        documentScanner.SetParameters(template);
    }
}

private void dBRToolStripMenuItem_Click(object sender, EventArgs e)
{
    string template = InputBox("Set DBR Template", "", "");
    if (template != null && template != "")
    {
        barcodeScanner.SetParameters(template);
    }
}
Enter fullscreen mode Exit fullscreen mode

Load Image Files from the Local Disk

To load an image file from the local disk, you can use the OpenFileDialog class. The ListBox control can be used to store the history of loaded images.

private void buttonFile_Click(object sender, EventArgs e)
{
    StopScan();
    using (OpenFileDialog dlg = new OpenFileDialog())
    {
        dlg.Title = "Open Image";
        dlg.Filter = "Image files (*.bmp, *.jpg, *.png) | *.bmp; *.jpg; *.png";

        if (dlg.ShowDialog() == DialogResult.OK)
        {
            listBox1.Items.Add(dlg.FileName);
        }
    }
}
Enter fullscreen mode Exit fullscreen mode

Show Camera Video Stream

The camera video stream is implemented using OpenCV's VideoCapture class. A worker thread is created, running an infinite loop, to constantly capture the video stream and display frames in the PictureBox control.

private void StartScan()
{
    buttonCamera.Text = "Stop";
    isCapturing = true;
    thread = new Thread(new ThreadStart(FrameCallback));
    thread.Start();
}

private void StopScan()
{
    buttonCamera.Text = "Camera Scan";
    isCapturing = false;
    if (thread != null) thread.Join();
}

private void FrameCallback()
{
    while (isCapturing)
    {
        capture.Read(_mat);
        Mat copy = new Mat(_mat.Rows, _mat.Cols, MatType.CV_8UC3);
        _mat.CopyTo(copy);

        ...

        pictureBoxSrc.Image = BitmapConverter.ToBitmap(copy);
    }
    ...
}
Enter fullscreen mode Exit fullscreen mode

Detect and Rectify Documents

When a frame is captured as a Mat object, we initially create a duplicate of it. We use one frame buffer as an input for the DetectBuffer() method, while the other is employed to display the detection results using the DrawContours() method.

private Mat DetectDocument(Mat mat, Mat canvas)
{
    int length = mat.Cols * mat.Rows * mat.ElemSize();
    byte[] bytes = new byte[length];
    Marshal.Copy(mat.Data, bytes, 0, length);

    _docResults = documentScanner.DetectBuffer(bytes, mat.Cols, mat.Rows, (int)mat.Step(), DocumentScanner.ImagePixelFormat.IPF_RGB_888);
    if (_docResults != null)
    {
        DocResult result = _docResults[0];
        if (result.Points != null)
        {
            Point[] points = new Point[4];
            for (int i = 0; i < 4; i++)
            {
                points[i] = new Point(result.Points[i * 2], result.Points[i * 2 + 1]);
            }
            Cv2.DrawContours(canvas, new Point[][] { points }, 0, Scalar.Blue, 2);
        }
    }
    return canvas;
}
Enter fullscreen mode Exit fullscreen mode

Once the document is detected, the NormalizeBuffer() method can be employed to crop and rectify the document. This method returns a NormalizedImage object. To display the NormalizedImage object in the PictureBox control, you need to convert it to a Mat object first. Then use BitmapConverter class to convert the Mat object to a Bitmap object:

private void PreviewNormalizedImage()
{
    if (_docResults != null)
    {
        DocResult result = _docResults[0];
        int length = _mat.Cols * _mat.Rows * _mat.ElemSize();
        byte[] bytes = new byte[length];
        Marshal.Copy(_mat.Data, bytes, 0, length);

        NormalizedImage image = documentScanner.NormalizeBuffer(bytes, _mat.Cols, _mat.Rows, (int)_mat.Step(), DocumentScanner.ImagePixelFormat.IPF_RGB_888, result.Points);
        if (image != null && image.Data != null)
        {
            Mat newMat;
            if (image.Stride < image.Width)
            {
                // binary
                byte[] data = image.Binary2Grayscale();
                newMat = new Mat(image.Height, image.Width, MatType.CV_8UC1, data);
            }
            else if (image.Stride >= image.Width * 3)
            {
                // color
                newMat = new Mat(image.Height, image.Width, MatType.CV_8UC3, image.Data);
            }
            else
            {
                // grayscale
                newMat = new Mat(image.Height, image.Stride, MatType.CV_8UC1, image.Data);
            }

            Mat copy = new Mat(_mat.Rows, _mat.Cols, MatType.CV_8UC3);
            newMat.CopyTo(copy);

            pictureBoxDest.Image = BitmapConverter.ToBitmap(copy);
        }
    }
}
Enter fullscreen mode Exit fullscreen mode

Read Barcode and QR Code

Similar to the document detection process, we use the DecodeBuffer() method to detect barcodes and QR codes:

private Mat DetectBarcode(Mat mat, Mat canvas)
{
    int length = mat.Cols * mat.Rows * mat.ElemSize();
    byte[] bytes = new byte[length];
    Marshal.Copy(mat.Data, bytes, 0, length);

    BarcodeResult[]? results = barcodeScanner.DecodeBuffer(bytes, mat.Cols, mat.Rows, (int)mat.Step(), BarcodeQRCodeReader.ImagePixelFormat.IPF_RGB_888);
    if (results != null)
    {
        foreach (BarcodeResult result in results)
        {
            string output = "Text: " + result.Text + Environment.NewLine + "Format: " + result.Format1 + Environment.NewLine;
            this.BeginInvoke((MethodInvoker)delegate {
                richTextBoxInfo.AppendText(output);
                richTextBoxInfo.AppendText(Environment.NewLine);
            });

            int[]? points = result.Points;
            if (points != null)
            {
                OpenCvSharp.Point[] all = new OpenCvSharp.Point[4];
                int xMin = points[0], yMax = points[1];
                all[0] = new OpenCvSharp.Point(xMin, yMax);
                for (int i = 2; i < 7; i += 2)
                {
                    int x = points[i];
                    int y = points[i + 1];
                    OpenCvSharp.Point p = new OpenCvSharp.Point(x, y);
                    xMin = x < xMin ? x : xMin;
                    yMax = y > yMax ? y : yMax;
                    all[i / 2] = p;
                }
                OpenCvSharp.Point[][] contours = new OpenCvSharp.Point[][] { all };
                Cv2.DrawContours(canvas, contours, 0, new Scalar(0, 255, 0), 2);
                if (result.Text != null) Cv2.PutText(canvas, result.Text, new OpenCvSharp.Point(xMin, yMax), HersheyFonts.HersheySimplex, 1, new Scalar(0, 0, 255), 2);
            }
        }
    }

    return canvas;
}
Enter fullscreen mode Exit fullscreen mode

The BeginInvoke() method is used to update the UI from a non-UI thread.

Enhance MRZ Detection with Document Rectification

MRZ detection relies heavily on the alignment of characters. It is better to make the MRZ area horizontal for more accurate detection. Although the MRZ SDK does not feature quadrilateral distortion correction, it can employ the document scanner SDK to achieve the preprocessing. By combining these two SDKs, MRZ detection from the document image can be significantly optimized.

private Mat DetectMrz(Mat mat, Mat canvas)
{
    int length = mat.Cols * mat.Rows * mat.ElemSize();
    byte[] bytes = new byte[length];
    Marshal.Copy(mat.Data, bytes, 0, length);
    _mrzResults = mrzScanner.DetectBuffer(bytes, mat.Cols, mat.Rows, (int)mat.Step(), MrzScanner.ImagePixelFormat.IPF_RGB_888);
    if (_mrzResults != null)
    {
        string[] lines = new string[_mrzResults.Length];
        var index = 0;
        foreach (MrzResult result in _mrzResults)
        {
            lines[index++] = result.Text;
            this.BeginInvoke((MethodInvoker)delegate {
                richTextBoxInfo.Text += result.Text + Environment.NewLine;
            });

            if (result.Points != null)
            {
                Point[] points = new Point[4];
                for (int i = 0; i < 4; i++)
                {
                    points[i] = new Point(result.Points[i * 2], result.Points[i * 2 + 1]);
                }
                Cv2.DrawContours(canvas, new Point[][] { points }, 0, Scalar.Red, 2);
            }
        }

        JsonNode? info = Parse(lines);
        if (info != null)
        {
            this.BeginInvoke((MethodInvoker)delegate {
                richTextBoxInfo.Text = info.ToString();
            });

        }
    }

    return canvas;
}

private void PreviewNormalizedImage()
{
    if (_docResults != null)
    {
        ...
        if (checkBoxMrz.Checked)
        {
            copy = DetectMrz(newMat, copy);
        }
        pictureBoxDest.Image = BitmapConverter.ToBitmap(copy);
    }
}
Enter fullscreen mode Exit fullscreen mode

MRZ SDK only

only MRZ detection

Combine document detection and MRZ detection

combine document detection and MRZ detection

Source Code

https://github.com/yushulx/dotnet-barcode-qr-code-sdk/tree/main/example/barcode_document_mrz

Image
Kurs Dollar ke Rupiah Hari Ini: Informasi Penting tentang Nilai Tukar USD ke IDR
Image
File Uploads: How They Work, Where to Use Them, and How to Keep Them Secure

s3, webdev, website, beginners
Image
Stwórz Idealną Oazę Spokoju: Najmodniejsze Meble do Sypialni 2023
Image
Các hình thức đặt cược trò chơi bắn cá đổi thưởng đơn giản nhất
Image
Comprehensive Guide to Docker Compose: Use Cases, Examples, and Best Practices

docker, 100daysofcode, devops
Image
SCAM is one of the biggest fears of any crypto investor.

cryptocurrency, beginners
Image
Đa dạng cách chơi trò chơi sòng bạc Kubet
Image
I Really don't like chat gpt filters
Image
Hiring WordPress Developers: The Whole Procedure for Creating a Rich Website

wordpress, developer, webdev, javascript
Image
CKA Full Course 2024: Day 7/40 Pod in Kubernetes Explained
Image
Retrospectives Done Right: How to Make Your Team Meetings Actually Matter

retrospectives
Image
THE M.O.I.S.T Principle

principles, code, dry, programming
Image
Magia Aranżacji Wnętrz: Stwórz Przestrzeń, Która Inspirować Będzie Każdego Dnia!
Image
What is HTTP 422 Error? (Unprocessable Entity)

http
Image
Mastering Code Review in GitHub: Common Mistakes to Avoid

code, codereview, github, unittest
Image
Mount Up & Power Up: The Magnetic Wireless Charger Car Phone Holder You Need

mobile
Image
Vegas79 - Vstars79 Co - Cá cược Casino Top 1 UY tín nhất Việt Nam
Image
Automating Customs Financial Documents with IDP

ai, productivity, learning, discuss
Image
The Call for Justice: Severely Punish Guo Wengui and Give Justice to the Deceived
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Terabox Video Player