How to Build Mobile Document Scanning App with Xamarin.Forms for Android and iOS

Xiao Ling - Nov 11 '22 - - Dev Community

Mobile document scanning apps bring convenience to users by allowing them to scan travel documents, ID cards, passports and other types of documents on the go. This article aims to help C# developers, who do not have the background in computer vision, to build a mobile document scanning app for both Android and iOS using Xamarin.Forms and Dynamsoft Document Normalizer SDK.

Dynamsoft.DocumentNormalizer.Xamarin.Forms

Xamarin.Forms enables developers to build cross-platform mobile apps with C# and XAML from a single codebase. Dynamsoft.DocumentNormalizer.Xamarin.Forms is a Xamarin.Forms wrapper for Dynamsoft Document Normalizer SDK. The primary features of the SDK include:

  • Camera view for real-time scanning
  • Document edge detection
  • Perspective correction
  • Parameter settings for image normalization
  • File saving

Online Documentation

https://www.dynamsoft.com/document-normalizer/docs/programming/xamarin/user-guide.html

Steps to Build a Mobile Document Scanning App with Xamarin.Forms

In the following paragraphs, we will use Visual Studio 2022 for Windows to build the document scanning app.

Create and Configure Xamarin.Forms Project for Android and iOS

Create an empty Xamarin.Forms project in Visual Studio 2022 and pair the project to a remote macOS.

Xamarin Forms project

Try to build the project for Android and iOS respectively. If you suffered from the iOS build issue Error MT4109: Failed to compile the generated registrar code caused by Xcode 14, you can visit https://github.com/xamarin/xamarin-macios/issues/15954 to find the solution. The issue can be fixed by installing xamarin.ios-16.0.0.72.pkg. To run the app on iOS 16, turn on Settings > Privacy & Security > Developer Mode.

The Xamarin.Forms version used by Dynamsoft.DocumentNormalizer.Xamarin.Forms is 5.0.0.2515, whereas the default version of Xamarin.Forms in Visual Studio 2022 is 5.0.0.2196. To install the SDK, you need to update the version of Xamarin.Forms by right-clicking the project and selecting Manage NuGet Packages.

Update Xamarin.Forms version

Then install the SDK by searching Dynamsoft.DocumentNormalizer.Xamarin.Forms in the NuGet package manager.

Xamarin.Forms document scanning SDK

The final step is to add the camera permission for Android and iOS.

AndroidManifest.xml

<uses-permission android:name="android.permission.CAMERA" />
Enter fullscreen mode Exit fullscreen mode

Info.plist

<key>NSCameraUsageDescription</key>
<string>This app is using the camera</string>
Enter fullscreen mode Exit fullscreen mode

Initialize the Document Normalizer SDK

Before getting started to write the shared code, you need to initialize the SDK in the platform-specific code. It is a little bit different between Android and iOS.

// Android MainActivity.cs
using DDNXamarin.Droid;

protected override void OnCreate(Bundle savedInstanceState)
{
    LoadApplication(new App(new DCVCameraEnhancer(this), new DCVDocumentNormalizer(), new DCVLicenseManager(this)));
}

// iOS AppDelegate.cs
using DDNXamarin.iOS;
public override bool FinishedLaunching(UIApplication app, NSDictionary options)
{
    LoadApplication(new App(new DCVCameraEnhancer(), new DCVDocumentNormalizer(), new DCVLicenseManager()));
}
Enter fullscreen mode Exit fullscreen mode

In the shared code App.xaml.cs, we register the license key and create static variables to store the DCVCameraEnhancer and DCVDocumentNormalizer instances for later use. The license key can be obtained from https://www.dynamsoft.com/customer/license/trialLicense.

using DDNXamarin;

public partial class App : Application, ILicenseVerificationListener
{
    public static ICameraEnhancer dce;
    public static IDocumentNormalizer ddn;
    public static ILicenseManager licenseManager;

    public App(ICameraEnhancer enhancer, IDocumentNormalizer normalizer, ILicenseManager manager)
    {
        ...

        dce = enhancer;
        ddn = normalizer;

        licenseManager = manager;
        licenseManager.InitLicense("DLS2eyJoYW5kc2hha2VDb2RlIjoiMjAwMDAxLTE2NDk4Mjk3OTI2MzUiLCJvcmdhbml6YXRpb25JRCI6IjIwMDAwMSIsInNlc3Npb25QYXNzd29yZCI6IndTcGR6Vm05WDJrcEQ5YUoifQ==", this);
        ...
    }

    public void LicenseVerificationCallback(bool isSuccess, string msg)
    {

    }
}
Enter fullscreen mode Exit fullscreen mode

Content Page for Camera Preview and Real-time Scanning

Create a new content page CustomRendererPage.xaml and add a DCVCameraView control to it.

<?xml version="1.0" encoding="utf-8" ?>
<ContentPage xmlns="http://xamarin.com/schemas/2014/forms"
             xmlns:x="http://schemas.microsoft.com/winfx/2009/xaml"
             xmlns:dynamsoft = "clr-namespace:DDNXamarin;assembly=DDN-Xamarin"
             x:Class="DocumentScanner.CustomRendererPage">
    <ContentPage.Content>
        <AbsoluteLayout>
            <dynamsoft:DCVCameraView AbsoluteLayout.LayoutBounds="0,0,1,1" AbsoluteLayout.LayoutFlags="All" x:Name="preview">
            </dynamsoft:DCVCameraView>
            <Button x:Name="capture" Text="Capture"
                    AbsoluteLayout.LayoutBounds="0.5,0.5,120,50" AbsoluteLayout.LayoutFlags="PositionProportional"
                    Clicked="OnButtonClicked">
            </Button>
        </AbsoluteLayout>
    </ContentPage.Content>
</ContentPage>
Enter fullscreen mode Exit fullscreen mode

The DCVCameraView control is a custom renderer for Xamarin.Forms. It does not only wrap the native camera view, but also provides the real-time document edge detection feature.

public partial class CustomRendererPage : ContentPage, IDetectResultListener
{
    public static ICameraEnhancer dce;
    public static IDocumentNormalizer ddn;


    public CustomRendererPage()
    {
        InitializeComponent();
        App.ddn.SetCameraEnhancer(App.dce);
        App.ddn.AddResultListener(this);
    }

    public void DetectResultCallback(int id, ImageData imageData, DetectedQuadResult[] quadResults)
    {

        if (imageData != null && quadResults != null)

        {

            Device.BeginInvokeOnMainThread(async () => {
                await Navigation.PushAsync(new QuadEditorPage(imageData, quadResults));
            });
        }
    }

    protected override void OnAppearing()
    {

        base.OnAppearing();
        App.dce.Open();
        App.ddn.StartDetecting();
    }

    protected override void OnDisappearing()
    {
        base.OnDisappearing();
        App.dce.Close();
        App.ddn.StopDetecting();

    }

    void OnButtonClicked(object sender, EventArgs e)
    {
        App.ddn.EnableReturnImageOnNextCallback();
    }
}
Enter fullscreen mode Exit fullscreen mode

Android

Xamarin.Forms document edge detection for Android

iOS

Xamarin.Forms document edge detection for iOS

Once we get the detected document edges via the callback function, we can launch a new page to edit the document edges.

Content Page for Editing Document Edges

Create a new content page QuadEditorPage.xaml and add a DCVImageEditorView control to it.

<?xml version="1.0" encoding="utf-8" ?>
<ContentPage xmlns="http://xamarin.com/schemas/2014/forms"
             xmlns:x="http://schemas.microsoft.com/winfx/2009/xaml"
             xmlns:dynamsoft = "clr-namespace:DDNXamarin;assembly=DDN-Xamarin"
             x:Class="DocumentScanner.QuadEditorPage">
    <ContentPage.Content>
        <AbsoluteLayout>
            <dynamsoft:DCVImageEditorView AbsoluteLayout.LayoutBounds="0,0,1,1" AbsoluteLayout.LayoutFlags="All"
                x:Name="imageEditor">
            </dynamsoft:DCVImageEditorView>

            <Button
                x:Name="normalize"
                Clicked="OnNormalizeClicked"
                Text="Save"
                AbsoluteLayout.LayoutBounds="0.5,0.5,120,50"
                AbsoluteLayout.LayoutFlags="PositionProportional">
            </Button>

        </AbsoluteLayout>

    </ContentPage.Content>
</ContentPage>
Enter fullscreen mode Exit fullscreen mode

The DCVImageEditorView control allows you to edit the document edges and save the points to a Quadrilateral object.

 public partial class QuadEditorPage : ContentPage
{
    ImageData data;
    DetectedQuadResult[] results;
    public QuadEditorPage(ImageData imageData, DetectedQuadResult[] results)
    {
        InitializeComponent();
        data = imageData;
        this.results = results;
    }

    protected override void OnAppearing()
    {
        base.OnAppearing();
        if (data != null)
        {
            imageEditor.OriginalImage = data;
        }
        if (results != null)
        {
            imageEditor.DetectedQuadResults = results;
        }
    }

    void OnNormalizeClicked(object sender, EventArgs e)
    {
        try
        {
            var quad = imageEditor.getSelectedQuadResult();
            if (quad != null)
            {
                Navigation.PushAsync(new NormalizedPage(data, quad));
            }
        }
        catch (Exception exception)
        {
            Device.BeginInvokeOnMainThread(async () => {
                await DisplayAlert("Error", exception.ToString(), "OK");
            });
        }

    }
}
Enter fullscreen mode Exit fullscreen mode

Android

Xamarin.Forms document edge editor for Android

iOS

Xamarin.Forms document edge editor for iOS

After checking and editing the document edges, we can launch a new page to normalize the document based on the selected quadrilateral.

Content Page for Document Cropping and Perspective Correction

Create a new content page NormalizedPage.xaml to show the normalized document.

<?xml version="1.0" encoding="utf-8" ?>
<ContentPage xmlns="http://xamarin.com/schemas/2014/forms"
             xmlns:x="http://schemas.microsoft.com/winfx/2009/xaml"
             xmlns:local="clr-namespace:DocumentScanner"
             x:Class="DocumentScanner.NormalizedPage">
    <ContentPage.Content>
        <AbsoluteLayout>
            <local:GestureView x:Name="gestureView" AbsoluteLayout.LayoutBounds="0.5,0.5,1024,1024"
                AbsoluteLayout.LayoutFlags="PositionProportional">
            </local:GestureView>

            <Button
                x:Name="shareButton"
                Clicked="OnShareClicked"
                Text="Share"
                AbsoluteLayout.LayoutBounds="0.5,1,120,50"
                AbsoluteLayout.LayoutFlags="PositionProportional">
            </Button>

            <StackLayout RadioButtonGroup.GroupName="colors" Orientation="Horizontal">
                <RadioButton Content="Binary" IsChecked="True" CheckedChanged="RadioButton_CheckedChanged"/>
                <RadioButton Content="Color" CheckedChanged="RadioButton_CheckedChanged"/>
                <RadioButton Content="Grayscale" CheckedChanged="RadioButton_CheckedChanged"/>
            </StackLayout>

        </AbsoluteLayout>
    </ContentPage.Content>
</ContentPage>
Enter fullscreen mode Exit fullscreen mode

We put the image in a GestureView.xaml control to allow the user to zoom and pan the image.

<?xml version="1.0" encoding="UTF-8"?>
<ContentView xmlns="http://xamarin.com/schemas/2014/forms" 
             xmlns:x="http://schemas.microsoft.com/winfx/2009/xaml"
             x:Class="DocumentScanner.GestureView">
  <ContentView.Content>
        <StackLayout  HorizontalOptions="CenterAndExpand"
                 VerticalOptions="CenterAndExpand"
                     >
            <Image 
                    x:Name="image">
            </Image>

        </StackLayout>
    </ContentView.Content>
</ContentView>
Enter fullscreen mode Exit fullscreen mode

The gesture detection code is from Microsoft's Xamarin.Forms Cookbook.

public partial class GestureView : ContentView
{
    double currentScale = 1;
    double startScale = 1;
    double x = 0;
    double y = 0;

    public Image getImage()
    {
        return image;
    }
    void OnPinchUpdated(object sender, PinchGestureUpdatedEventArgs e)
    {
        ...
    }

    void OnPanUpdated(object sender, PanUpdatedEventArgs e)
    {
        switch (e.StatusType)
        {
            case GestureStatus.Running:
                Content.TranslationX = Math.Max(Math.Min(0, x + e.TotalX), -Math.Abs(Content.Width * Content.Scale - App.ScreenWidth));
                ...
                break;

            ...
        }
    }

    public GestureView()
    {
        InitializeComponent();

        var pinchGesture = new PinchGestureRecognizer();
        pinchGesture.PinchUpdated += OnPinchUpdated;
        GestureRecognizers.Add(pinchGesture);

        var panGesture = new PanGestureRecognizer();
        panGesture.PanUpdated += OnPanUpdated;
        GestureRecognizers.Add(panGesture);
    }
}
Enter fullscreen mode Exit fullscreen mode

Code change is required for pan translation calculation.

- Content.TranslationX = Math.Max (Math.Min (0, x + e.TotalX), -Math.Abs (Content.Width - App.ScreenWidth));
- Content.TranslationY = Math.Max (Math.Min (0, y + e.TotalY), -Math.Abs (Content.Height - App.ScreenHeight));
+ Content.TranslationX = Math.Max(Math.Min(0, x + e.TotalX), -Math.Abs(Content.Width * Content.Scale - App.ScreenWidth));
+ Content.TranslationY = Math.Max(Math.Min(0, y + e.TotalY), -Math.Abs(Content.Height * Content.Scale - App.ScreenHeight));
Enter fullscreen mode Exit fullscreen mode

Back to the NormalizedPage, we specify the color template by selecting a radio button.

private void UpdateNormalizedImage(string template)
{
    App.ddn.InitRuntimeSettings(template);
    normalizedImage = App.ddn.Normalize(imageData, quadrilateral);

    Image image = gestureView.getImage();

    image.Source = normalizedImage.image.ToImageSource();
    if (Device.RuntimePlatform == Device.iOS)
    {
        image.RotateTo(normalizedImage.image.orientation);
    }
}

private void RadioButton_CheckedChanged(object sender, CheckedChangedEventArgs e)
{
    RadioButton button = sender as RadioButton;
    if (button != null)
    {
        if (button.Content.Equals("Binary") && button.IsChecked)
        {
            UpdateNormalizedImage(Templates.binary);
        }
        if (button.Content.Equals("Color") && button.IsChecked)
        {
            UpdateNormalizedImage(Templates.color);
        }
        if (button.Content.Equals("Grayscale") && button.IsChecked)
        {
            UpdateNormalizedImage(Templates.grayscale);
        }
    }
}
Enter fullscreen mode Exit fullscreen mode

The templates are defined in the Templates class.

public class Templates
{
    public static string binary = @"{
    ""GlobalParameter"":{
        ""Name"":""GP""
    },
    ""ImageParameterArray"":[
        {
            ""Name"":""IP-1"",
            ""NormalizerParameterName"":""NP-1""
        }
    ],
    ""NormalizerParameterArray"":[
        {
            ""Name"":""NP-1"",
            ""ColourMode"": ""ICM_BINARY"" 
        }
    ]
}";

    public static string color = @"{
    ""GlobalParameter"":{
        ""Name"":""GP""
    },
    ""ImageParameterArray"":[
        {
            ""Name"":""IP-1"",
            ""NormalizerParameterName"":""NP-1""
        }
    ],
    ""NormalizerParameterArray"":[
        {
            ""Name"":""NP-1"",
            ""ColourMode"": ""ICM_COLOUR"" 
        }
    ]
}";

    public static string grayscale = @"{
    ""GlobalParameter"":{
        ""Name"":""GP""
    },
    ""ImageParameterArray"":[
        {
            ""Name"":""IP-1"",
            ""NormalizerParameterName"":""NP-1""
        }
    ],
    ""NormalizerParameterArray"":[
        {
            ""Name"":""NP-1"",
            ""ColourMode"": ""ICM_GRAYSCALE"" 
        }
    ]
}";
}
Enter fullscreen mode Exit fullscreen mode

Finally, we can save and share the normalized document image as JPEG, PNG or PDF.

string file = Path.Combine(FileSystem.CacheDirectory, "normalized.png");
App.ddn.SaveToFile(normalizedImage, file);
Device.BeginInvokeOnMainThread(async () => {

    await Share.RequestAsync(new ShareFileRequest
    {
        Title = Title,
        File = new ShareFile(file),
        PresentationSourceBounds = DeviceInfo.Platform == DevicePlatform.iOS && DeviceInfo.Idiom == DeviceIdiom.Tablet
        ? new System.Drawing.Rectangle(0, 20, 0, 0)
        : System.Drawing.Rectangle.Empty
    });
});
Enter fullscreen mode Exit fullscreen mode

Android

Xamarin.Forms document normalization for Android

iOS

Xamarin.Forms document normalization for iOS

Source Code

https://github.com/yushulx/Xamarin-forms-document-scanner

Image
Installing a upvc Door Panel Cat Flap
Image
How To Tell The Bi Fold Door Repair Right For You
Image
Uma introdução a Padrões de projeto

designpatterns
Image
Why Professional Assignment Help is a Smart Choice for Struggling Students?

domyprogramminghomework, programmingassignmenthelp, assignmenthelpservice, finalyearprojecthelp
Image
How To Determine If You're Prepared For Asbestos Law Firm
Image
The Useless Type Calculator in Typescript

typescript, calculator, typings
Image
Pieces for Developers: Revolutionize Your Coding Workflow with AI and Smart Organization

codesnippetmanager, besttoolsfordevelope, productivity, developerproductivit
Image
You Will Meet The Steve Jobs Of The Non Asbestos Causes Of Mesothelioma Industry
Image
Build your No-Code AI Agent with VAKX

ai, rpa, nocode, python
Image
The Ugly The Truth About Lost Car Key
Image
9 Signs You're A Drip Coffeee Expert
Image
Tile
Image
Share a random number

probtrix
Image
Wondershare Filmora Crack 14.0.7 + Key Full Download [Latest]
Image
Guia básico de Spring Security

java, springsecurity
Image
20 Top Tweets Of All Time About Asbestos Lawsuit Attorney
Image
Asbestos Illness Compensation's History History Of Asbestos Illness Compensation
Image
My First Week in DevOps: Starting with Linux

cloud, devops, linux, beginners
Image
Why Net2Secure Needs Linux Dedicated Server Hosting Benefits
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Terabox Video Player