How to Create a .NET MAUI Plugin for Camera Barcode Qr Code Scanning

Xiao Ling - Jun 20 '23 - - Dev Community

As .NET MAUI continues to gain traction, numerous .NET developers are exploring ways to transition their existing desktop and mobile applications to this new framework. Previously, I developed a .NET MAUI application capable of scanning barcodes and QR codes from camera frames across Windows, Android, and iOS platforms. For reuse, I decided to isolate the camera and barcode scanning functionality and encapsulate them into a .NET MAUI plugin. In this article, I will guide you through the creation of a .NET MAUI plugin, featuring a custom camera view and the integration of the Dynamsoft Barcode Reader SDK.

Demo Video

Try Capture.Vision.Maui

https://www.nuget.org/packages/Capture.Vision.Maui/

The Shortcomings of My Current .NET MAUI Application

Check out the source code: https://github.com/yushulx/Capture-Vision-Maui. Before getting started with the plugin, let's take a look at the shortcomings of my current .NET MAUI application:

  • Windows: The camera stream is captured using OpenCVSharp. You need to install OpenCVSharp4 and OpenCV Windows runtime to make it work. Since the camera stream is displayed by rendering the OpenCVSharp Mat object to a SkiaSharp SKCanvas, the UI code cannot be shared across Android and iOS platforms.
  • Android and iOS: The code for Android and iOS is ported from my Xamarin.Forms Qr code scanner application. To make the code compatible with .NET MAUI, the AddCompatibilityRenderer() method is used. However, within the .NET MAUI environment, the AddHandler() method is recommended.

What's the difference between AddCompatibilityRenderer() and AddHandler()?

  • AddCompatibilityRenderer: This method is used to map Xamarin.Forms renderers to .NET MAUI handlers. It's a way of maintaining compatibility with existing Xamarin.Forms custom renderers as you transition your apps to .NET MAUI.

  • AddHandler: This is a method used in the context of the new architecture in .NET MAUI. Handlers are a more lightweight, efficient replacement for the old Xamarin.Forms renderers, and they allow greater control and easier customization of your controls.

Our objective is to develop a unified camera view equipped with barcode scanning capabilities, compatible with Windows, Android, and iOS platforms. The following resources will be helpful:

Initiate a .NET MAUI Plugin Project

Let's set up a .NET MAUI plugin project and configure the dependencies:

  1. Create a .NET MAUI class library project in Visual Studio 2022.

    .NET MAUI Class Library

  2. Double-click the project file and then add BarcodeQRCodeSDK as a dependency. The package is a .NET wrapper for the Dynamsoft Barcode Reader SDK, supporting Windows, Linux, macOS, Android, and iOS platforms. A valid license key is required to use the package. Note that the BarcodeQRCodeSDK package does not support Maccatalyst framework.

    <ItemGroup>
    <PackageReference Include="BarcodeQRCodeSDK" Version="2.3.4" />     
</ItemGroup>

  3. In order to test the plugin, add a new .NET MAUI app project to the solution.

    .NET MAUI APP

    After that, add a reference to the class library project in the app project.

    Add Reference

.NET MAUI Custom View with Platform-Specific Code

In .NET MAUI, a custom view is created by extending the View class. Simultaneously, an associated ViewHandler must be established to handle platform-specific rendering.

A view is responsible for defining what elements are present on the user interface and their properties. In our case, the camera view should be able to display the camera stream, retrieve camera frames and return barcode results.

using System.Collections.ObjectModel;
using static Capture.Vision.Maui.CameraInfo;

namespace Capture.Vision.Maui
{
    public class ResultReadyEventArgs : EventArgs
    {
        public ResultReadyEventArgs(object result, int previewWidth, int previewHeight)
        {
            Result = result;
            PreviewWidth = previewWidth;
            PreviewHeight = previewHeight;
        }

        public object Result { get; private set; }
        public int PreviewWidth { get; private set; }
        public int PreviewHeight { get; private set; }

    }

    public class FrameReadyEventArgs : EventArgs
    { 
        public enum PixelFormat
        {
            GRAYSCALE,
            RGB888,
            BGR888,
            RGBA8888,
            BGRA8888,
        }
        public FrameReadyEventArgs(byte[] buffer, int width, int height, int stride, PixelFormat pixelFormat)
        {
            Buffer = buffer;
            Width = width;
            Height = height;
            Stride = stride;
            Format = pixelFormat;
        }

        public byte[] Buffer { get; private set; }
        public int Width { get; private set; }
        public int Height { get; private set; }
        public int Stride { get; private set; }
        public PixelFormat Format { get; private set; }
    }

    public class CameraView : View
    {
        public static readonly BindableProperty CamerasProperty = BindableProperty.Create(nameof(Cameras), typeof(ObservableCollection<CameraInfo>), typeof(CameraView), new ObservableCollection<CameraInfo>());
        public static readonly BindableProperty CameraProperty = BindableProperty.Create(nameof(Camera), typeof(CameraInfo), typeof(CameraView), null);
        public static readonly BindableProperty EnableBarcodeProperty = BindableProperty.Create(nameof(EnableBarcode), typeof(bool), typeof(CameraView), false);
        public static readonly BindableProperty ShowCameraViewProperty = BindableProperty.Create(nameof(ShowCameraView), typeof(bool), typeof(CameraView), false, propertyChanged: ShowCameraViewChanged);
        public event EventHandler<ResultReadyEventArgs> ResultReady;
        public event EventHandler<FrameReadyEventArgs> FrameReady;
    }
}
Enter fullscreen mode Exit fullscreen mode

A handler is responsible for taking the view's definitions and translating them into platform-specific code that can be rendered on the screen.

using Microsoft.Maui.Handlers;
using static Capture.Vision.Maui.CameraInfo;
#if IOS
using PlatformView = Capture.Vision.Maui.Platforms.iOS.NativeCameraView;
#elif ANDROID
using PlatformView = Capture.Vision.Maui.Platforms.Android.NativeCameraView;
#elif WINDOWS
using PlatformView = Capture.Vision.Maui.Platforms.Windows.NativeCameraView;
#else
using PlatformView = System.Object;
#endif

namespace Capture.Vision.Maui
{
    internal partial class CameraViewHandler : ViewHandler<CameraView, PlatformView>
    {
        public static IPropertyMapper<CameraView, CameraViewHandler> PropertyMapper = new PropertyMapper<CameraView, CameraViewHandler>(ViewMapper)
        {
        };
        public static CommandMapper<CameraView, CameraViewHandler> CommandMapper = new(ViewCommandMapper)
        {
        };
        public CameraViewHandler() : base(PropertyMapper, CommandMapper)
        {
        }

#if ANDROID
    protected override PlatformView CreatePlatformView() => new(Context, VirtualView);
#elif IOS || WINDOWS
        protected override PlatformView CreatePlatformView() => new(VirtualView);
#else
    protected override PlatformView CreatePlatformView() => new();
#endif
        protected override void ConnectHandler(PlatformView platformView)
        {
            base.ConnectHandler(platformView);
        }

        protected override void DisconnectHandler(PlatformView platformView)
        {
#if WINDOWS || IOS || ANDROID
            platformView.DisposeControl();
#endif
            base.DisconnectHandler(platformView);
        }

        public Task<Status> StartCameraAsync()
        {
            if (PlatformView != null)
            {
#if WINDOWS || ANDROID || IOS
                return PlatformView.StartCameraAsync();
#endif
            }
            return Task.Run(() => { return Status.Unavailable; });
        }

        public Task<Status> StopCameraAsync()
        {
            if (PlatformView != null)
            {
#if WINDOWS
            return PlatformView.StopCameraAsync();
#elif ANDROID || IOS
                var task = new Task<Status>(() => { return PlatformView.StopCamera(); });
                task.Start();
                return task;
#endif
            }
            return Task.Run(() => { return Status.Unavailable; });
        }
    }

}
Enter fullscreen mode Exit fullscreen mode

The code snippet below demonstrates the platform-relevant classes used to display video across various platforms.

  • Windows: MediaPlayerElement

    public sealed partial class NativeCameraView : UserControl, IDisposable
{
    private readonly MediaPlayerElement mediaElement;
    private readonly CameraView cameraView;
    ...

    public NativeCameraView(CameraView cameraView)
    {
        this.cameraView = cameraView;
        mediaElement = new MediaPlayerElement
        {
            HorizontalAlignment = Microsoft.UI.Xaml.HorizontalAlignment.Stretch,
            VerticalAlignment = Microsoft.UI.Xaml.VerticalAlignment.Stretch
        };
        Content = mediaElement;
        ...
    }
    ...
}

  • Android: TextureView

    internal class NativeCameraView : FrameLayout
{
    private readonly CameraView cameraView;
    private readonly Context context;
    private readonly TextureView textureView;
    ...

    public NativeCameraView(Context context, CameraView cameraView) : base(context)
    {
        this.context = context;
        this.cameraView = cameraView;

        textureView = new(context);

        AddView(textureView);

        ...
    }
    ...
}

  • iOS: AVCaptureVideoPreviewLayer

    internal class NativeCameraView : UIView, IAVCaptureVideoDataOutputSampleBufferDelegate, IAVCapturePhotoCaptureDelegate
{
    private readonly CameraView cameraView;
    private readonly AVCaptureVideoPreviewLayer PreviewLayer;
    ...

    public NativeCameraView(CameraView cameraView)
    {
        this.cameraView = cameraView;

        captureSession = new AVCaptureSession
        {
            SessionPreset = AVCaptureSession.PresetPhoto
        };
        PreviewLayer = new(captureSession)
        {
            VideoGravity = AVLayerVideoGravity.ResizeAspectFill
        };
        Layer.AddSublayer(PreviewLayer);
        ...
    }
    ...
}

Our primary concern is determining how to obtain camera frames for image processing across different platforms.

Windows

  1. Create a MediaFrameReader:

    MediaFrameSource frameSource;
MediaFrameReader frameReader = await mediaCapture.CreateFrameReaderAsync(frameSource);
frameReader.AcquisitionMode = MediaFrameReaderAcquisitionMode.Realtime;
if (frameReader != null)
{
    frameReader.FrameArrived += OnFrameAvailable;
    var status = await frameReader.StartAsync();
}

  2. Retrieve the latest frame via the OnFrameAvailable event handler.

    private void OnFrameAvailable(MediaFrameReader sender, MediaFrameArrivedEventArgs args)
    {
        var frame = sender.TryAcquireLatestFrame();
        if (frame == null) return;

        SoftwareBitmap bitmap = frame.VideoMediaFrame.SoftwareBitmap;
        // process image
        bitmap.Dispose();
    }

Android

  1. Create an ImageReader and bind it to a background thread:

    private ImageReader imageReader;
imageReader = ImageReader.NewInstance(videoSize.Width, videoSize.Height, ImageFormatType.Yuv420888, 1);
backgroundThread = new HandlerThread("CameraBackground");
backgroundThread.Start();
backgroundHandler = new Handler(backgroundThread.Looper);
frameListener = new ImageAvailableListener(cameraView, barcodeReader);
imageReader.SetOnImageAvailableListener(frameListener, backgroundHandler);
surfaces.Add(new OutputConfiguration(imageReader.Surface));
previewBuilder.AddTarget(imageReader.Surface);

  2. Fetch the latest frame via the OnImageAvailable event handler.

    class ImageAvailableListener : Java.Lang.Object, ImageReader.IOnImageAvailableListener
{
    ...

    public void OnImageAvailable(ImageReader reader)
    {
        try
        {
            var image = reader?.AcquireLatestImage();
            if (image == null)
                return;

            Image.Plane[] planes = image.GetPlanes();
            if (planes == null) return;

            int width = image.Width;
            int height = image.Height;
            ByteBuffer buffer = planes[0].Buffer;
            byte[] bytes = new byte[buffer.Remaining()];
            buffer.Get(bytes);
            int nRowStride = planes[0].RowStride;
            int nPixelStride = planes[0].PixelStride;
            image.Close();

            // process image
        }
        catch (Exception ex)
        {
        }
    }
}

iOS

  1. Create an AVCaptureVideoDataOutput and set the SampleBufferDelegate to the current view.

    AVCaptureVideoDataOutput videoDataOutput = new AVCaptureVideoDataOutput();
        var videoSettings = NSDictionary.FromObjectAndKey(
            new NSNumber((int)CVPixelFormatType.CV32BGRA),
            CVPixelBuffer.PixelFormatTypeKey);
videoDataOutput.WeakVideoSettings = videoSettings;
videoDataOutput.AlwaysDiscardsLateVideoFrames = true;
cameraDispacher = new DispatchQueue("CameraDispacher");

videoDataOutput.SetSampleBufferDelegate(this, cameraDispacher);

  2. Fetch the latest frame via the DidOutputSampleBuffer event handler.

    [Export("captureOutput:didOutputSampleBuffer:fromConnection:")]
public void DidOutputSampleBuffer(AVCaptureOutput captureOutput, CMSampleBuffer sampleBuffer, AVCaptureConnection connection)
{
    CVPixelBuffer cVPixelBuffer = (CVPixelBuffer)sampleBuffer.GetImageBuffer();
    cVPixelBuffer.Lock(CVPixelBufferLock.ReadOnly);
    nint dataSize = cVPixelBuffer.DataSize;
    width = cVPixelBuffer.Width;
    height = cVPixelBuffer.Height;
    IntPtr baseAddress = cVPixelBuffer.BaseAddress;
    bpr = cVPixelBuffer.BytesPerRow;
    cVPixelBuffer.Unlock(CVPixelBufferLock.ReadOnly);
    buffer = NSData.FromBytes(baseAddress, (nuint)dataSize);
    // process image
}

How to Scan Barcodes from Camera Frames

As the camera frames are retrieved, we can now use the Dynamsoft Barcode Reader SDK to scan barcodes from the frames.

  1. Create a BarcodeReader instance:

    BarcodeQRCodeReader barcodeReader = BarcodeQRCodeReader.Create();

  2. To decode barcodes from the camera frame, you're required to retrieve the byte data, frame width, height, stride, and pixel format.

    // Windows
Result[] results = barcodeReader.DecodeBuffer(buffer, bitmap.PixelWidth, bitmap.PixelHeight, bitmap.PixelWidth, BarcodeQRCodeReader.ImagePixelFormat.IPF_GRAYSCALED);
// Android
Result[] results = barcodeReader.DecodeBuffer(bytes, width, height, nPixelStride * nRowStride, BarcodeQRCodeReader.ImagePixelFormat.IPF_GRAYSCALED);
// iOS
Result[] results = barcodeReader.DecodeBuffer(bytearray, (int)width, (int)height, (int)bpr, BarcodeQRCodeReader.ImagePixelFormat.IPF_ARGB_8888);

How to Use the Plugin in a .NET MAUI Application

Now that we have a .NET MAUI plugin in place, let's see how to use it in a .NET MAUI application.

  1. Add the plugin to the .NET MAUI app project via NuGet Manager.

    .NET MAUI camera barcode plugin

  2. In MauiProgram.cs, add the following code to register the plugin:

    using Microsoft.Extensions.Logging;

namespace Capture.Vision.Maui.Example
{
    public static class MauiProgram
    {
        public static MauiApp CreateMauiApp()
        {
            var builder = MauiApp.CreateBuilder();
            builder.UseNativeCameraView()
                .UseMauiApp<App>()
                .ConfigureFonts(fonts =>
                {
                    fonts.AddFont("OpenSans-Regular.ttf", "OpenSansRegular");
                    fonts.AddFont("OpenSans-Semibold.ttf", "OpenSansSemibold");
                });

#if DEBUG
        builder.Logging.AddDebug();
#endif

            return builder.Build();
        }
    }
}

  3. Apply for a trial license and then activate Dynamsoft Barcode Reader in MainPage.xaml.cs:

    using Dynamsoft;

namespace Capture.Vision.Maui.Example
{
    public partial class MainPage : ContentPage
    {
        public MainPage()
        {
            InitializeComponent();
            InitService();
        }

        private async void InitService()
        {
            await Task.Run(() =>
            {
                BarcodeQRCodeReader.InitLicense("LICENSE-KEY");

                return Task.CompletedTask;
            });
        }
    }
}

  4. Create a .NET MAUI content page and add a CameraView to it:

    <?xml version="1.0" encoding="utf-8" ?>
<ContentPage xmlns="http://schemas.microsoft.com/dotnet/2021/maui"
            xmlns:x="http://schemas.microsoft.com/winfx/2009/xaml"
            xmlns:skia="clr-namespace:SkiaSharp.Views.Maui.Controls;assembly=SkiaSharp.Views.Maui.Controls"
            xmlns:cv="clr-namespace:Capture.Vision.Maui;assembly=Capture.Vision.Maui"
            x:Class="Capture.Vision.Maui.Example.CameraPage"
            Title="CameraPage">
    <ScrollView>
        <Grid>
            <cv:CameraView x:Name="cameraView" HorizontalOptions="FillAndExpand"
            VerticalOptions="FillAndExpand" EnableBarcode="True" 
                                ResultReady="cameraView_ResultReady" FrameReady="cameraView_FrameReady"
                            />
            <skia:SKCanvasView x:Name="canvasView" 
                        Margin="0"
                        HorizontalOptions="FillAndExpand" VerticalOptions="FillAndExpand"
                        PaintSurface="OnCanvasViewPaintSurface" />
        </Grid>
    </ScrollView>
</ContentPage>

    The cameraView_ResultReady event handler is used to display the barcode result:

    private void cameraView_ResultReady(object sender, ResultReadyEventArgs e)
{
    if (e.Result != null)
    {
        Result[] results = (Result[])e.Result;
        foreach (Result result in results)
        {
            System.Diagnostics.Debug.WriteLine(result.Text);
        }
    }
}

    The cameraView_FrameReady event handler is used to retrieve the camera frame for image processing:

    private void cameraView_FrameReady(object sender, FrameReadyEventArgs e)
{
    // process image
}

    .NET MAUI Windows QR code scanner

Source Code

https://github.com/yushulx/Capture-Vision-Maui

Image
10 Great Books On Car Keys Repairs
Image
Why Nobody Cares About Automotive Key Repair
Image
10 Reasons Why People Hate Lost Keys Of Car
Image
Some Wisdom On Bifold Repairs Near Me From A Five-Year-Old
Image
Tech Debt: A Bigger Problem Than AI Takeover

technicaldebt, softwareengineering, ai, cleancode
Image
Announcing My Portfolio Site: A Digital Playground for Projects, Growth, and Coding Resources

webdev, javascript, portfolio, react
Image
How To Beat Your Boss On Mesothelioma Asbestos Claim
Image
Adding a Composite Door Cat Flap
Image
No Backend Skills? No Problem! Manage Databases Easily with custom-db-cli
Image
The Reason Why Adding A Bi Fold Repairs To Your Life Can Make All The Difference
Image
10 Myths Your Boss Has Concerning Bi Fold Door Repairs Near Me
Image
The Most Popular Van Hook Locks Fitted Is Gurus. 3 Things
Image
Top Benefits of Cloud Services for Small and Medium Businesses
Image
What Is Van Hook Locks Fitted And How To Use It?
Image
UPVC Door Panel With Cat Flap
Image
The Best Exposure To Asbestos Lawsuit Techniques To Change Your Life
Image
Sending Emails with Gmail using API

webdev, api, google, programming
Image
upvc Door Panels With Cat Flap
Image
How to Train Your Cat to Use a Cat Flap
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Terabox Video Player