How to Upload Images in Power Apps for Barcode, MRZ, and Document Detection

Xiao Ling - Dec 11 '23 - - Dev Community

In our previous article, we explained how to develop API endpoints for reading barcodes, recognizing MRZ, and rectifying document images in Node.js. In this article, we will integrate these REST API endpoints with the low-code platform Power Apps.

Loading and Displaying Images within Power Apps

  1. Click Insert and search for Add picture control, then drag it onto the canvas. This control allows you to upload images from your local disk or capture photos using a camera.

    Power app add picture

    The control is comprised of an AddMediaButton and an image display component.

    add picture control

  2. Set the Image property of the image control using this formula:

    If(IsBlank(AddMediaButton.Media), SampleImage, AddMediaButton.Media)

Now, we can upload an image and it will be displayed in the image control. You may need to adjust the position and size of both the button and the image for optimal layout.

Uploading Images from Power Apps to a Self-hosted Node.js Server

Uploading images from Power Apps to a self-hosted Node.js server involves several steps:

  1. Convert the image to a base64-encoded string.
  2. Establish an endpoint on the Node.js server for receiving the base64-encoded string and saving it as an image file.
  3. Develop a Power Automate flow to transmit the base64 string to the server.

How to Convert an Image to a Base64-Encoded String in Power Apps

In Power Apps, the JSON function can be used to convert an image to a base64-encoded string. This is achieved by utilizing the IncludeBinaryData parameter. To perform this conversion and store the result in a variable named ImageJSON, use the following formula:

Set(ImageJSON, JSON(UploadedImage.Image, JSONFormat.IncludeBinaryData));
Enter fullscreen mode Exit fullscreen mode

You can view the generated base64 string by navigating to Variables > ImageJSON:

image base64

How to Create an Endpoint for Uploading Images in Node.js

  1. Install the Express package:

    npm install express

  2. Create a web server using Express. Here's an example of basic server setup code:

    const express = require('express');
const app = express();

const port = process.env.PORT || 3000;

server.listen(port, '0.0.0.0', () => {
    host = server.address().address;
    console.log(`Server running at http://0.0.0.0:${port}/`);
});

  3. Create an endpoint to receive a base64-encoded string and save it as an image file on the server. This typically involves parsing the base64 string and writing it to a file. Here's a basic structure of such an endpoint:

    app.post('/dynamsoft/dlr/DetectMrz/base64', (req, res) => {
let jsonObject = req.body;
let size = Object.keys(jsonObject).length;
if (size == 0) {
    return res.status(400).send('No file uploaded.');
}

Object.keys(jsonObject).forEach(key => {
    let base64Image = jsonObject[key].split(';base64,').pop();
    const imageBuffer = Buffer.from(base64Image, 'base64');

    // Write buffer to a file
    const timestamp = Date.now();
    let filePath = 'uploads/' + timestamp;
    fs.writeFile(filePath, imageBuffer, async (err) => {
            if (err) {
                console.error('Error writing file:', err);
            } else {
                console.log('File saved:', filePath);
            }
        });
    });
});

    The endpoint is named /dynamsoft/dlr/DetectMrz/base64. This can be modified for various image processing tasks. It's designed to accept a POST request that includes a JSON object in the request body. This JSON object should contain the base64-encoded string of the image. The provided code will save the image in the uploads directory on the server, naming the file with a timestamp.

  4. For a quick test, we can use Ngrok to expose the local server to the Internet.

    ngrok http 3000

    ngrok

How to Upload a Base64-Encoded String via Power Automate

  1. Open Power Automate and create a new flow.

  2. Select Text as the input type.

    power automate flow input

  3. Add an HTTP action. Set the Method to POST. For the URI, use the endpoint we created previously. The Body should contain the input text.

    power automate http action

  4. Incorporate a Response action to display the server's response.

    power automate response action

    Note: The JSON schema for the response needs to be defined according to the server's response format. This will be covered in the following section.

Reading Barcodes, Recognizing MRZ, and Rectifying Documents in Power Apps

First, let's finalize the server-side code by defining three API endpoints for barcode reading, MRZ recognition, and document rectification:

app.post('/dynamsoft/dbr/DecodeBarcode/base64', (req, res) => {
    // ...
});

app.post('/dynamsoft/dlr/DetectMrz/base64', (req, res) => {
    // ...
});

app.post('/dynamsoft/dlr/DetectMrz/base64', (req, res) => {
    // ...
});
Enter fullscreen mode Exit fullscreen mode

For barcode processing: Utilize the decodeBase64Async method to decode the base64-encoded string directly:

let base64Image = jsonObject[key].split(';base64,').pop();
try {
    barcode4nodejs.decodeBase64Async(base64Image, barcode4nodejs.barcodeTypes).then((result) => {
        console.log(result);
        res.status(200).send(result);
    }).catch((err) => {
        console.error(err);
        return res.status(500).send('An error occurred while processing the image.');
    });
}
catch (err) {
    console.error(err);
    return res.status(500).send('An error occurred while processing the image.');
}
Enter fullscreen mode Exit fullscreen mode

For MRZ and document processing: Since the SDKs for MRZ and documents do not have methods to decode base64 strings, first save the base64 string as a file. Then, pass the file path to the respective SDK methods for further processing.

  • MRZ Recognition. Recognize the machine-readable zone (MRZ) from an image file and return the parsed results as a JSON string.

    try {
    let result = await mrzScanner.decodeFileAsync(filePath);
    let output = "";
    if (result.length == 2) {
        output = mrzScanner.parseTwoLines(result[0].text, result[1].text);
    }
    else if (result.length == 3) {
        output = mrzScanner.parseThreeLines(result[0].text, result[1].text, result[2].text);
    }
    console.log(output);
    res.status(200).send(output);
}
catch (err) {
    console.error(err);
    return res.status(500).send('An error occurred while processing the image.');
}

  • Document Rectification. Rectify a document image and save it as a JPEG file. Then, return the file path to display the rectified image in Power Apps.

    try {
    let results = await docRectifier.detectFileAsync(filePath);
    let result = results[0];
    result = await docRectifier.normalizeFileAsync(filePath, result['x1'], result['y1'], result['x2'], result['y2'], result['x3'], result['y3'], result['x4'], result['y4']);
    let data = result['data']
    let width = result['width']
    let height = result['height']
    for (let i = 0; i < data.length; i += 4) {
        const red = data[i];
        const blue = data[i + 2];
        data[i] = blue;
        data[i + 2] = red;
    }

    timestamp = Date.now();

    sharp(data, {
        raw: {
            width: width,
            height: height,
            channels: 4
        }
    }).toFile('uploads/' + timestamp + '.jpeg', (err, info) => {
        if (err) {
            console.error('Error:', err);
        } else {
            res.send(JSON.stringify({
                'image': 'uploads/' + timestamp + '.jpeg'
            }));
        }
    });
}
catch (err) {
    console.error(err);
    return res.status(500).send('An error occurred while processing the image.');
}

We create three Power Automate Flows for the three endpoints. While these flows are similar in structure, they differ in their response actions.

  • The JSON schema of the barcode reading response is:

    {
    "type": "array",
    "items": {
        "type": "object",
        "properties": {
            "format": {
                "type": "string"
            },
            "value": {
                "type": "string"
            },
            "x1": {
                "type": "integer"
            },
            "y1": {
                "type": "integer"
            },
            "x2": {
                "type": "integer"
            },
            "y2": {
                "type": "integer"
            },
            "x3": {
                "type": "integer"
            },
            "y3": {
                "type": "integer"
            },
            "x4": {
                "type": "integer"
            },
            "y4": {
                "type": "integer"
            },
            "page": {
                "type": "integer"
            },
            "time": {
                "type": "integer"
            }
        },
        "required": [
            "format",
            "value",
            "x1",
            "y1",
            "x2",
            "y2",
            "x3",
            "y3",
            "x4",
            "y4",
            "page",
            "time"
        ]
    }
}

  • The JSON schema of the MRZ recognition response is:

    {
    "type": "object",
    "properties": {
        "type": {
            "type": "string"
        },
        "nationality": {
            "type": "string"
        },
        "surname": {
            "type": "string"
        },
        "givenname": {
            "type": "string"
        },
        "passportnumber": {
            "type": "string"
        },
        "issuecountry": {
            "type": "string"
        },
        "birth": {
            "type": "string"
        },
        "gender": {
            "type": "string"
        },
        "expiry": {
            "type": "string"
        }
    }
}

  • The JSON schema of the document rectification response is:

    {
    "type": "object",
    "properties": {
        "image": {
            "type": "string"
        }
    }
}

Finally, to complete the app, add more controls to the canvas:

  1. Insert a Drop down control and set the Items property to ["Dynamsoft Barcode Reader", "Dynamsoft Label Recognizer", "Dynamsoft Document Normalizer"].

  2. Add a Button control and set the OnSelect property to the following formula:

    Set(ImageJSON, JSON(UploadedImage.Image, JSONFormat.IncludeBinaryData));
Switch(DropDown.SelectedText.Value, "Dynamsoft Barcode Reader", ClearCollect(Results, BarcodeUpload.Run(ImageJSON)), "Dynamsoft Label Recognizer", ClearCollect(Results, MrzUpload.Run(ImageJSON)), "Dynamsoft Document Normalizer", Set(URL, DocumentUpload.Run(ImageJSON)))

    This formula will call the corresponding Power Automate flow based on the selection in the dropdown control, utilizing the Switch function. The ClearCollect function will store the response in the Results collection, while the Set function will assign the response to the URL variable.

  3. Add a Text input control and set the Visible property to the following formula:

    If(DropDown.SelectedText.Value = "Dynamsoft Barcode Reader" Or DropDown.SelectedText.Value = "Dynamsoft Label Recognizer", true, false)

    Only the results from barcode and MRZ recognition will be displayed in the Text input control. The Default property is set to JSON(Results).

  4. Add an Image control and set the Visible property to the following formula:

    If(DropDown.SelectedText.Value = "Dynamsoft Document Normalizer", true, false)

    The Image property is set to Concatenate("https://563f-98-96-222-66.ngrok-free.app/", URL.image).

Test the Power App

Barcode Detection

barcode Detection

MRZ Recognition

MRZ Recognition

Document Rectification

Document Rectification

Source Code

https://github.com/yushulx/dynamsoft-sdk-nodejs-REST-API.git

Image
7 Simple Secrets To Totally You Into Bi-Fold Door Repair
Image
Cisza w Twoim Domu: Jak Wyciszenie Ścian Zmienia Przestrzeń
Image
How To Create An Awesome Instagram Video About Diagnose ADHD
Image
Area code Speedy Sales: The Ultimate Guide to Selling Your House Fast!
Image
Day 6/90: File Permissions and Access Control Lists in Linux 🔐 #90DaysOfDevOps

linux, devops, security, bash
Image
Cat Flap Installation Cost
Image
Chapter 1: The Bloginning

learning, coding, community
Image
Turbocharge Your Move: Professional Tips to Promote Your House Fast!
Image
How Much Does a Cat Flap Cost?
Image
Elevate Your Wardrobe: Essential Style Tips for the Modern Man
Image
Introducing a new PHP microservice template: your feedback needed!

webdev, php, microservices, programming
Image
The Reasons You Should Experience Bifold Door Repair Near Me At Least Once In Your Lifetime
Image
Breaking Down The Top WPF Datagrids

webdev, devops, dotnet, development
Image
15 Reasons Not To Be Ignoring Bifold Door Repair Near Me
Image
Turbocharge Your Move: Pro Tips to Promote Your House Fast!
Image
T
Image
Speedy Sales: Unlocking the Secrets to Promoting Your House throughout Record Time!
Image
Cat Flap Double Glazing
Image
Cat Flap Installation - How to Fit a Cat Flap in a Door
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Terabox Video Player