How to Optimize QR Recognition Performance by Image Preprocessing and Parameter Tuning

Xiao Ling - Sep 13 '21 - - Dev Community

When using a Barcode reader or scanner SDK, we primarily focus on the detection speed and recognition accuracy. There are many factors that may affect the performance of the Barcode detection SDK. The factors can be approximately classified into two categories: image quality and algorithm performance. In this article, we take Dynamsoft Barcode Reader Python edition as the example to discuss how to optimize the performance of the Barcode detection SDK by preprocessing input QR code images and adjusting the algorithm parameters on desktop platforms.

Prerequisites

  • OpenCV Python

    pip install opencv-python
pip install opencv-contrib-python

  • Dynamsoft Barcode Reader SDK

    pip install dbr

  • Download some QR code images by Google Image Search

Preprocessing QR Images

To evaluate the SDK performance, we use color image, grayscale image, and binary image as the input images.

Color Image

Let's get started with a color image of a QR code.

Air-Force-Reserve-San-Diego

The following code loads and displays the image using OpenCV and reads barcode information using Dynamsoft Barcode Reader.

import cv2
import numpy as np
import dbr
import time

reader = dbr.BarcodeReader()
reader.init_license("LICENSE-KEY") # https://www.dynamsoft.com/customer/license/trialLicense?product=dbr
image = cv2.imread('Air-Force-Reserve-San-Diego.jpg')

def detect(windowName, image, pixel_format):
    try:
        buffer = image.tobytes()
        height = image.shape[0]
        width = image.shape[1]
        stride = image.strides[0]
        start = time.time()
        results = reader.decode_buffer_manually(buffer, width, height, stride, pixel_format, "")
        end = time.time()
        print("Time taken: {:.3f}".format(end - start) + " seconds")
        cv2.putText(image, "Time taken: {:.3f}".format(end - start) + " seconds", (10, 30), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 0, 255), 2)

        if results != None:
            for result in results:
                print("Barcode Format : ")
                print(result.barcode_format_string)
                print("Barcode Text : ")
                print(result.barcode_text)

                points = result.localization_result.localization_points
                data = np.array([[points[0][0], points[0][1]], [points[1][0], points[1][1]], [points[2][0], points[2][1]], [points[3][0], points[3][1]]])
                cv2.drawContours(image=image, contours=[data], contourIdx=-1, color=(0, 255, 0), thickness=2, lineType=cv2.LINE_AA)

                x = min(points[0][0], points[1][0], points[2][0], points[3][0])
                y = min(points[0][1], points[1][1], points[2][1], points[3][1])
                cv2.putText(image, result.barcode_text, (x, y), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 0, 255), 2)

        cv2.imshow(windowName, image)
    except BarcodeReaderError as bre:
        print(bre)

detect('Color', image, dbr.EnumImagePixelFormat.IPF_RGB_888)
cv2.waitKey(0)
Enter fullscreen mode Exit fullscreen mode

QR code recognition

We initialize Dynamsoft Barcode Reader with the license key.

reader = dbr.BarcodeReader()
reader.init_license("LICENSE-KEY")
Enter fullscreen mode Exit fullscreen mode

By default, OpenCV loads images in BGR format. The corresponding enum value of pixel format provided by Dynamsoft Barcode Reader is dbr.EnumImagePixelFormat.IPF_RGB_888.

Then we call the decode_buffer_manually function to decode the image.

buffer = image.tobytes()
height = image.shape[0]
width = image.shape[1]
stride = image.strides[0]
results = reader.decode_buffer_manually(buffer, width, height, stride, dbr.EnumImagePixelFormat.IPF_RGB_888, "")
Enter fullscreen mode Exit fullscreen mode

The function returns a list of BarcodeResult objects. Each BarcodeResult object contains the barcode information and the localization result. The drawContours function and putText function can be used to draw the barcode information and localization result on the image.

points = result.localization_result.localization_points
data = np.array([[points[0][0], points[0][1]], [points[1][0], points[1][1]], [points[2][0], points[2][1]], [points[3][0], points[3][1]]])
cv2.drawContours(image=image, contours=[data], contourIdx=-1, color=(0, 255, 0), thickness=2, lineType=cv2.LINE_AA)

x = min(points[0][0], points[1][0], points[2][0], points[3][0])
y = min(points[0][1], points[1][1], points[2][1], points[3][1])
cv2.putText(image, result.barcode_text, (x, y), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 0, 255), 2)
Enter fullscreen mode Exit fullscreen mode

Grayscale Image

Grayscale image only carries intensity information. The barcode recognition API internally converts a color image to a grayscale image. So
theoretically, if the input image is a grayscale image, the recognition speed is faster.

We run the following code to see the time cost:

grayscale_image = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
detect('Grayscale', grayscale_image, dbr.EnumImagePixelFormat.IPF_GRAYSCALED)
Enter fullscreen mode Exit fullscreen mode

QR code recognition for grayscale image

The pixel format is changed to dbr.EnumImagePixelFormat.IPF_GRAYSCALED accordingly. As expected, decoding the grayscale image converted from the original color image saves a little time.

Binary Image

Thresholding the grayscale image to binary image can make it easier to analyze. We can also try it out by using the OpenCV function threshold:

# https://docs.opencv.org/4.5.1/d7/d4d/tutorial_py_thresholding.html
blur = cv2.GaussianBlur(grayscale_image,(5,5),0)
ret, thresh = cv2.threshold(blur, 0,255,cv2.THRESH_BINARY + cv2.THRESH_OTSU)
detect('Binary', thresh, dbr.EnumImagePixelFormat.IPF_GRAYSCALED)
Enter fullscreen mode Exit fullscreen mode

QR code recognition for binary image

We use Gaussian blur to reduce noise and smooth the image, and then use the OpenCV function threshold to convert the grayscale image to binary image. The Otsu's method is used to automatically determine the optimal global threshold value from the image histogram.

Image Time Cost
Color Image 0.014 s
Grayscale Image 0.010 s
Binary Image 0.009 s

More Tests

Let's take a look at some complicated cases.

Perspective Distortion

Here is a perspective distorted QR code image.

perspective distorted QR code image

If we use the code above, there is no barcode recognized, no matter color image, grayscale image or binary image. However, it does not mean the barcode image cannot be recognized. I managed to recognize the QR code by manually adjusting and setting the threshold value:

image = cv2.imread('perspective-distortion-qr-code.jpg')
grayscale_image = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
ret, thresh = cv2.threshold(grayscale_image, 125, 255,cv2.THRESH_BINARY) 
detect('Binary', thresh, dbr.EnumImagePixelFormat.IPF_GRAYSCALED)
Enter fullscreen mode Exit fullscreen mode 
Time taken: 0.0080 seconds
Barcode Format :
QR_CODE
Barcode Text :
http://goo.by/wb0Ric
Enter fullscreen mode Exit fullscreen mode

recognition for perspective distorted QR

Inverted Color

If we try to read the following QR code image printed with inverted color, it will fail to do the recognition.

inverted color QR code

After converting the image to binary image, we can see the black and white are inverted for QR code:

convert color image to binary image

To make the QR code recognizable, we use OpenCV function bitwise_not:

ret, thresh = cv2.threshold(grayscale_image, 150,255,cv2.THRESH_BINARY )
cv2.bitwise_not(thresh, thresh) 
detect('Inverted', thresh, dbr.EnumImagePixelFormat.IPF_GRAYSCALED)
Enter fullscreen mode Exit fullscreen mode 
Time taken: 0.0541 seconds
Barcode Format : 
QR_CODE
Barcode Text :
https://www.jalna.com.au/
Enter fullscreen mode Exit fullscreen mode

recognize inverted binary image

Adjusting the Algorithm Parameters

So far, we have tried some image preprocessing ways to read QR code. Actually, the barcode SDK contains more image processing and computer vision algorithms than the ones we have tried. To cover all cases, we need to know how to tune the parameters of the barcode SDK.

We use the following Python code to print the runtime parameters in console:

print(reader.get_runtime_settings().__dict__)
Enter fullscreen mode Exit fullscreen mode 
{'terminate_phase': 32, 'timeout': 10000, 'max_algorithm_thread_count': 4, 'expected_barcodes_count': 0, 'barcode_format_ids': -31457281, 'barcode_format_ids_2': 0, 'pdf_raster_dpi': 300, 'scale_down_threshold': 2300, 'binarization_modes': [2, 0, 0, 0, 0, 0, 0, 0], 'localization_modes': [2, 16, 4, 8, 0, 0, 0, 0], 'colour_clustering_modes': [0, 0, 0, 0, 0, 0, 0, 0], 
'colour_conversion_modes': [1, 0, 0, 0, 0, 0, 0, 0], 'grayscale_transformation_modes': [2, 0, 0, 0, 0, 0, 0, 0], 'region_predetection_modes': [2, 0, 0, 0, 0, 0, 0, 0], 'image_preprocessing_modes': [2, 0, 0, 0, 0, 0, 0, 0], 'texture_detection_modes': [2, 0, 0, 0, 0, 0, 0, 0], 'text_filter_modes': [2, 0, 0, 0, 0, 0, 0, 0], 'dpm_code_reading_modes': [0, 0, 0, 0, 0, 0, 
0, 0], 'deformation_resisting_modes': [0, 0, 0, 0, 0, 0, 0, 0], 'barcode_complement_modes': [0, 0, 0, 0, 0, 0, 0, 0], 'barcode_colour_modes': [1, 0, 0, 0, 0, 0, 0, 0], 'text_result_order_modes': [1, 2, 4, 0, 0, 0, 0, 0], 'text_assisted_correction_mode': 2, 'deblur_level': 9, 'intermediate_result_types': 0, 'intermediate_result_saving_mode': 1, 'result_coordinate_type': 1, 'return_barcode_zone_clarity': 0, 'region_top': 0, 'region_bottom': 0, 'region_left': 0, 'region_right': 0, 'region_measured_by_percentage': 0, 'min_barcode_text_length': 0, 'min_result_confidence': 0, 'scale_up_modes': [1, 0, 0, 0, 0, 0, 0, 0], 'accompanying_text_recognition_modes': [0, 0, 0, 0, 0, 0, 0, 0], 'pdf_reading_mode': 1, 'deblur_modes': [0, 0, 0, 0, 0, 0, 0, 0, 0, 0], 'barcode_zone_min_distance_to_image_borders': 0}
Enter fullscreen mode Exit fullscreen mode

There are quite a lot of parameters to tune, but you cannot find a crash course on how to tune these parameters. Fortunately, we can leverage the parameter tuning tool of Dynamsoft Barcode Reader online demo to customize parameters for specific images.

QR code parameter settings

There are five modes available for selection. You can click Advanced Settings to expand parameters of a mode. For better studying the parameter differences among different modes, we can scroll down the page and save all parameters to a json file.

QR code parameter settings

There is a tradeoff between the barcode decoding speed and recognition accuracy. The more algorithms are enabled for barcode recognition, the more time it takes.

Best coverage vs. Best speed
QR code parameter settings

Now, we upload the perspective distorted QR image to see the difference between the best speed and best coverage.

Best speed
It takes 15 milliseconds but fails to recognize the QR code.

QR perspective speed

Best coverage
It successfully recognizes the QR code but takes 828 milliseconds.
QR perspective coverage

Both of them are not ideal according to our test in Python code. To guarantee the best performance:

  1. Go to Advanced Settings > Binarization mode > Threshold.
  2. Change the threshold value to 125.

Now the QR code is recognized extremely fast.

QR perspective threshold

We can then export the template to a JSON file and load it in our Python code:

error = reader.init_runtime_settings_with_file('perspective-qr.json')
image = cv2.imread('perspective-distortion-qr-code.jpg')
detect('Color', image_copy, dbr.EnumImagePixelFormat.IPF_RGB_888)
Enter fullscreen mode Exit fullscreen mode

QR perspective speed

The result is satisfying.

Time taken: 0.0080 seconds
Barcode Format :
QR_CODE
Barcode Text :
http://goo.by/wb0Ric
Enter fullscreen mode Exit fullscreen mode

As for the inverted QR code, the setting is as follows:

inverted QR code

The performance is even better when reading the inverted QR code in Python.

inverted QR code

Time taken: 0.0160 seconds
Barcode Format : 
QR_CODE
Barcode Text :
https://www.jalna.com.au/
Enter fullscreen mode Exit fullscreen mode

In a nutshell, to get best performance of Dynamsoft Barcode Reader SDK, you should start with the simplest settings and then gradually increase and tune algorithm parameters.

Source Code

https://github.com/yushulx/python-barcode-qrcode-sdk/tree/main/examples/official/9.x/qrcode_template

Image
14 Savvy Ways To Spend Extra Money All Terrain Self Propelled Wheelchair Budget
Image
The Self-Propelled Wheelchairs Mistake That Every Beginner Makes
Image
<h1>Gevelreiniging Zwolle</h1>
Image
This Is The History Of Personal Injury Lawsuit In 10 Milestones
Image
10 Things You Learned In Kindergarden Which Will Aid You In Obtaining Drip Coffee Brewer
Image
11 Ways To Completely Revamp Your Repairing Bifold Doors
Image
Graffiti verwijderen & aanbrengen antigraffiticoating
Image
The History Of Bifold Door Seal Replacement In 10 Milestones
Image
soledast2
Image
This Is How Best ADHD Medication For Adults Will Look Like In 10 Years Time
Image
The People Closest To What Is A Self Propelled Wheelchair Uncover Big Secrets
Image
Building a High-Quality Stock Report Generator with Node.js, Express, and OpenAI API

ai, openai, express, javascript
Image
14 Cartoons About Car Key Repair Cost That'll Brighten Your Day
Image
Some notes on SharedWorkers

javascript, sharedworker, vite
Image
Five Tools Everybody Is In The Electric Fireplace Built In Wall Industry Should Be Using
Image
Check Out: How How To Get ADHD Medication UK Is Taking Over And What To Do About It
Image
4 Reasons Why You Should move from Lombok to Java Records

codeproject, java, javadeserializations
Image
Here's An Interesting Fact Concerning Traffic Accident Lawyer Near Me
Image
Why Adding A Coffee Makers Drip To Your Life's Routine Will Make The Difference
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Terabox Video Player