How to Build Flutter Document Scanning Plugin for Windows and Linux

Xiao Ling - Dec 7 '22 - - Dev Community

Last week, we finished a Flutter document scanning plugin for web, which is just a start. This week, we continue to empower the plugin for desktop development using Dynamsoft Document Normalizer C++ SDK. The plugin will support Windows and Linux. In this article, you will see the steps to build the Flutter desktop document scanning plugin and the differences of Flutter C++ code for Windows and Linux.

Flutter Document Scan SDK

https://pub.dev/packages/flutter_document_scan_sdk

Development Environment

  • Flutter 3.3.9
  • Ubuntu 22.04
  • Windows 10

Add Support for Windows and Linux in Flutter Plugin

To support Windows and Linux, run the following command in the project root directory:

flutter create --org com.dynamsoft --template=plugin --platforms=windows,linux .
Enter fullscreen mode Exit fullscreen mode

After generating the platform-specific code, we update the pubspec.yaml file:

plugin:
    platforms:
      linux:
        pluginClass: FlutterDocumentScanSdkPlugin
      windows:
        pluginClass: FlutterDocumentScanSdkPluginCApi
      web:
        pluginClass: FlutterDocumentScanSdkWeb
        fileName: flutter_document_scan_sdk_web.dart
Enter fullscreen mode Exit fullscreen mode

You should have noticed that the Flutter plugin project contains flutter_document_scan_sdk_web.dart and flutter_document_scan_sdk_method_channel.dart files. The former is used for web. We had added implementations last week. The latter is used for mobile and desktop. It uses method channel to interact with the native platform:

class MethodChannelFlutterDocumentScanSdk
    extends FlutterDocumentScanSdkPlatform {
  @visibleForTesting
  final methodChannel = const MethodChannel('flutter_document_scan_sdk');

  @override
  Future<String?> getPlatformVersion() async {
    final version =
        await methodChannel.invokeMethod<String>('getPlatformVersion');
    return version;
  }

  @override
  Future<int?> init(String path, String key) async {
    return await methodChannel
        .invokeMethod<int>('init', {'path': path, 'key': key});
  }

  @override
  Future<int?> setParameters(String params) async {
    return await methodChannel
        .invokeMethod<int>('setParameters', {'params': params});
  }

  @override
  Future<String?> getParameters() async {
    return await methodChannel.invokeMethod<String>('getParameters');
  }

  @override
  Future<List<DocumentResult>> detect(String file) async {
    List? results = await methodChannel.invokeListMethod<dynamic>(
      'detect',
      {'file': file},
    );

    return _resultWrapper(results);
  }

  List<DocumentResult> _resultWrapper(List<dynamic>? results) {
    List<DocumentResult> output = [];

    if (results != null) {
      for (var result in results) {
        int confidence = result['confidence'];
        List<Offset> offsets = [];
        int x1 = result['x1'];
        int y1 = result['y1'];
        int x2 = result['x2'];
        int y2 = result['y2'];
        int x3 = result['x3'];
        int y3 = result['y3'];
        int x4 = result['x4'];
        int y4 = result['y4'];
        offsets.add(Offset(x1.toDouble(), y1.toDouble()));
        offsets.add(Offset(x2.toDouble(), y2.toDouble()));
        offsets.add(Offset(x3.toDouble(), y3.toDouble()));
        offsets.add(Offset(x4.toDouble(), y4.toDouble()));
        DocumentResult documentResult = DocumentResult(confidence, offsets, []);
        output.add(documentResult);
      }
    }

    return output;
  }

  @override
  Future<NormalizedImage?> normalize(String file, dynamic points) async {
    Offset offset = points[0];
    int x1 = offset.dx.toInt();
    int y1 = offset.dy.toInt();

    offset = points[1];
    int x2 = offset.dx.toInt();
    int y2 = offset.dy.toInt();

    offset = points[2];
    int x3 = offset.dx.toInt();
    int y3 = offset.dy.toInt();

    offset = points[3];
    int x4 = offset.dx.toInt();
    int y4 = offset.dy.toInt();
    Map? result = await methodChannel.invokeMapMethod<String, dynamic>(
      'normalize',
      {
        'file': file,
        'x1': x1,
        'y1': y1,
        'x2': x2,
        'y2': y2,
        'x3': x3,
        'y3': y3,
        'x4': x4,
        'y4': y4
      },
    );

    if (result != null) {
      return NormalizedImage(
        result['data'],
        result['width'],
        result['height'],
      );
    }

    return null;
  }

  @override
  Future<int?> save(String filename) async {
    return await methodChannel
        .invokeMethod<int>('save', {'filename': filename});
  }
}
Enter fullscreen mode Exit fullscreen mode

Linking Libraries and Writing C/C++ Code

Linking Shared Libraries in Flutter Plugin

Flutter uses CMake to build the Windows and Linux plugins. So the first step is to setting up the CMakeLists.txt file. We need to configure link_directories, target_link_libraries, and flutter_document_scan_sdk_bundled_libraries.

Windows

link_directories("${PROJECT_SOURCE_DIR}/lib/") 

target_link_libraries(${PLUGIN_NAME} PRIVATE flutter flutter_wrapper_plugin "DynamsoftCorex64" "DynamsoftDocumentNormalizerx64")

set(flutter_document_scan_sdk_bundled_libraries
  "${PROJECT_SOURCE_DIR}/bin/"
  PARENT_SCOPE
)
Enter fullscreen mode Exit fullscreen mode

Linux

link_directories("${PROJECT_SOURCE_DIR}/lib/") 

target_link_libraries(${PLUGIN_NAME} PRIVATE flutter "DynamsoftCore" "DynamsoftDocumentNormalizer")

set(flutter_document_scan_sdk_bundled_libraries

  PARENT_SCOPE
)
Enter fullscreen mode Exit fullscreen mode

The linking library files of Dynamsoft Document Normalizer SDK are DynamsoftCore and DynamsoftDocumentNormalizer. On Windows, they are named DynamsoftCorex64.lib and DynamsoftDocumentNormalizerx64.lib, whereas on Linux, they are named libDynamsoftCore.so and libDynamsoftDocumentNormalizer.so. To make Windows plugin work, take one more step to bundle the corresponding dll files.

Implementing Flutter C/C++ Code Logic for Windows and Linux

The native entry points of the plugin are FlutterDocumentScanSdkPlugin::HandleMethodCall and flutter_document_scan_sdk_plugin_handle_method_call respectively for Windows and Linux. Because the Flutter header files and data types are different between the two platforms, you have to write code accordingly.

Windows

#include "flutter_document_scan_sdk_plugin.h"

#include <windows.h>

#include <VersionHelpers.h>

#include <flutter/method_channel.h>
#include <flutter/plugin_registrar_windows.h>
#include <flutter/standard_method_codec.h>

#include <memory>
#include <sstream>

void FlutterDocumentScanSdkPlugin::HandleMethodCall(
      const flutter::MethodCall<flutter::EncodableValue> &method_call,
      std::unique_ptr<flutter::MethodResult<flutter::EncodableValue>> result){}
Enter fullscreen mode Exit fullscreen mode

Linux

#include "include/flutter_document_scan_sdk/flutter_document_scan_sdk_plugin.h"

#include <flutter_linux/flutter_linux.h>
#include <gtk/gtk.h>
#include <sys/utsname.h>

#include <cstring>

static void flutter_document_scan_sdk_plugin_handle_method_call(
    FlutterDocumentScanSdkPlugin *self,
    FlMethodCall *method_call){}
Enter fullscreen mode Exit fullscreen mode

To handle the method call, we need to parse the method name and arguments firstly.

Windows

const auto *arguments = std::get_if<EncodableMap>(method_call.arguments());

if (method_call.method_name().compare("init") == 0)
{
  std::string license;
  int ret = 0;

  if (arguments)
  {
    auto license_it = arguments->find(EncodableValue("key"));
    if (license_it != arguments->end())
    {
      license = std::get<std::string>(license_it->second);
    }
    ret = DocumentManager::SetLicense(license.c_str());
  }

  result->Success(EncodableValue(ret));
}
...
Enter fullscreen mode Exit fullscreen mode

Linux

if (strcmp(method, "init") == 0)
{
  if (fl_value_get_type(args) != FL_VALUE_TYPE_MAP)
  {
    return;
  }

  FlValue *value = fl_value_lookup_string(args, "key");
  if (value == nullptr)
  {
    return;
  }
  const char *license = fl_value_get_string(value);

  int ret = DocumentManager::SetLicense(license);
  g_autoptr(FlValue) result = fl_value_new_int(ret);
  response = FL_METHOD_RESPONSE(fl_method_success_response_new(result));
}
...
Enter fullscreen mode Exit fullscreen mode

As you can see, not only the APIs of parsing the method name are different, but also the data type of the arguments. For example, the license argument is std::string on Windows, whereas it is const char* on Linux.

Now, we create a document_manager.h file to implement the methods that calls Dynamsoft Document Normalizer API. Since Dynamsoft Document Normalizer allows you to write code in the same way on Windows and Linux, most of the code can be shared except for wrapping the return values as Flutter data type.

  • Import Dynamsoft Document Normalizer SDK header files: 
  #include "DynamsoftCore.h"
  #include "DynamsoftDocumentNormalizer.h"

  using namespace std;
  using namespace dynamsoft::ddn;
  using namespace dynamsoft::core;
Enter fullscreen mode Exit fullscreen mode
  • Initialize and destroy Dynamsoft Document Normalizer: 
  class DocumentManager
  {
  public:
      ~DocumentManager()
      {
          if (normalizer != NULL)
          {
              DDN_DestroyInstance(normalizer);
              normalizer = NULL;
          }

          FreeImage();
      };

      const char *GetVersion()
      {
          return DDN_GetVersion();
    }

    void Init()
    {
        normalizer = DDN_CreateInstance();
        imageResult = NULL;
    }
    private:
      void *normalizer;
      NormalizedImageResult *imageResult;

      void FreeImage()
      {
          if (imageResult != NULL)
          {
              DDN_FreeNormalizedImageResult(&imageResult);
              imageResult = NULL;
          }
      }
  };
Enter fullscreen mode Exit fullscreen mode 
  static int SetLicense(const char *license)
  {
      char errorMsgBuffer[512];
      // Click https://www.dynamsoft.com/customer/license/trialLicense/?product=ddn to get a trial license.
      int ret = DC_InitLicense(license, errorMsgBuffer, 512);
      if (ret != DM_OK)
      {
          cout << errorMsgBuffer << endl;
      }
      return ret;
  }
Enter fullscreen mode Exit fullscreen mode
  • Detect document edges by calling DDN_DetectQuadFromFile(). On Windows, the detected information is stored in EncodableMap, whereas on Linux, it is stored in FlValue created by fl_value_new_map().

Windows

  EncodableList Detect(const char *filename)
  {
      EncodableList out;
      if (normalizer == NULL)
          return out;

      DetectedQuadResultArray *pResults = NULL;

      int ret = DDN_DetectQuadFromFile(normalizer, filename, "", &pResults);
      if (ret)
      {
          printf("Detection error: %s\n", DC_GetErrorString(ret));
      }

      if (pResults)
      {
          int count = pResults->resultsCount;

          for (int i = 0; i < count; i++)
          {
              EncodableMap map;

              DetectedQuadResult *quadResult = pResults->detectedQuadResults[i];
              int confidence = quadResult->confidenceAsDocumentBoundary;
              DM_Point *points = quadResult->location->points;
              int x1 = points[0].coordinate[0];
              int y1 = points[0].coordinate[1];
              int x2 = points[1].coordinate[0];
              int y2 = points[1].coordinate[1];
              int x3 = points[2].coordinate[0];
              int y3 = points[2].coordinate[1];
              int x4 = points[3].coordinate[0];
              int y4 = points[3].coordinate[1];

              map[EncodableValue("confidence")] = EncodableValue(confidence);
              map[EncodableValue("x1")] = EncodableValue(x1);
              map[EncodableValue("y1")] = EncodableValue(y1);
              map[EncodableValue("x2")] = EncodableValue(x2);
              map[EncodableValue("y2")] = EncodableValue(y2);
              map[EncodableValue("x3")] = EncodableValue(x3);
              map[EncodableValue("y3")] = EncodableValue(y3);
              map[EncodableValue("x4")] = EncodableValue(x4);
              map[EncodableValue("y4")] = EncodableValue(y4);
              out.push_back(map);
          }
      }

      if (pResults != NULL)
          DDN_FreeDetectedQuadResultArray(&pResults);

      return out;
  }
Enter fullscreen mode Exit fullscreen mode

Linux

  FlValue* Detect(const char *filename)
  {
      FlValue* out = fl_value_new_list();
      if (normalizer == NULL)
          return out;

      DetectedQuadResultArray *pResults = NULL;

      int ret = DDN_DetectQuadFromFile(normalizer, filename, "", &pResults);
      if (ret)
      {
          printf("Detection error: %s\n", DC_GetErrorString(ret));
      }

      if (pResults)
      {
          int count = pResults->resultsCount;

          for (int i = 0; i < count; i++)
          {
              FlValue* result = fl_value_new_map ();

              DetectedQuadResult *quadResult = pResults->detectedQuadResults[i];
              int confidence = quadResult->confidenceAsDocumentBoundary;
              DM_Point *points = quadResult->location->points;
              int x1 = points[0].coordinate[0];
              int y1 = points[0].coordinate[1];
              int x2 = points[1].coordinate[0];
              int y2 = points[1].coordinate[1];
              int x3 = points[2].coordinate[0];
              int y3 = points[2].coordinate[1];
              int x4 = points[3].coordinate[0];
              int y4 = points[3].coordinate[1];

              fl_value_set_string_take (result, "confidence", fl_value_new_int(confidence));
              fl_value_set_string_take (result, "x1", fl_value_new_int(x1));
              fl_value_set_string_take (result, "y1", fl_value_new_int(y1));
              fl_value_set_string_take (result, "x2", fl_value_new_int(x2));
              fl_value_set_string_take (result, "y2", fl_value_new_int(y2));
              fl_value_set_string_take (result, "x3", fl_value_new_int(x3));
              fl_value_set_string_take (result, "y3", fl_value_new_int(y3));
              fl_value_set_string_take (result, "x4", fl_value_new_int(x4));
              fl_value_set_string_take (result, "y4", fl_value_new_int(y4));

              fl_value_append_take (out, result);
          }
      }

      if (pResults != NULL)
          DDN_FreeDetectedQuadResultArray(&pResults);

      return out;
  }
Enter fullscreen mode Exit fullscreen mode
  • Call DDN_NormalizeFile() to normalize the document based on detected corner coordinates: 
  Quadrilateral quad;
  quad.points[0].coordinate[0] = x1;
  quad.points[0].coordinate[1] = y1;
  quad.points[1].coordinate[0] = x2;
  quad.points[1].coordinate[1] = y2;
  quad.points[2].coordinate[0] = x3;
  quad.points[2].coordinate[1] = y3;
  quad.points[3].coordinate[0] = x4;
  quad.points[3].coordinate[1] = y4;

  int errorCode = DDN_NormalizeFile(normalizer, filename, "", &quad, &imageResult);
  if (errorCode != DM_OK)
      printf("%s\r\n", DC_GetErrorString(errorCode));

  if (imageResult)
  {
      ImageData *imageData = imageResult->image;
      int width = imageData->width;
      int height = imageData->height;
      int stride = imageData->stride;
      int format = (int)imageData->format;
      unsigned char* data = imageData->bytes;
      int orientation = imageData->orientation;
      int length = imageData->bytesLength;
  }
Enter fullscreen mode Exit fullscreen mode
  • Create a Flutter C++ map object to pass the normalized image data to Flutter:

Windows:

  EncodableMap map;
  map[EncodableValue("width")] = EncodableValue(width);
  map[EncodableValue("height")] = EncodableValue(height);
  map[EncodableValue("stride")] = EncodableValue(stride);
  map[EncodableValue("format")] = EncodableValue(format);
  map[EncodableValue("orientation")] = EncodableValue(orientation);
  map[EncodableValue("length")] = EncodableValue(length);
  std::vector<uint8_t> rawBytes(rgba, rgba + width * height * 4);
  map[EncodableValue("data")] = rawBytes;
Enter fullscreen mode Exit fullscreen mode

Linux:

  FlValue* result = fl_value_new_map ();
  fl_value_set_string_take (result, "width", fl_value_new_int(width));
  fl_value_set_string_take (result, "height", fl_value_new_int(height));
  fl_value_set_string_take (result, "stride", fl_value_new_int(stride));
  fl_value_set_string_take (result, "format", fl_value_new_int(format));
  fl_value_set_string_take (result, "orientation", fl_value_new_int(orientation));
  fl_value_set_string_take (result, "length", fl_value_new_int(length));
  fl_value_set_string_take (result, "data", fl_value_new_uint8_list(rgba, width * height * 4));
Enter fullscreen mode Exit fullscreen mode

Note: According to the image color format, we need to do the following conversion before wrapping the data:

  • If the output image is black and white, each byte represents 8 pixels. Each bit of a byte needs to be converted to a byte as 0 or 255. The final length of the image bytes should be the same as the grayscale image.

    unsigned char *grayscale = new unsigned char[width * height];
binary2grayscale(data, grayscale, width, height, stride, length);

void binary2grayscale(unsigned char* data, unsigned char* output, int width, int height, int stride, int length) 
{
    int index = 0;

    int skip = stride * 8 - width;
    int shift = 0;
    int n = 1;

    for (int i = 0; i < length; ++i)
    {
        unsigned char b = data[i];
        int byteCount = 7;
        while (byteCount >= 0)
        {
            int tmp = (b & (1 << byteCount)) >> byteCount;

            if (shift < stride * 8 * n - skip) {
                if (tmp == 1)
                    output[index] = 255;
                else
                    output[index] = 0;
                index += 1;
            }

            byteCount -= 1;
            shift += 1;
        }

        if (shift == stride * 8 * n) {
            n += 1;
        }
    }
}

  • The format of pixel data given to the Flutter decodeImageFromPixels() method only supports rgba8888 and bgra8888. So, we need to construct the pixel data in the format of rgba8888 or bgra8888 based on the output image color format.

    unsigned char *rgba = new unsigned char[width * height * 4];
memset(rgba, 0, width * height * 4);
if (format == IPF_RGB_888)
{
    int dataIndex = 0;
    for (int i = 0; i < height; i++)
    {
        for (int j = 0; j < width; j++)
        {
            int index = i * width + j;

            rgba[index * 4] = data[dataIndex + 2];     // red
            rgba[index * 4 + 1] = data[dataIndex + 1]; // green
            rgba[index * 4 + 2] = data[dataIndex];     // blue
            rgba[index * 4 + 3] = 255;                 // alpha
            dataIndex += 3;
        }
    }
}
else if (format == IPF_GRAYSCALED)
{
    int dataIndex = 0;
    for (int i = 0; i < height; i++)
    {
        for (int j = 0; j < width; j++)
        {
            int index = i * width + j;
            rgba[index * 4] = data[dataIndex];
            rgba[index * 4 + 1] = data[dataIndex];
            rgba[index * 4 + 2] = data[dataIndex];
            rgba[index * 4 + 3] = 255;
            dataIndex += 1;
        }
    }
}
else if (format == IPF_BINARY)
{
    unsigned char *grayscale = new unsigned char[width * height];
    binary2grayscale(data, grayscale, width, height, stride, length);

    int dataIndex = 0;
    for (int i = 0; i < height; i++)
    {
        for (int j = 0; j < width; j++)
        {
            int index = i * width + j;
            rgba[index * 4] = grayscale[dataIndex];
            rgba[index * 4 + 1] = grayscale[dataIndex];
            rgba[index * 4 + 2] = grayscale[dataIndex];
            rgba[index * 4 + 3] = 255;
            dataIndex += 1;
        }
    }

    free(grayscale);
}

So far, we have completed the implementation of the Flutter Document Scan SDK plugin for Windows and Linux. The next step is to test the plugin on desktop platforms.

Test Flutter Document Scan SDK Plugin on Windows and Linux

Since the example code we implemented for web is shared with Windows and Linux, there is no code change required. We can directly run the example code on Windows and Linux.

Windows

flutter run -d windows
Enter fullscreen mode Exit fullscreen mode

Flutter windows document edge detection and normalization

Linux

flutter run -d linux
Enter fullscreen mode Exit fullscreen mode

Flutter Linux document edge detection and normalization

Source Code

https://github.com/yushulx/flutter_document_scan_sdk

Image
How To Beat Your Boss Mesothelioma Asbestos Cancer
Image
What How Much Is A Spare Car Key Experts Would Like You To Know
Image
14 Cartoons About Mental Health UK To Brighten Your Day
Image
How Do I Explain Gas Certified Engineer To A 5-Year-Old
Image
10 Amazing Graphics About Accident Lawyer Dallas
Image
Kanapy Marzeń: Twój Klucz do Komfortowego Wnętrza!
Image
The Reasons To Focus On Improving Accident Lawyer Dallas
Image
NxWallet: The Crypto Asset Management Tool for the Future
Image
setw() и setfill C++
Image
The Reasons Why Spare Car Keys Cut Has Become The Obsession Of Everyone In 2023
Image
So , You've Purchased Best Accident Lawyer Near Me ... Now What?
Image
Strategi Mendapat Beberapa ratus Juta Dari Judi Online
Image
10 Quick Tips On Asbestos Lawyers Near Me
Image
Kanapy, które zmieniają każde wnętrze: Komfort i styl w jednym!
Image
10 Books To Read On Asbestos Lawsuit
Image
The History Of Asbestos Compensation After Death In 10 Milestones
Image
apa yang memberikan keuntungan dari judi slots online?
Image
A The Complete Guide To Recessed In-Wall Electric Fireplace From Beginning To End
Image
Who Is Responsible For An Electric Fireplace Wall Unit Budget? 12 Ways To Spend Your Money
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Terabox Video Player