How to Turn Smartphone into a Peripheral Keyboard and Barcode QR Scanner in Flutter

Xiao Ling - Feb 23 '23 - - Dev Community

Barcode scanning technology is crucial for many enterprises, such as those in the supermarket, logistics, and warehousing industries. The data obtained from scanning barcodes is used to track inventory, manage orders, and process payments. Handheld barcode scanners are widely used as peripheral devices in these industries. However, traditional handheld laser barcode scanners can only detect one-dimensional codes, while more expensive scanners with an embedded camera are needed to detect two-dimensional codes like QR codes. Smartphones with built-in cameras can detect both one-dimensional and two-dimensional codes. Nowadays, smartphones have become ubiquitous. Everyone has at least one smartphone in their pocket. If smartphone can do the same job as a handheld barcode scanner, no additional hardware is needed. This article presents a solution that combines technologies like Flutter, Python, Bonjour, web sockets, PyAutoGUI, and Dynamsoft Barcode Reader SDK to help enterprises save costs by using smartphones instead of barcode scanners.

no handheld barcode scanner

Supported Platforms

  • Android
  • iOS

Demo Video: Smartphone as a Peripheral Keyboard and Barcode Scanner

The demo shows how to input text from an Android phone to Windows notepad and macOS notes simultaneously.

Overview of the Utilized Technologies and Why They Are Used

Before diving into the details of the solution, let’s first take a look at the technologies used in this solution.

  • Flutter: Flutter is a cross-platform UI framework. It enables developers build native applications for Windows, Linux, macOS, Android, iOS and web from a single codebase. Using Flutter can save a lot of time and effort for building the client user interface.
  • Python: With Python, it only takes a few lines of code to implement network services and simulate keyboard input on operating systems.
  • Bonjour: Bonjour is a zero-configuration networking (zeroconf) protocol developed by Apple. It allows devices on the same network to discover each other. In this solution, Bonjour is used to discover the web socket server on the same network.
  • WebSocket: Web sockets are a computer communications protocol that allows for bidirectional communication between a client and a server. In this solution, web sockets are used to communicate between mobile devices and PCs.
  • PyAutoGUI: PyAutoGUI is a Python module that can be used to automate mouse movements and keyboard presses. In this solution, PyAutoGUI is used to simulate keyboard input.
  • Dynamsoft Barcode Reader SDK: Dynamsoft Barcode Reader SDK is a cross-platform barcode recognition SDK that can be used to recognize 1D and 2D barcodes in images. In this solution, Dynamsoft Barcode Reader SDK is used to recognize barcodes in images captured by the smartphone camera.

Building a Discoverable Web Socket Server with Bonjour in Python

Let's get started with the server-side implementation. The server-side implementation is written in Python.

Install the Required Python Modules 

pip install pyautogui websockets zeroconf
Enter fullscreen mode Exit fullscreen mode

Get the Local IP Address in Python

There may be multiple network interfaces on your computer. A simple way to list all of them is to use the ipconfig command on Windows or the ifconfig command on Linux and macOS. The Python subprocess.run() method can be used to execute the command and get the output. The desired IP address can be obtained from the output by using a regular expression, such as r'(\d+\.\d+\.\d+\.\d+)'.

import subprocess
import re
import sys

def get_ip_address():
    ip_address = ''
    if sys.platform == 'win32':
        result = subprocess.run(['ipconfig', '/all'], capture_output=True, text=True)   
    else:
        result = subprocess.run(['ifconfig'], capture_output=True, text=True)

    foundEthernet = False
    for line in result.stdout.split('\n'):
        if sys.platform == 'win32':
            match = re.search(r'Ethernet adapter Ethernet:', line)
            if match:
                foundEthernet = True

            if foundEthernet:
                match = re.search(r'IPv4 Address.*? : (\d+\.\d+\.\d+\.\d+)', line)
                if match:
                    ip_address = match.group(1)
                    break
        else:
            if sys.platform == "linux" or sys.platform == "linux2":
                match = re.search(r'eth0:', line)
            else:
                match = re.search(r'en0:', line)
            if match:
                foundEthernet = True

            if foundEthernet:
                match = re.search(r'(\d+\.\d+\.\d+\.\d+)', line)
                if match:
                    ip_address = match.group(1)
                    break

    return ip_address
Enter fullscreen mode Exit fullscreen mode

Note: You may need to modify the regular expression used in the above code to match the network interface name on your computer.

Implement the Web Socket Server

After obtaining the local IP address, the web socket server can be implemented with asyncio and websockets.

import asyncio
import websockets
import socket
import pyautogui
import signal

connected = set()
isShutdown = False

async def handle_message(websocket, path):
    async for message in websocket:
        if isinstance(message, str):
            pass

        elif isinstance(message, bytes):
            pass            


async def server(websocket, path):
    connected.add(websocket)
    try:
        await handle_message(websocket, path)
    except:
        connected.remove(websocket)


def ctrl_c(signum, frame):
    global isShutdown
    isShutdown = True

async def main(ip_address):
    global isShutdown
    # Start the server
    s = await websockets.serve(server, ip_address, 4000)

    while not isShutdown:
        await asyncio.sleep(1)

    s.close()
    await s.wait_closed()

signal.signal(signal.SIGINT, ctrl_c)
asyncio.run(main(ip_address_str))
Enter fullscreen mode Exit fullscreen mode

We use signal.signal() to handle keyboard input for interrupting the infinite loop of the server.

Simulate Keyboard Input with PyAutoGUI

Once a message is received from the mobile client, the server can simulate keyboard input with PyAutoGUI.

if message != '':
    if message == 'backspace':
        pyautogui.press('backspace')
    elif message == 'enter':
        pyautogui.press('enter')
    else:
        pyautogui.typewrite(message)
Enter fullscreen mode Exit fullscreen mode

The pyautogui.press method can be used to simulate pressing a key on the keyboard. The pyautogui.typewrite method can be used to simulate typing text on the keyboard.

Integrate Bonjour into the Web Socket Server

The last step is to make the web socket server discoverable with Bonjour. The zeroconf module can be used to integrate Bonjour into the web socket server.


from zeroconf import ServiceBrowser, ServiceInfo, ServiceListener, Zeroconf

class MyListener(ServiceListener):

    def update_service(self, zc: Zeroconf, type_: str, name: str) -> None:
        print(f"Service {name} updated")

    def remove_service(self, zc: Zeroconf, type_: str, name: str) -> None:
        print(f"Service {name} removed")

    def add_service(self, zc: Zeroconf, type_: str, name: str) -> None:
        info = zc.get_service_info(type_, name)
        print(f"Service {name} added, service info: {info}")

ip_address_str = get_ip_address()
ip_address = socket.inet_aton(ip_address_str)

info = ServiceInfo("_bonsoirdemo._tcp.local.", socket.gethostname().split('.')[0] +
                " Web Socket Server._bonsoirdemo._tcp.local.",
                port=7000, addresses=[ip_address])

zeroconf = Zeroconf()
# Register the Bonjour service
zeroconf.register_service(info)
listener = MyListener()
browser = ServiceBrowser(zeroconf, "_bonsoirdemo._tcp.local.", listener)

# Start the web socket server
signal.signal(signal.SIGINT, ctrl_c)
asyncio.run(main(ip_address_str))

# Unregister the Bonjour service
print("Unregistering Bonjour service")
zeroconf.unregister_service(info)
zeroconf.close()
Enter fullscreen mode Exit fullscreen mode

The socket.gethostname() may return a host name that contains a . inside. The . must be filtered out, otherwise the Bonjour service name will be invalid.

Building a Mobile App with Flutter, Bonjour, WebSocket and Dynamsoft Barcode Reader SDK

In this section, we will build a mobile app with Flutter, Bonjour, WebSocket and Dynamsoft Barcode Reader SDK. The mobile app can automatically discover the web socket server and send barcode data via the web socket connection.

Install the Required Flutter Packages 

flutter pub add bonsoir flutter_riverpod device_info web_socket_channel dynamsoft_capture_vision_flutter
Enter fullscreen mode Exit fullscreen mode
  • For Android, change the minimum SDK version to 21 in android/app/build.gradle: 
  defaultConfig {
      minSdkVersion 21
  }
Enter fullscreen mode Exit fullscreen mode
  • For iOS, add the following description to ios/Runner/Info.plist: 
  <key>NSCameraUsageDescription</key>
  <string>Can I use the camera please?</string>
  <key>NSLocalNetworkUsageDescription</key>
  <string>Looking for local tcp Bonjour service</string>
  <key>NSBonjourServices</key>
  <array>
    <string>_bonsoirdemo._tcp</string>
Enter fullscreen mode Exit fullscreen mode

The value of NSBonjourServices must be the same as the service type of your Bonjour service.

Implement Web Socket Connection and Bonjour Service Discovery in Flutter

We use Bonjour service discovery because we want to avoid manually entering the IP address of the web socket server.

The example code of bonsoir is helpful for implementing the Bonjour service discovery. We can copy app_service.dart, discovery.dart, and service_list.dart files from the example code to our project, and then modify them to meet our needs.

  • app_service.dart: Change the service type to _bonsoirdemo._tcp. 

  static const String type = '_bonsoirdemo._tcp';
Enter fullscreen mode Exit fullscreen mode
  • discovery.dart: No change is needed.
  • service_list.dart: Add an elevated button to trigger the web socket connection. 
  class ItemWidgetState extends State<ItemWidget> {
    IOWebSocketChannel? _channel;
    String _connectAction = 'Connect';
    bool _connected = false;

    void _connect(String msg) {
      if (_connected) {
        print('disconnect to $msg');
        _channel!.sink.close(status.goingAway);
        channels.remove(_channel!);
        _connected = false;
        _connectAction = 'Connect';
        setState(() {});
        return;
      }

      _channel = IOWebSocketChannel.connect('ws://$msg');

      _channel!.ready.then((_) {
        channels.add(_channel!);
        print('connected to $msg');
        _connected = true;
        _connectAction = 'Disconnect';

        setState(() {});
        _channel!.stream.listen((message) {
          print('received: $message');
        }, onError: (error) {
          print('error: $error');
        }, onDone: () {
          if (_channel!.closeCode != null) {
            print('WebSocket is closed with code: ${_channel!.closeCode}');
          } else {
            print('WebSocket is closed');
          }
          _connected = false;
          _connectAction = 'Connect';
          setState(() {});
        });
      });
    }

    @override
    void dispose() {
      if (_channel != null) {
        _channel!.sink.close(status.goingAway);
        channels.remove(_channel!);
      }
      super.dispose();
    }

    @override
    Widget build(BuildContext context) {
      return Padding(
        padding: const EdgeInsets.all(8.0),
        child: Row(
          children: [
            Expanded(
              child: ListTile(
                title: Text(widget.service.name),
                subtitle: Text('IP : ${widget.service.ip}, Port: 4000'),
              ),
            ),
            ElevatedButton(
              onPressed: () {
                _connect('${widget.service.ip}:4000');
              },
              child: Text(_connectAction),
            ),
          ],
        ),
      );
    }
  }
Enter fullscreen mode Exit fullscreen mode

Implement Barcode QR Code Scanning in Flutter

The dynamsoft_capture_vision_flutter package also contains excellent example code, that guides you how to use the API of the Dynamsoft Barcode Reader SDK in Flutter.

Here are the basic steps to implement barcode QR code scanning in Flutter:

  1. Initialize the SDK with a license key.

    late final DCVCameraEnhancer _cameraEnhancer;
late final DCVBarcodeReader _barcodeReader;

final DCVCameraView _cameraView = DCVCameraView();

@override
void initState() {
  super.initState();
  WidgetsBinding.instance.addObserver(this);
  _sdkInit();
}

Future<void> _sdkInit() async {
  try {
    await DCVBarcodeReader.initLicense(
        'DLS2eyJoYW5kc2hha2VDb2RlIjoiMjAwMDAxLTE2NDk4Mjk3OTI2MzUiLCJvcmdhbml6YXRpb25JRCI6IjIwMDAwMSIsInNlc3Npb25QYXNzd29yZCI6IndTcGR6Vm05WDJrcEQ5YUoifQ==');
  } catch (e) {
    print(e);
  }
  _barcodeReader = await DCVBarcodeReader.createInstance();
  _cameraEnhancer = await DCVCameraEnhancer.createInstance();

  DBRRuntimeSettings currentSettings =
      await _barcodeReader.getRuntimeSettings();
  currentSettings.barcodeFormatIds =
      EnumBarcodeFormat.BF_ONED | EnumBarcodeFormat.BF_QR_CODE;

  currentSettings.expectedBarcodeCount = 0;
  await _barcodeReader
      .updateRuntimeSettingsFromTemplate(EnumDBRPresetTemplate.DEFAULT);
  await _barcodeReader.updateRuntimeSettings(currentSettings);

  _cameraEnhancer.setScanRegion(Region(
      regionTop: 30,
      regionLeft: 15,
      regionBottom: 70,
      regionRight: 85,
      regionMeasuredByPercentage: 1));

  _cameraView.overlayVisible = true;

  _cameraView.torchButton = TorchButton(
    visible: true,
  );

  await _barcodeReader.enableResultVerification(true);

  _barcodeReader.receiveResultStream().listen((List<BarcodeResult>? res) {
    if (mounted) {
      decodeRes = res ?? [];

      setState(() {});
    }
  });

  await _cameraEnhancer.open();

  _barcodeReader.startScanning();
}

  2. Add the camera view to the widget tree.

    Widget listItem(BuildContext context, int index) {
  BarcodeResult res = decodeRes[index];

  return ListTileTheme(
      textColor: Colors.white,
      // tileColor: Colors.green,
      child: ListTile(
        title: Text(res.barcodeFormatString),
        subtitle: Text(res.barcodeText),
      ));
}

@override
Widget build(BuildContext context) {
  return Scaffold(
      appBar: AppBar(
        title: const Text('Barcode Scanner'),
      ),
      body: Stack(
        children: [
          Container(
            child: _cameraView,
          ),
          SizedBox(
            height: 100,
            child: ListView.builder(
              itemBuilder: listItem,
              itemCount: decodeRes.length,
            ),
          ),
        ],
      ));
}

  3. Send the barcode result to the server.

    Set<IOWebSocketChannel> channels = {};

void sendMessage(String msg) {
  if (channels.isEmpty) {
    return;
  }

  for (final channel in channels) {
    channel.sink.add(msg);
  }
}

_barcodeReader.receiveResultStream().listen((List<BarcodeResult>? res) {
  if (mounted) {
    decodeRes = res ?? [];
    String msg = '';
    for (var i = 0; i < decodeRes.length; i++) {
      msg += '${decodeRes[i].barcodeText}\n';
    }
    if (msg.isNotEmpty && _ready) {
      _ready = false;
      sendMessage(msg);

      Future.delayed(const Duration(seconds: 2), () async {
        _ready = true;
      });
    }

    setState(() {});
  }
});

You can now run the app on your Android or iOS device.

flutter run
Enter fullscreen mode Exit fullscreen mode

flutter barcode qr code scanner

flutter bonjour web socket

Source Code

https://github.com/yushulx/flutter-peripheral-keyboard-barcode-scanner

Image
How to Route Multiple Paystack Webhooks with one Webhook URL

webdev, javascript, beginners, tutorial
Image
I created a video tutorial on how to make a responsive sidebar using CSS and vanilla Javascript

webdev, nav, css, javascript
Image
Tìm hiểu về lưới thép hàn: Cấu tạo, ưu điểm và ứng dụng
Image
Declutter Your Space: The best Guide to Junk Removal Services inside Raleigh, NC
Image
Filma24 my
Image
Building a Single-Page App with a New Paradigm Using TargetJ
Image
Doors With Cat Flap
Image
10 Meetups On Bioethanol Freestanding Stove You Should Attend
Image
5 Essential Software Metrics for Quality Assurance

webdev, javascript, programming, scrum
Image
20 Myths About Freestanding Bioethanol Fireplace: Dispelled
Image
upvc Door Panels With Cat Flap
Image
Remodel Your Space: The supreme Guide to Junk Removal Services within Raleigh, NC
Image
Fitting a Cat Flap
Image
langkah transaksi bisnis slots online yang umum dipakai
Image
Simple Pop-Up Newsletter Project

webdev, javascript, css, programming
Image
The Smart POS Benefit: Real-Time Analytics for Your Organization
Image
Subsequent Page Load Optimization 🚀

webdev, javascript, webperf, website
Image
5 Laws Everyone Working In Light Wheelchairs Folding Should Be Aware Of
Image
UPVC Door Panel With Cat Flap
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Terabox Video Player