Implementing Flutter QR Code Scanner with Swift and AVFoundation for iOS

Xiao Ling - May 8 - - Dev Community

In the previous article, we implemented a Flutter barcode and QR code scanner for Android using Kotlin and CameraX. Since the Dart code is platform-independent, no changes are necessary. In this article, we will take steps to implement the native camera and barcode scanning logic for iOS using Swift, AVFoundation, and the Dynamsoft Barcode Reader SDK.

Prerequisites

Step 1: Installing Dynamsoft Barcode Reader for iOS

We use CocoaPods to install the Dynamsoft Barcode Reader SDK for iOS. If you haven't installed CocoaPods yet, please follow the official instructions here to do so.

Once CocoaPods is ready, create a Podfile in the iOS folder of your Flutter project:

cd ios
pod init
Enter fullscreen mode Exit fullscreen mode

Next, edit the Podfile to include the Dynamsoft Barcode Reader SDK:

target 'Runner' do
  use_frameworks!

  pod 'DynamsoftBarcodeReader','9.6.40'

end
Enter fullscreen mode Exit fullscreen mode

Save the Podfile and run pod install. This command will install or update the CocoaPods dependencies, including the Flutter framework required for your iOS project.

Step 2: Adding Camera Permission to Info.plist

To enable camera access on iOS, open the Info.plist file located in the ios/Runner folder. Add the following keys:

<key>NSCameraUsageDescription</key>
<string>your usage description here</string>
<key>NSMicrophoneUsageDescription</key>
<string>your usage description here</string>
Enter fullscreen mode Exit fullscreen mode

Step 3: Implementing Camera Preview with Flutter Texture in Swift

The Runner/AppDelegate.swift file serves as the entry point of the Flutter application. By default, it contains the following boilerplate code:

override func application(
    _ application: UIApplication,
    didFinishLaunchingWithOptions launchOptions: [UIApplication.LaunchOptionsKey: Any]?
  ) -> Bool {
    GeneratedPluginRegistrant.register(with: self)
    return super.application(application, didFinishLaunchingWithOptions: launchOptions)
  }
Enter fullscreen mode Exit fullscreen mode

To integrate the camera functionality effectively, you'll need to enhance the application method with the following steps:

  1. Establish a Flutter method channel. This is crucial for seamless communication between the Dart environment and Swift, allowing commands and data to be exchanged between the Flutter UI and native code.
  2. Implement a startCamera() method. This method should initiate the camera preview and continuously render this preview into a Flutter texture. This involves setting up the camera capture session, configuring input and output, and linking the camera output to a Flutter texture that can be displayed in the UI.

Flutter Method Channel in Swift

The Flutter method channel is a named channel that facilitates the sending of data between Dart and platform-specific code.

private var channel: FlutterMethodChannel?
private var width = 1920
private var height = 1080

override func application(
    _ application: UIApplication,
    didFinishLaunchingWithOptions launchOptions: [UIApplication.LaunchOptionsKey: Any]?
  ) -> Bool {

    channel = FlutterMethodChannel(name: CHANNEL, binaryMessenger: flutterViewController.binaryMessenger)
    channel?.setMethodCallHandler({
      (call: FlutterMethodCall, result: @escaping FlutterResult) -> Void in
      if call.method == "startCamera" {
        self.startCamera(result: result)
      } else if call.method == "getPreviewWidth" {
        result(self.width)
      } else if call.method == "getPreviewHeight" {
        result(self.height)
      }

      else {
        result(FlutterMethodNotImplemented)
      }
    })

    GeneratedPluginRegistrant.register(with: self)
    return super.application(application, didFinishLaunchingWithOptions: launchOptions)
  }
Enter fullscreen mode Exit fullscreen mode
  • The channel variable is an instance of FlutterMethodChannel. It receives method calls from Dart using the setMethodCallHandler method. Additionally, the invokeMethod method is used to send data from Swift to Dart.
  • The width and height variables, which store the camera preview size, are hardcoded to 1920x1080 here. The methods getPreviewWidth and getPreviewHeight retrieve and return these values to Dart, respectively.

Creating Flutter Texture and Camera Preview

Define the CustomCameraTexture class that extends NSObject and implements the FlutterTexture protocol:

class CustomCameraTexture: NSObject, FlutterTexture {
  private weak var textureRegistry: FlutterTextureRegistry?
  var textureId: Int64?
  private var cameraPreviewLayer: AVCaptureVideoPreviewLayer?
  private let bufferQueue = DispatchQueue(label: "com.example.flutter/barcode_scan")
  private var _lastSampleBuffer: CMSampleBuffer?
  private var customCameraTexture: CustomCameraTexture?

  private var lastSampleBuffer: CMSampleBuffer? {
    get {
      var result: CMSampleBuffer?
      bufferQueue.sync {
        result = _lastSampleBuffer
      }
      return result
    }
    set {
      bufferQueue.sync {
        _lastSampleBuffer = newValue
      }
    }
  }

  init(cameraPreviewLayer: AVCaptureVideoPreviewLayer, registry: FlutterTextureRegistry) {
    self.cameraPreviewLayer = cameraPreviewLayer
    self.textureRegistry = registry
    super.init()
    self.textureId = registry.register(self)
  }

  func copyPixelBuffer() -> Unmanaged<CVPixelBuffer>? {
    guard let sampleBuffer = lastSampleBuffer, let pixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer) else {
      return nil
    }

    return Unmanaged.passRetained(pixelBuffer)
  }

  func update(sampleBuffer: CMSampleBuffer) {
    lastSampleBuffer = sampleBuffer
    textureRegistry?.textureFrameAvailable(textureId!)
  }

  deinit {
    if let textureId = textureId {
      textureRegistry?.unregisterTexture(textureId)
    }
  }
}
Enter fullscreen mode Exit fullscreen mode
  • The textureRegistry variable is an instance of FlutterTextureRegistry. It is used to register and unregister the Flutter texture.
  • The textureId variable stores the Flutter texture ID, which will be used to render the camera preview in Flutter.
  • The copyPixelBuffer() method returns the latest pixel buffer for texture rendering.
  • The update() method appends a new camera frame and then notifies the Flutter texture that it needs to be updated. When the textureFrameAvailable() method is invoked, the copyPixelBuffer() method is triggered to fetch the latest pixel buffer.

Create a flutterTextureEntry variable in the AppDelegate class and obtain the Flutter texture registry within the application method:

private var flutterTextureEntry: FlutterTextureRegistry?

override func application(
    _ application: UIApplication,
    didFinishLaunchingWithOptions launchOptions: [UIApplication.LaunchOptionsKey: Any]?
  ) -> Bool {
    ...

    guard let flutterViewController = window?.rootViewController as? FlutterViewController else {
      return super.application(application, didFinishLaunchingWithOptions: launchOptions)
    }
    flutterTextureEntry = flutterViewController.engine!.textureRegistry

    GeneratedPluginRegistrant.register(with: self)
    return super.application(application, didFinishLaunchingWithOptions: launchOptions)
  }
Enter fullscreen mode Exit fullscreen mode

In the startCamera() initiate a camera session and add a video data output to capture the camera frames. When a new frame is captured in the captureOutput() method, update the CustomCameraTexture instance with the latest sample buffer:

private func startCamera(result: @escaping FlutterResult) {
    if cameraSession != nil {
      result(self.customCameraTexture?.textureId)
      return
    }

    cameraSession = AVCaptureSession()
    cameraSession?.sessionPreset = .hd1920x1080

    guard let backCamera = AVCaptureDevice.default(for: .video), let input = try? AVCaptureDeviceInput(device: backCamera) else {
      result(FlutterError(code: "no_camera", message: "No camera available", details: nil))
      return
    }

    cameraSession?.addInput(input)
    cameraPreviewLayer = AVCaptureVideoPreviewLayer(session: cameraSession!)
    cameraPreviewLayer?.videoGravity = .resizeAspectFill

    let cameraOutput = AVCaptureVideoDataOutput()
    cameraOutput.videoSettings = [kCVPixelBufferPixelFormatTypeKey as String: kCVPixelFormatType_32BGRA]
    cameraOutput.setSampleBufferDelegate(self, queue: DispatchQueue(label: "camera_frame_queue"))
    cameraSession?.addOutput(cameraOutput)

    self.customCameraTexture = CustomCameraTexture(cameraPreviewLayer: cameraPreviewLayer!, registry: flutterTextureEntry!)
    cameraSession?.startRunning()

    result(self.customCameraTexture?.textureId)
  }

  func captureOutput(_ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, from connection: AVCaptureConnection) {
    if connection.isVideoOrientationSupported {
      connection.videoOrientation = currentVideoOrientation()
    }
    self.customCameraTexture?.update(sampleBuffer: sampleBuffer)
  }
Enter fullscreen mode Exit fullscreen mode

At this point, the camera preview should function correctly on iOS/iPadOS. Next, we will integrate the Dynamsoft Barcode Reader SDK to decode barcodes and QR codes.

Step 4: Integrating Dynamsoft Barcode Reader SDK for iOS in Swfit

  1. Import the Dynamsoft Barcode Reader SDK in the AppDelegate.swift file:

    import DynamsoftBarcodeReader

  2. Create an instance of Dynamsoft Barcode Reader and activate it with a valid license key:

    @UIApplicationMain
@objc class AppDelegate: FlutterAppDelegate, AVCaptureVideoDataOutputSampleBufferDelegate, DBRLicenseVerificationListener {

private let reader = DynamsoftBarcodeReader()

override func application(
    _ application: UIApplication,
    didFinishLaunchingWithOptions launchOptions: [UIApplication.LaunchOptionsKey: Any]?
  ) -> Bool {
    DynamsoftBarcodeReader.initLicense("LICENSE-KEY", verificationDelegate: self)

    do {
      let settings = try? reader.getRuntimeSettings()
      settings!.expectedBarcodesCount = 999
      try reader.updateRuntimeSettings(settings!)
    } catch {
      print("Error getting runtime settings")
    }

    ...

    GeneratedPluginRegistrant.register(with: self)
    return super.application(application, didFinishLaunchingWithOptions: launchOptions)
  }

    func dbrLicenseVerificationCallback(_ isSuccess: Bool, error: Error?) {
        if isSuccess {
        print("License verification passed")
        } else {
        print("License verification failed: \(error?.localizedDescription ?? "Unknown error")")
        }
    }
}

  3. Decode barcode and QR code from the camera frame in the captureOutput() method. Since the decoding API is CPU-intensive, to avoid blocking the camera preview rendering, we move the decoding logic to a separate thread:

    func captureOutput(_ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, from connection: AVCaptureConnection) {
    if connection.isVideoOrientationSupported {
      connection.videoOrientation = currentVideoOrientation()
    }
    self.customCameraTexture?.update(sampleBuffer: sampleBuffer)

    if !isProcessing {
      isProcessing = true
      DispatchQueue.global(qos: .background).async {
        self.processImage(sampleBuffer)
        self.isProcessing = false
      }
    }
  }

    The isProcessing boolean variable ensures that only one frame is processed at a time, and new frames are ignored until the processing is complete. This approach helps mitigate the accumulation of asynchronous tasks and prevents the app from crashing due to memory exhaustion.

  4. Implement the processImage() method to decode barcodes from CMSampleBuffer and send the results to the Flutter UI via the method channel:

    func processImage(_ sampleBuffer: CMSampleBuffer) {
    let imageBuffer:CVImageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer)!
    CVPixelBufferLockBaseAddress(imageBuffer, .readOnly)
    let baseAddress = CVPixelBufferGetBaseAddress(imageBuffer)
    let bufferSize = CVPixelBufferGetDataSize(imageBuffer)
    let width = CVPixelBufferGetWidth(imageBuffer)
    let height = CVPixelBufferGetHeight(imageBuffer)
    let bpr = CVPixelBufferGetBytesPerRow(imageBuffer)
    CVPixelBufferUnlockBaseAddress(imageBuffer, .readOnly)
    let buffer = Data(bytes: baseAddress!, count: bufferSize)

    let imageData = iImageData.init()
    imageData.bytes = buffer
    imageData.width = width
    imageData.height = height
    imageData.stride = bpr
    imageData.format = .ARGB_8888
    imageData.orientation = 0

    let results = try? reader.decodeBuffer(imageData)
    DispatchQueue.main.async {
      self.channel?.invokeMethod("onBarcodeDetected", arguments: self.wrapResults(results: results))
    }
  }

  func wrapResults(results:[iTextResult]?) -> NSArray {
        let outResults = NSMutableArray(capacity: 8)
        if results == nil {
            return outResults
        }
        for item in results! {
            let subDic = NSMutableDictionary(capacity: 11)
            if item.barcodeFormat_2 != EnumBarcodeFormat2.Null {
                subDic.setObject(item.barcodeFormatString_2 ?? "", forKey: "format" as NSCopying)
            }else{
                subDic.setObject(item.barcodeFormatString ?? "", forKey: "format" as NSCopying)
            }
            let points = item.localizationResult?.resultPoints as! [CGPoint]
            subDic.setObject(Int(points[0].x), forKey: "x1" as NSCopying)
            subDic.setObject(Int(points[0].y), forKey: "y1" as NSCopying)
            subDic.setObject(Int(points[1].x), forKey: "x2" as NSCopying)
            subDic.setObject(Int(points[1].y), forKey: "y2" as NSCopying)
            subDic.setObject(Int(points[2].x), forKey: "x3" as NSCopying)
            subDic.setObject(Int(points[2].y), forKey: "y3" as NSCopying)
            subDic.setObject(Int(points[3].x), forKey: "x4" as NSCopying)
            subDic.setObject(Int(points[3].y), forKey: "y4" as NSCopying)
            subDic.setObject(item.localizationResult?.angle ?? 0, forKey: "angle" as NSCopying)
            subDic.setObject(item.barcodeBytes ?? "", forKey: "barcodeBytes" as NSCopying)
            outResults.add(subDic)
        }

        return outResults
    }

Running the Flutter QR Code Scanner on iOS 

flutter run
Enter fullscreen mode Exit fullscreen mode

Flutter iOS QR code scanner

Source Code

https://github.com/yushulx/flutter-barcode-scanner/tree/main/examples/native_camera

Image
Rahasia Togel: Menembus Angka dan Mencetak Keberuntungan
Image
The Ultimate Guide To Bifold Door Repairs
Image
Apa Jadi Bandar Judi Online itu Simpel?
Image
What Is Asbestos Payout? Heck Is Asbestos Payout?
Image
10 Things You Learned In Kindergarden That Will Help You Get Accident Lawyers In My Area
Image
What You Should Be Focusing On Enhancing Asbestos Lawyer
Image
mengapa mesti main slots online sah terunggul?
Image
The Most Underrated Companies To Watch In Accident Lawyer In Houston Industry
Image
Lawyer Injury Accident Explained In Fewer Than 140 Characters
Image
Langkah Menang Taruhan Online Buat Beberapa Pemain Pemula
Image
10 No-Fuss Ways To Figuring The Accident Injury Lawyers Near Me You're Looking For
Image
Cat Flap Installers Near Me
Image
The Most Hilarious Complaints We've Seen About Truck Key
Image
This is a test.
Image
For making an application.

development, programming, webdev, beginners
Image
10 Accident Lawyer Tricks All Experts Recommend
Image
14 Companies Doing An Excellent Job At Lost Truck Key
Image
10 Websites To Aid You Learn To Be An Expert In Asbestos Cancer Lawsuit Lawyer Mesothelioma
Image
Is there an Alternative to Debezium + Kafka?

data, architecture, eventdriven, kafka
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Terabox Video Player