Open Text Shield (OTS)

Ameed Jamous - Sep 19 - - Dev Community

Open Text Shield (OTS) is an open-source, real-time, high-performance filtering and classification tool for SMS and messaging traffic, designed to deliver responses in under a second on standard hardware. It detects and blocks spam, phishing, and malicious content in 10 languages, making it a powerful solution for telecom operators, messaging providers, and any application handling message traffic, including those using SMPP, SIP, RCS, or SMTP.

OTS version 2.1 now proudly offers text classification predictions across 10 diverse languages, reaffirming our commitment to breaking language barriers and enhancing global communication:

🇬🇧 English 🇸🇦 Arabic 🇮🇩 Indonesian 🇩🇪 German 🇮🇹 Italian 🇪🇸 Spanish 🇷🇺 Russian 🇫🇷 French 🇱🇰 Sinhala 🇮🇳 Tamil

Features

  • Real-Time Detection: Instantly classify and block spam, phishing, and other unwanted messages.
  • Machine Learning Models: Leverages advanced models like multi-lingual BERT to ensure high accuracy.
  • Flexible Integration: Compatible with any application capable of making HTTP requests, including those using SMPP, SIP, RCS, and SMTP.
  • Web API: Simple and efficient API for message analysis and predictions.
  • Open Source: Free to use and modify, with commercial support available for advanced use cases.
  • Quick Start with Docker

Setting up Open Text Shield is quick and easy with Docker. Follow these steps to get started:

Note: Our Dockerized version currently supports only ARM architecture. We plan to release a version for AMD/Intel architectures soon.

  1. Pull the Latest Docker Image

shell docker pull telecomsxchange/opentextshield:latest

  1. Run the Docker Container

shell docker run -d -p 8002:8002 telecomsxchange/opentextshield:latest

  1. Send a Message for Prediction

Once the container is running, you can send HTTP requests to the API to classify messages.

Example curl request:

curl -X POST "http://localhost:8002/predict/" \
-H "accept: application/json" \
-H "Content-Type: application/json" \
-d "{\"text\":\"Your SMS content here\",\"model\":\"bert\"}"
Example Response:

{
  "label": "ham",
  "probability": 0.9971883893013,
  "processing_time": 0.6801116466522217,
  "Model_Name": "OTS_mBERT",
  "Model_Version": "bert-base-uncased",
  "Model_Author": "TelecomsXChange (TCXC)",
  "Last_Training": "2024-03-20"
}
Enter fullscreen mode Exit fullscreen mode

Learn More

For full documentation and source code, visit the Open Text Shield GitHub repository⁠.

SMPP + OTS for real-time, fast Ai driven protection

This is an example implementation of an SMPP Proxy performing real-time classification via OTS before routing the message to the upstream provider.

Image description

Sample SMPP Server code:


const smpp = require('smpp');
const axios = require('axios');
const tls = require('tls');
const fs = require('fs');
const uuid = require('uuid'); 

// Configuration for the upstream SMPP provider (with optional TLS)
const upstreamSMPPConfig = {
  host: '127.0.0.2',     // Replace with your upstream SMPP provider's host
  port: 2776,                     // Replace with your upstream SMPP provider's port
  system_id: 'USER',      // Replace with your upstream SMPP provider's system_id
  password: 'PASSWORD',       // Replace with your upstream SMPP provider's password
  tls: true                        // Set to true to enable TLS for upstream connection
};

// TLS options for inbound connections (replace with your own certificates if available)
let tlsOptions = null;
try {
  tlsOptions = {
    key: fs.readFileSync('server-key.pem'),  // Replace with path to your private key
    cert: fs.readFileSync('server-cert.pem') // Replace with path to your certificate
  };
} catch (err) {
  console.log('TLS configuration for inbound connections not found, falling back to non-TLS.');
}

// TLS options for outbound (upstream) connections (replace with your own certificates if available)
let upstreamTlsOptions = null;
if (upstreamSMPPConfig.tls) {
  try {
    upstreamTlsOptions = {
      key: fs.readFileSync('client-key.pem'),  // Replace with your client key
      cert: fs.readFileSync('client-cert.pem'), // Replace with your client certificate
      rejectUnauthorized: false                // Set to true if you want to enforce certificate validation
    };
  } catch (err) {
    console.log('TLS configuration for upstream not found, falling back to non-TLS.');
    upstreamSMPPConfig.tls = false; // Disable TLS for upstream if config is invalid
  }
}

// Action configuration: customize the behavior for each label type (ham, spam, phishing)
const actionsConfig = {
  ham: {
    action: 'pass_through', // Available options: 'pass_through', 'reject', 'submit_but_not_send'
    logMessage: 'Message is safe (HAM), passing through...'
  },
  spam: {
    action: 'reject', // Available options: 'pass_through', 'reject', 'submit_but_not_send'
    logMessage: 'Message is classified as SPAM, rejecting...'
  },
  phishing: {
    action: 'submit_but_not_send', // Available options: 'pass_through', 'reject', 'submit_but_not_send'
    logMessage: 'Message is classified as PHISHING, submitting but not sending...'
  }
};

// Store for multipart messages
let multipartStore = {};

// Function to create a session to connect to the upstream SMPP provider
let upstreamSession;
if (upstreamSMPPConfig.tls && upstreamTlsOptions) {
  // Use TLS for upstream connection
  upstreamSession = new smpp.Session({
    socket: tls.connect(upstreamSMPPConfig.port, upstreamSMPPConfig.host, upstreamTlsOptions, () => {
      console.log('Connected to upstream SMPP provider using TLS.');
      bindToUpstream();
    })
  });
} else {
  // No TLS, plain SMPP session
  upstreamSession = new smpp.Session({
    host: upstreamSMPPConfig.host,
    port: upstreamSMPPConfig.port
  });
  bindToUpstream();
}

// Function to bind to upstream SMPP provider
function bindToUpstream() {
  upstreamSession.bind_transceiver({
    system_id: upstreamSMPPConfig.system_id,
    password: upstreamSMPPConfig.password
  }, (pdu) => {
    if (pdu.command_status === 0) {
      console.log('Upstream SMPP provider bind successful.');
    } else {
      console.error('Upstream SMPP provider bind failed.');
    }
  });
}

// Conditional inbound connection handler (TLS or non-TLS)
let server;
if (tlsOptions) {
  // Start TLS-based SMPP Server for inbound connections
  server = smpp.createServer({ tls: true }, (session) => {
    handleSession(session);
  });
  server.listen({
    port: 2776,
    key: tlsOptions.key,
    cert: tlsOptions.cert
  }, '127.0.0.1', () => {
    console.log('Smart SMPP Proxy running with TLS on 127.0.0.1:2776');
    simulatePhishingMessage(); // Simulate phishing message after server starts
  });
} else {
  // Start non-TLS SMPP Server for inbound connections
  server = smpp.createServer((session) => {
    handleSession(session);
  });
  server.listen(2776, '127.0.0.1', () => {
    console.log('Smart SMPP Proxy running without TLS on 127.0.0.1:2776');
    simulatePhishingMessage(); // Simulate phishing message after server starts
  });
}

// Handle the SMPP session (bind and message submission)
function handleSession(session) {
  // Listen for bind_transceiver events (incoming bind requests)
  session.on('bind_transceiver', (pdu) => {
    console.log('Received bind request:', pdu);
    // Authenticate the incoming connection
    if (pdu.system_id === 'ots' && pdu.password === 'ots_secret') {
      // Successful bind
      session.send(pdu.response());
      console.log('Client bound successfully with system_id: ots');
    } else {
      // Authentication failure, send bind failure response
      session.send(pdu.response({
        command_status: smpp.ESME_RBINDFAIL
      }));
      console.log('Client bind failed. Invalid system_id or password.');
    }
  });

  // Handle incoming SMS messages via submit_sm PDU
  session.on('submit_sm', (pdu) => {
    console.log('Received submit_sm request:', pdu);

    // Log the message content
    const smsContent = pdu.short_message.message;
    console.log('SMS content:', smsContent);

    // Send response to acknowledge message receipt
    // Modify the response to include the message_id
    const message_id = uuid.v4();  // Generate a unique message ID
    session.send(pdu.response({
      command_status: smpp.ESME_ROK,  // Message sent successfully
      message_id: message_id          // Include the generated message_id
    }));

    console.log(`Generated message_id: ${message_id}`);  // Optional: Log the message_id for debugging

    // Analyze the SMS content via OTS
    processMessage(smsContent, session, pdu);
  });

  // Catch any unexpected disconnection or error
  session.on('unbind', () => {
    console.log('Client has unbound.');
  });

  session.on('close', () => {
    console.log('Connection closed.');
  });

  session.on('error', (err) => {
    console.log('SMPP error:', err);
  });
}

// Process message based on OTS analysis and action configuration
async function processMessage(smsContent, session, pdu) {
  console.log(`Processing SMS: ${smsContent}`);

  // Analyze SMS using OTS
  try {
    const response = await analyzeSmsWithOTS(smsContent);
    const label = response.label; // Assume OTS returns 'ham', 'spam', or 'phishing'
    const config = actionsConfig[label];

    // Execute the action based on the configuration
    if (config) {
      console.log(config.logMessage);
      handleAction(session, pdu, config.action);
    } else {
      console.log('Unknown classification, rejecting message.');
      session.send(pdu.response({
        command_status: smpp.ESME_RSUBMITFAIL
      }));
    }
  } catch (error) {
    console.error('Error analyzing SMS with OTS:', error);
    session.send(pdu.response({
      command_status: smpp.ESME_RSUBMITFAIL
    }));
  }
}

// Simulate a phishing message when the proxy starts
function simulatePhishingMessage() {
  const phishingMessage = "Urgent! Your bank account has been compromised. Click here to secure it now: http://fakebank.com";
  console.log("Simulating phishing message...");
  processMessage(phishingMessage, null, { short_message: { message: phishingMessage } });
}

// Function to handle concatenated SMS
function handleConcatenatedSMS(pdu) {
  const udh = pdu.short_message.udh;
  const reference = udh[3]; // Unique reference number for the multipart message
  const totalParts = udh[4]; // Total number of parts
  const currentPart = udh[5]; // Current part number

  // Initialize multipart store for the reference number if not already present
  if (!multipartStore[reference]) {
    multipartStore[reference] = new Array(totalParts).fill(null);
  }

  // Store the current part
  multipartStore[reference][currentPart - 1] = pdu.short_message.message;

  // Check if all parts are received
  if (multipartStore[reference].every(part => part !== null)) {
    const fullMessage = multipartStore[reference].join('');
    delete multipartStore[reference]; // Clear the store for this reference
    return fullMessage;
  }

  return null; // Return null until all parts are received
}

// Function to handle different actions based on the label
function handleAction(session, pdu, action) {
  switch (action) {
    case 'pass_through':
      // Pass-through logic: Forward the SMS to the upstream SMPP provider
      console.log('Forwarding SMS to upstream provider...');
      forwardToUpstream(pdu); // Forward the message to the upstream SMPP provider
      break;
    case 'reject':
      // Reject the SMS
      if (session) {
        session.send(pdu.response({
          command_status: smpp.ESME_RSUBMITFAIL
        }));
      }
      break;
      case 'submit_but_not_send':
        // Acknowledge submission but do not forward to upstream
        const message_id = uuid.v4();  // Generate a unique message ID
        if (session) {
          session.send(pdu.response({
            command_status: smpp.ESME_ROK,  // Acknowledge success
            message_id: message_id          // Include the generated message_id
          }));
        }
        console.log(`SMS classified as PHISHING, message_id: ${message_id}, submitted but not forwarded to upstream. DING DING 💰💰💰`);
        break;
    default:
      // Default case: reject message
      if (session) {
        session.send(pdu.response({
          command_status: smpp.ESME_RSUBMITFAIL
        }));
      }
      console.log('Invalid action specified, rejecting the message.');
  }
}

// Function to forward the message to the upstream SMPP provider
function forwardToUpstream(pdu) {
  upstreamSession.submit_sm({
    destination_addr: pdu.destination_addr,
    source_addr: pdu.source_addr,
    short_message: pdu.short_message
  }, (responsePdu) => {
    if (responsePdu.command_status === 0) {
      console.log('Message forwarded to upstream successfully.');
    } else {
      console.error('Failed to forward message to upstream:', responsePdu.command_status);
    }
  });
}

// Function to send SMS content to OTS for analysis, with request/response logging
async function analyzeSmsWithOTS(smsContent) {
  console.log("Sending request to OTS with message:", smsContent);
  try {
    const response = await axios.post('http://localhost:8002/predict/', {
      text: smsContent,
      model: 'bert' // Assuming you're using the BERT model
    }, {
      headers: {
        'Content-Type': 'application/json'
      }
    });

    // Log the OTS response
    console.log("Received response from OTS:", response.data);

    // Return the OTS response (assumed format: { label: 'ham', probability: 0.95 })
    return response.data;
  } catch (error) {
    console.error('Failed to contact OTS service:', error);
    throw new Error('Failed to contact OTS service.');
  }
}


Enter fullscreen mode Exit fullscreen mode
. . . . . . . . . . . . . . . . . . . . . .
Terabox Video Player