Disclosure: This article was commissioned by Novu.

Keeping users informed and engaged is critical for the success of any application or SAAS. In order to accomplish this, development and product teams need to either build their own notifications system, or buy a purpose-built solution. A capable, notification infrastructure consists of a backend system that manages the delivery of messages to users across various channels. These systems ensure that users receive timely and relevant updates, helping them stay connected and informed about important events, updates, and actions within the app or service.

Setting up a robust notification infrastructure involves several components, including message routing, user preference management, content personalization/customization, and delivery tracking. The goal is to provide a seamless and reliable notification experience that can scale with user growth and evolving needs. For solution-aware users, understanding the intricacies of notification infrastructure is crucial as it impacts user engagement, retention, and overall satisfaction. Whether you’re a startup building a new product or an enterprise looking to streamline your notification systems, the choice between building in-house or leveraging an third party service is a significant decision that can influence your project’s success.

The notification landscape

Organizations face the challenge of managing an overwhelming workload, including major architectural refactors, new cloud applications, and internal technology adoption. Amid these tasks, handling notifications has become particularly complex. Notifications have evolved from an overlooked feature to an essential component for user engagement across various applications. However, the rapid increase in notification requirements and compressed timelines often lead to fragmented, inconsistent systems developed by different teams without coordination, resulting in "notification chaos."

This chaos is marked by the proliferation of siloed notification components, complicating management and creating inefficiencies. Supporting multiple notification channels further adds to the complexity, as users demand control over their notification preferences. The disjointed systems can lead to inconsistent delivery, ignored user preferences, and increased maintenance costs.

Key Issues with Notification Chaos

• Fragmentation: Disconnected notification systems developed independently by various teams.
• Complexity: Multiple channels and user preference demands increase system complexity.
• Operational challenges: Inconsistent delivery, user dissatisfaction, and high maintenance costs.
• Strategic decision: Organizations must choose between building a custom solution or adopting an vendor like Novu, balancing customization needs with operational efficiencies.

Ultimately, the right choice will depend on the specific needs and resources of the organization, as well as its long-term strategic goals.

Organization 1:

A startup needs to add notifications to their app
Imagine you're running a tech startup and need a robust notification system for task updates, deadlines, and collaborative activities. You're facing notification chaos with multiple channels like email, SMS, push notifications, and in-app messages. You must decide whether to build this system in-house or leverage a third partyservice. Building in-house offers complete control and customization but diverts engineering resources from core product development. It's resource-intensive and time-consuming, which could slow down your product's growth.

On the other hand, using an third part service streamlines the process, offering multi-channel support, user preference management, and scalable delivery out of the box. This approach allows your team to focus on core features, accelerating your product's time-to-market and ensuring reliability and scalability. As a solution-aware startup, you must weigh the pros and cons: customization and control versus efficiency and focus on your core product. Choosing the right path will help you manage notification chaos while driving growth and innovation.

Organization 2: Enterprise notifications consolidation from disparate platforms

Consider a large enterprise with multiple applications, each using its own notification system, leading to notification chaos with inconsistent user experiences and redundant systems. The enterprise wants to centralize their notification infrastructure to improve consistency and enhance user experience across all platforms.

Developing an in-house unified notification platform offers maximum control and customization, integrating deeply with existing systems. However, it demands significant engineering resources and ongoing maintenance, making it a complex and resource-intensive endeavor.

Alternatively, adopting a centralized notification infrastructure provides an efficient and scalable solution. Such a service manages notification complexities, offering features like intelligent message routing and user preference management. This approach frees up engineering resources, allowing them to focus on core business objectives.

How to build a notifications system: core requirements

Building a notification system requires 7 core pieces to make it successful, maintainable, and compliant. These requirements encompass an inbox for centralized message management, user-configurable preferences, intelligent message routing, dynamic content management, team interactivity features, comprehensive analytics, and considerations for ongoing maintenance, scaling, and debugging. Each of these components is crucial for delivering timely, relevant, and personalized notifications while maintaining system efficiency and scalability. Leveraging the right infrastructure and third party dependencies to support these requirements is essential for meeting user expectations and ensuring the reliability of the notification system.

Core requirement 1: Inbox

The inbox provides a centralized place to view and manage user messages. The inbox can take different forms, such as a notification bell within an application or a dedicated notification box. Each type has its unique requirements and challenges:

• Centralized message repository: Collects messages from various channels into one accessible stream for easy navigation and management.
• Notification read/Unread status management: Allows users to distinguish and manage new and read notifications using visual cues.
• User-friendly access and management: Enables intuitive navigation, marking, deleting, and archiving of notifications with simple controls and search functionality.

Supporting infrastructure and SAAS for user inbox requirements

Several infrastructure components and SAAS support these inbox features:

1. Message queues (e.g., RabbitMQ, Kafka, Nats): These ensure reliable delivery and processing of notifications, making sure messages are not lost and are delivered in the correct order. Databases (e.g., MongoDB, PostgreSQL): Used for storing notification data, including the read/unread state, in a structured and scalable manner.
2. Notification management platforms (e.g., Firebase Cloud Messaging, OneSignal): These SAAS provide tools for managing and delivering notifications across multiple channels.
3. Front-End frameworks (e.g., React, Vue.js, Angular, Svelte): Essential for building user-friendly interfaces that allow users to interact with their inbox efficiently.
4. Search and filtering tools (e.g., Elasticsearch ): These tools enhance the ability to search and filter notifications, especially in email inbox scenarios.

Choosing the right infrastructure and SAAS depends on the specific requirements of your notification system and the type of inbox you aim to implement. Whether it's a notification bell or an email box, having a robust and user-friendly inbox is key to maintaining an effective communication system and ensuring a positive user experience.

Core requirement 2: Subscriber preferences

Managing subscriber preferences is crucial yet often overlooked in notification systems. User-specified preferences allow for a tailored notification experience, enhancing satisfaction and engagement. Here are the key requirements for user-configurable notification settings and the supporting infrastructure.

• User-configurable notification settings: Users should easily choose types, frequency, and channels for notifications through a user-friendly interface to manage their preferences.
• Customizable delivery channels and schedules: Users can select preferred channels (email, SMS, push notifications) and schedule notifications for specific times, enhancing relevance and reducing ignored messages.
• Unsubscribes (individual, category, and site-wide): Users should easily unsubscribe from individual, category, or all notifications, reducing fatigue and retaining engagement.
• Language Integrations for multi-lingual support: Users can receive notifications in their preferred language, requiring robust translation capabilities for accurate and culturally appropriate messages.
• Framework integrations for seamless embedding: Notification preferences should integrate smoothly with third party systems and frameworks like React, Angular, or Vue.js for a seamless user experience.

Supporting infrastructure and SAAS for user preference requirements

Several infrastructure components and SAAS can support these advanced notification preferences:

1. User preference management tools (e.g., Customer.io, UserEngage): These tools provide interfaces and backend systems for managing user preferences, ensuring users can easily customize their notification settings.
2. Messaging platforms (e.g., Twilio, SendGrid): These SAAS facilitate multi-channel delivery of notifications, including email, SMS, and push notifications, allowing for customizable delivery options.
3. Scheduling SAAS (e.g., Quartz Scheduler, DKron): Used to schedule notifications, ensuring they are delivered at user-specified times.
4. Internationalization libraries (e.g., i18next, Globalize): These libraries support multi-lingual capabilities, helping to translate and deliver notifications in various languages.
5. Integration platforms (e.g., Zapier, Integromat): These platforms help integrate notification systems with other applications and frameworks, ensuring smooth operation and user experience.

By leveraging these tools and SAAS, you can build a notification system that not only meets but exceeds user expectations, providing a highly personalized and efficient notification experience.

Core requirement 3: message routing

Effective message routing is essential for ensuring that notifications are timely, relevant, and delivered through the preferred channels. Advanced message routing capabilities can significantly enhance user satisfaction and engagement.

• Intelligent routing based on user preferences and notification context: Tailors notification delivery by analyzing user settings, behavior, and context to ensure relevance and welcome. Ensuring timely and relevant notifications: Delivers notifications promptly (<1 min) by considering user activity and context, maximizing impact and avoiding message overload.
• Fan-out capabilities for multiple recipients: Sends a single notification to multiple recipients efficiently, maintaining performance and delivery speed for collaborative applications. Timezone-aware delivery: Adjusts delivery times based on the recipient’s timezone to ensure notifications are received at optimal, non-disruptive times.
• Handling online/offline status: Queues notifications for offline users and delivers them upon reconnection, ensuring no important updates are missed.

Supporting infrastructure for message routing requirements

Several infrastructure components and SAAS support these advanced message routing capabilities:

1. Message queues (e.g., RabbitMQ, Apache Kafka, NATS, Apache Pulsar): These ensure reliable and efficient message delivery and queuing, allowing for intelligent routing and fan-out operations.
2. Notification hubs (e.g., AWS SNS, Azure Notification Hubs): These provide scalable platforms for sending notifications across multiple channels and to large audiences.
3. Scheduling SAAS (e.g., Quartz Scheduler, DKron): These help in scheduling notifications to ensure they are delivered at optimal times based on timezone and user preferences.
4. Presence SAAS (e.g., Firebase Realtime Database, PubNub): These SAAS can track user online/offline status and manage the queuing and delivery of notifications accordingly.
5. User analytics platforms (e.g., Mixpanel, Segment): These provide insights into user behavior and preferences, enabling intelligent routing decisions based on context and user settings.

Core requirement 4: team interactivity

Ensuring smooth team interactivity is vital for the effective management of notifications. This involves using collaboration tools, setting up roles and permissions, and empowering non-technical users to make changes without needing developer assistance. Here’s a detailed look at these requirements.

• Assessing engineering’s content management role: Determine if developers need to handle all content tasks or if they can delegate to improve resource management and productivity.
• Non-technical staff managing notifications: Allow non-technical staff to handle routine notification tasks, enhancing efficiency and enabling developers to focus on technical issues.
• Content management collaboration: Enables team collaboration on notification content with real-time updates and shared feedback using platforms like shared workspaces and document editors.
• Team member roles and permissions: Defines specific roles and permissions to ensure appropriate access, prevent unauthorized changes, and enhance accountability within the team.
• Non-technical users changing copy and images: Allows non-technical users to update text and images through user-friendly interfaces, reducing the need for developer involvement and speeding up notification deployment.

Supporting tools and infrastructure for team interactivity requirements

Several infrastructure components and SAAS support advanced team interactivity:

1. Several infrastructure components and SAAS support advanced team interactivity:
2. Collaboration platforms (e.g., Slack, Microsoft Teams, Discord, Google Chat): These platforms facilitate real-time communication and collaboration among team members, streamlining the process of managing notifications.
3. Content management systems (e.g., Contentful, WordPress): These systems often come with built-in roles and permissions, allowing for structured access control and enabling non-technical users to update content easily.
4. Document editors (e.g., Google Docs, Quip): These tools provide collaborative editing capabilities, enabling team members to work together on notification content and make real-time updates and comments.
5. WYSIWYG editors (e.g., TinyMCE, Froala): These editors allow non-technical users to make changes to notification content and images without needing to write code, making it easy to update and manage notifications.
6. Version control systems (e.g., Git, Bitbucket): While primarily used by developers, these systems can also support collaboration by tracking changes and ensuring that all updates are reviewed and approved before going live.

By leveraging these tools and SAAS, you can create a collaborative environment that enhances team interactivity, ensuring efficient content and message management while empowering non-technical users. This approach helps maintain a dynamic and responsive notification system that can quickly adapt to changing needs and user feedback.

Core requirement 5: analytics and metrics

Analytics and metrics are the backbone of an effective notification system. They provide insights into delivery success, user engagement, and system performance. Here's an in-depth look at these key aspects and the infrastructure that supports them.

• Tracking delivery success rates and user engagement: Monitors notification delivery across channels and analyzes user interactions to refine strategies and enhance satisfaction.
• Setting up delivery rates for various channels: Optimizes notification campaigns by understanding and adjusting delivery rates for different channels to ensure timely delivery.
• Monitoring and handling integration downtimes: Uses monitoring tools and alerts to quickly address integration issues, minimizing disruptions in notification delivery.
• Measuring user engagement and response rates: Analyzes metrics like open rates and click-through rates to determine notification effectiveness and improve content and delivery strategies.

Supporting infrastructure and tools for analytics requirements

Several infrastructure components and SAAS support advanced analytics and metrics for notification systems:

1. Analytics platforms (e.g., Google Analytics, Mixpanel): These platforms provide detailed insights into user behavior and engagement, helping you track and analyze key metrics.
2. Monitoring tools (e.g., Datadog, New Relic): These tools help monitor system performance and integration downtimes, ensuring timely detection and resolution of issues.
3. Email/SMS analytics (e.g., SendGrid, Twilio): These SAAS offer built-in analytics for tracking delivery success rates and user engagement across email and SMS channels.
4. Push notification SAAS (e.g., Firebase Cloud Messaging, OneSignal): These platforms provide detailed metrics on push notification delivery and engagement, helping you optimize your campaigns.
5. A/B testing tools (e.g., Optimizely, VWO): These tools allow you to test different notification strategies and measure their impact on user engagement and response rates.

By leveraging these tools and SAAS, you can build a comprehensive analytics and metrics framework that provides deep insights into the performance of your notification system. This enables you to make data-driven decisions, optimize your notification strategies, and ensure a high-quality user experience.

Core requirement 6: volume and scaling

Managing increasing notification loads: Design your system for peak performance to handle high volumes, scaling horizontally to accommodate growing demands.

• Scaling strategies: Use horizontal scaling based on network requests and queue sizes to distribute load, improve redundancy, and maintain performance during peak times.
• Custom metrics: Monitor metrics like CPU usage, memory, network latency, and queue lengths to make informed decisions about scaling and resource allocation for optimal performance.

Custom metrics challenges

Custom metrics require a thorough understanding of system architecture and the specific performance indicators that impact your notification infrastructure. These include:

1. Identifying relevant metrics: Determining which metrics are most indicative of system performance and resource needs.
2. Implementing monitoring tools: Integrating tools that can capture and report these metrics in real-time, such as Prometheus and Grafana or 3rd party monitoring system.
3. Configuring alerts and thresholds: Setting up alerts and thresholds to notify your team when certain metrics exceed predefined limits, indicating the need for scaling adjustments.
4. Testing and calibration: Continuously testing and calibrating the metrics to ensure they accurately reflect system performance and provide actionable insights.

Using custom metrics for dynamic scaling

Once custom metrics are in place, leveraging them for dynamic scaling involves:

1. Automated scaling policies: Developing automated policies that trigger scaling actions based on real-time metrics. For example, adding servers when CPU usage exceeds a certain percentage or reducing servers when queue sizes drop below a threshold.
2. Continuous monitoring and adjustment: Regularly monitoring the performance of the scaling policies and adjusting them as needed to respond to changing conditions and workloads.
3. Integrating with orchestration tools: Using orchestration tools like Kubernetes, nomad, or a hosted container solution to manage the scaling process seamlessly, ensuring that resources are allocated efficiently without manual intervention.

Challenges in using custom metrics

Using custom metrics for dynamic scaling presents several challenges:

1. Complexity: Managing the complexity of multiple metrics and ensuring they accurately reflect the system’s needs can be daunting.
2. Resource allocation: Balancing resource allocation to avoid over-provisioning (which leads to higher costs) and under-provisioning (which impacts performance) requires constant tuning.
3. Integration with existing systems: Ensuring that custom metrics and scaling policies integrate smoothly with existing infrastructure and workflows can be technically demanding.

Summary

Effectively setting up and leveraging custom metrics for dynamic scaling is critical for maintaining a robust, scalable, and efficient notification infrastructure. While the process is complex and requires significant effort, the benefits of improved performance, cost-efficiency, and resilience make it a worthwhile investment. By focusing on these strategies and overcoming the associated challenges, you can ensure your notification system can adapt to evolving demands and deliver a seamless user experience.

Core requirement 7: debugging and notification tracing

Using advanced tools and methods to troubleshoot message delivery issues allows you to diagnose and resolve problems promptly. Tracing message paths ensures accountability by tracking the journey of each message, confirming it reaches its destination. Analyzing queues helps identify bottlenecks and delivery failures, ensuring smooth message flow. Additionally, determining whether issues are caused by internal systems or third-party integrations helps isolate and address problems more efficiently.

• Tools and methods for troubleshooting delivery issues: Use diagnostic tools to detect and resolve message delivery problems, ensuring consistent notifications.
• Tracing message paths and ensuring accountability: Track each message to verify delivery and identify failure points, maintaining system integrity and improvement insights.
• Analyzing queues to identify bottlenecks and failures: Monitor and analyze queue data to detect bottlenecks and delivery failures, enabling corrective actions for smooth message processing.
• Determining problem source: Isolate issues to either internal systems or third-party integrations for efficient troubleshooting and resolution.

Summary

Advanced tools enable prompt problem diagnosis and resolution, while tracing message paths ensures accountability. Analyzing queues helps identify bottlenecks, and distinguishing between internal and third-party issues allows efficient troubleshooting. These practices ensure smooth and efficient notification operations.

Running a notifications platform: operational considerations

Ensuring a robust and reliable notification system requires careful attention to several operational aspects. These include ongoing maintenance, handling volume and scaling, debugging and tracing issues, and team considerations. Here’s a concise overview of these critical areas.

Ongoing maintenance/patching, updates, and security updates

Regular system updates and security patches are essential for protecting your notification infrastructure from vulnerabilities. By staying current with the latest patches, you can ensure smooth operation and prevent potential security breaches. This proactive approach minimizes downtime and maintains system integrity.

Handling evolving requirements and feature enhancements

Adapting your system to meet new user needs and incorporate advanced features is crucial for staying relevant. This involves regularly assessing user feedback and market trends to update your notification infrastructure. By doing so, you can enhance user experience and maintain a competitive edge.

Integration changes to adapt to new SAAS and platforms

As new SAAS and platforms emerge, ensuring compatibility is vital. Regularly updating your system to integrate with tools like Firebase Cloud Messaging (FCM) allows you to leverage new technologies and maintain seamless communication. This adaptability keeps your infrastructure flexible and future-proof.

Notification templates and content updates

Keeping notification templates and content up-to-date ensures that your messages remain relevant and engaging. Regular updates help you cater to changing user preferences and maintain high engagement rates. Fresh content and appealing templates can significantly enhance the effectiveness of your notifications.

Provider integrations

Maintaining and updating integrations with various service providers is crucial for seamless communication. This involves ensuring that your notification infrastructure can effectively interact with email, SMS, and push notification SAAS. Regular checks and updates keep these integrations functioning smoothly.

API and SDK maintenance and testing

Regular maintenance and testing of APIs and SDKs are necessary to ensure they work correctly with other system components. This includes updating to the latest versions and conducting thorough testing to identify and fix any issues. Effective API/SDK management ensures reliable and efficient system performance.

Credential management and security

Managing credentials securely is essential to prevent unauthorized access and protect data. Regularly updating and securely storing credentials reduces the risk of security breaches. Implementing strong authentication methods and monitoring access controls are key practices in credential management.

Summary

Regular system updates and security patches are essential for protecting your infrastructure from vulnerabilities, minimizing downtime, and maintaining system integrity. Adapting to evolving requirements and integrating with new SAAS keeps your system relevant and flexible, while continuous security upgrades protect against emerging threats.
Maintaining and updating notification templates ensures messages remain engaging and effective. Regular checks and updates of provider integrations, along with thorough maintenance and testing of APIs and SDKs, ensure seamless communication and reliable performance. Secure credential management prevents unauthorized access and protects data.
By focusing on these operational considerations, you can ensure your notification system remains robust, scalable, and efficient, delivering a seamless user experience and supporting the evolving needs of your business.

Resourcing

When deciding whether to build or buy a notification infrastructure, accurately estimating the time required is crucial. Here’s a breakdown of the different tiers of building notification infrastructure, focusing on the time commitment needed for each level, from simple integrations to complex systems, and including ongoing maintenance estimates.

1. Simple notification integration
   • Scope: Basic notifications for essential updates.
   • Resources: 1 engineer, 2 weeks.
   • Complexity: Low; minimal customization and features.
   • External packages: Typically 2-4 external packages (e.g., SMTP libraries, basic notification frameworks).
   • Ongoing maintenance: Low. Expect quarterly updates requiring 1-2 days of engineering time for testing and integration.
2. Multi-channel notifications
   • Scope: Support for multiple channels (e.g., email, SMS, push).
   • Resources: 1-3 engineers, 1 quarter.
   • Complexity: Moderate; requires integration with various communication APIs and preference management.
   • External packages: Approximately 6-24 external packages (e.g., Twilio for SMS, Firebase for push notifications, email service providers, etc).
   • Ongoing maintenance: Moderate. Monthly updates for at least one package, requiring around 1 week of engineering time per month for testing and integration.
3. Segmented notifications
   • Scope: Segmentation of users for targeted notifications.
   • Resources: 2-4 engineers, 2 quarters.
   • Complexity: High; involves building infrastructure for user segmentation, managing different user groups, and ensuring scalability.
   • External packages: Around 8-24 external packages (e.g., user segmentation libraries/systems, advanced notification SAAS).
   • Ongoing maintenance: High. Bi-weekly updates for multiple packages, with an estimated 2-3 weeks of engineering time per month for testing and integration.
4. Internationalization
   • Scope: Full support for multiple languages and regional preferences.
   • Resources: 3-5 engineers, 6 quarters.
   • Complexity: Very high; involves developing a robust system for language translations, managing different regional preferences, and ensuring compliance with international regulations.
   • External packages: Around 10-24 external packages (e.g., translation SAAS, localization frameworks).
   • Ongoing maintenance: Very high. Weekly updates for various packages, demanding about 4-6 weeks of engineering time per month for testing, integration, and continuous improvement.
5. Personalization
   • Scope: Personalized notifications based on user behavior and preferences.
   • Resources: 4-6 engineers, 1-2 Data Engineers, 2-4 Infrastructure Engineers at least 14 quarters.
   • Complexity: Extremely high; requires sophisticated data processing, user profiling, and dynamic content generation.
   • External Packages: Approximately 16-32 external packages (e.g., machine learning libraries, recommendation engines, infrastructure and data lake integrations).
   • Ongoing Maintenance: Extremely high. Weekly updates and compliance checks, requiring 6-8 weeks of engineering time per month to maintain, test, and integrate updates while ensuring regulatory adherence.

Decision time

By understanding the time requirements for each tier of building a notification infrastructure, you can better evaluate whether to build or buy. Consider your organization's specific needs, available resources, and long-term goals to make a decision that aligns with your strategic priorities. Whether opting for a simple integration or a fully personalized and internationalized system, ensuring that you have the right resources and timeline in place is crucial for success.

By examining the different tiers of notification infrastructure—from simple integrations to complex, personalized, and internationalized systems—you can better understand the time and effort required for each level of implementation. This knowledge will help you make an informed decision that aligns with your strategic priorities, ensuring that your notification system is robust, scalable, and capable of meeting the evolving needs of your users.

Whether you choose to build or buy, the goal is to create a seamless and effective notification experience that enhances user engagement and satisfaction while supporting the growth and success of your organization.