Introduction to System Design

System design is a crucial aspect of software engineering that focuses on defining the architecture, components, modules, and interfaces of a system to meet specific requirements. In essence, it’s about planning how the parts of a system will interact with each other to fulfill the intended use cases.

System design principles include scalability, reliability, performance, and maintainability. When creating a new system, understanding these fundamentals can guide us to a solution that can handle anticipated loads, adapt to growth, and provide a smooth user experience. In this blog, we’ll break down the system design process by building a referral system, specifically focusing on a case similar to Google Pay’s Diwali offer.

System Design Process

We'll approach the referral system design with six main steps:

1. Clarify the Problem and Define Requirements
2. Capacity Estimates
3. High-Level Design
4. API Design
5. Data Schema Creation
6. Detailed Design

Step 1: Define Problem Scope and Requirements

Project Example

Imagine a referral system for a payment app (e.g., Google Pay) where:

• The referrer (person who sends the invite) earns ₹201 when the invitee (person who joins via the invite) completes their first transaction.
• The invitee gets ₹51 upon joining and completing their first transaction.

Functional Requirements

1. The system should allow users to generate a unique invite link.
2. Users should be able to create an account with profile photos.
3. Referrers should be notified and rewarded when someone joins and completes a transaction using their referral link.
4. A user can view the list of people who joined using their referral link.

Non-functional Requirements

1. Scalability: The system should handle a large number of users and referral transactions.
2. Reliability: Referral tracking should be accurate and reward distribution consistent.
3. Performance: Users should receive rewards promptly, without long delays.

Step 2: Determine Capacity Estimates

Estimating capacity requirements helps in planning the resources and ensuring that the system can handle the expected load. For example:

• Profile Photos: Assume each profile photo is 3MB, and we expect 5000 new users daily. This means:

  • Daily storage for images: (5000 \times 3 \text{ MB} = 15 \text{ GB})
  • Daily bandwidth requirement: (15 \text{ GB} \times 1000 = 3 \text{ TB})

• Unique Code Storage: Using a 5-character alphanumeric code (A-Z, a-z, 0-9), we can support approximately 91 million unique combinations. This should be sufficient for our user base.

Step 3: High-Level Design

Unique Link Creation

The referral system will generate unique codes or links to ensure each invite is identifiable.

1. Length of Unique Code: For a 5-character alphanumeric code, we have enough combinations for our user volume.
2. Unique Code Generation:
   • Brute Force: Generate a random code, check if it’s unique, and assign it to the user.
   • Hash-Based (MD5): Use the MD5 hash of the user’s email with an incremental counter if duplicates arise.

Example Code for MD5 Generation:

def generate_unique_code(email, attempt=0):
    if attempt > 5:
        return None
    hash = calculate_hash(email + str(attempt))
    if is_unique(hash):
        return hash
    else:
        return generate_unique_code(email, attempt + 1)

Step 4: API Design

API endpoints will handle the interactions between users and the system. Key APIs include:

1. Generate Unique Link for User:

   • Endpoint: generateUniqueLink(email: string)
   • Parameters: email – User’s email
   • Function: Generates and returns a unique referral link for the user.

2. Get Unique Link:

   • Endpoint: getUniqueLink(email: string)
   • Parameters: email – User’s email
   • Function: Retrieves the unique referral link for a user.

3. Mark Referral:

   • Endpoint: markReferral(email: string, referralCode: string)
   • Parameters:
     • email – New user’s email
     • referralCode – Code of the referring user
   • Function: Marks the new user as referred and initiates the reward process.

4. Get All Referrals of a User:

   • Endpoint: getAllReferrals(email: string)
   • Parameters: email – User’s email
   • Function: Returns a list of users who joined via the referrer’s link.

Step 5: Database Schema Design

A well-structured database is essential for managing user data and referral tracking.

Step 6: Detailed Design

This phase addresses potential bottlenecks and performance optimization:

1. Handling High Traffic:

   • Solution: Use load balancing and server autoscaling to manage high traffic volumes.

2. Efficient Key Generation:

   • Solution: Consider longer codes or stronger algorithms to reduce the probability of duplicates.

3. Caching:

   • Solution: Implement caching for frequent API calls, such as retrieving a user’s referral list, to reduce database load.

4. Database Sharding:

   • Solution: Distribute data across multiple databases to manage growth and reduce latency.

Conclusion

Designing a referral system involves careful consideration of requirements, capacity, and scalability. Starting with a high-level overview and breaking down each aspect of the system, from API design to database schema, enables us to build a system that is scalable, efficient, and user-friendly. By applying these system design principles, you can create a robust referral platform that enhances user engagement and rewards loyalty.