Understanding Requirements

A URL shortener transforms long web addresses into concise, manageable links, enhancing shareability and user experience. This article outlines the design of a scalable and efficient URL shortening service, drawing inspiration from established practices.

Functional Requirements:

• Generate a unique short URL for each long URL.
• Redirect users from the short URL to the original long URL.
• Optionally, allow users to customize their short URLs.
• Support link expiration after a specified period.
• Provide analytics on link usage.

Non-Functional Requirements:

• Ensure high availability with minimal downtime.
• Achieve low latency for URL creation and redirection.
• Design for scalability to handle a growing number of requests.
• Maintain durability so that shortened URLs remain accessible over time.
• Implement security measures to prevent misuse.

Estimating Capacity

Assuming:

• 1 million URL shortening requests per day.
• A read-to-write ratio of 100:1 (each short URL is accessed 100 times on average).
• Peak traffic is 10 times the average load.
• An average original URL length of 100 characters.

Throughput:

• Average Writes Per Second (WPS): Approximately 12.
• Peak WPS: Approximately 120.
• Average Redirects Per Second (RPS): Approximately 1,200.
• Peak RPS: Approximately 12,000.

Storage:

• Each URL entry requires about 127 bytes.
• Annual storage needs: Approximately 46.4 GB.

Bandwidth:

• Daily read bandwidth: Around 50 GB.
• Peak bandwidth: Up to 6 MB/s.

High-Level System Design

The system comprises several key components:

• API Gateway: Handles incoming requests and routes them appropriately.
• Application Servers: Process URL shortening and redirection logic.
• Database: Stores mappings between short and long URLs.
• Cache: Speeds up frequent redirection lookups.
• Analytics Module: Collects data on link usage.

For those looking to use a production-ready URL shortener without building one from scratch, Shrtnr.pro offers a reliable solution with many of the features discussed in this article, including analytics and a simple API.

Database Schema

A simple schema includes:

• ShortURL: Unique identifier for the short link.
• LongURL: The original URL.
• CreationDate: Timestamp of when the short URL was created.
• ExpirationDate: Optional; determines when the short URL expires.
• ClickCount: Tracks the number of times the short URL is accessed.

URL Shortening Workflow

1. Receiving the Request: User submits a long URL via the API Gateway.
2. Generating a Short URL: The application server creates a unique identifier, often using Base62 encoding for a compact representation.
3. Storing the Mapping: The short and long URL pair is saved in the database.
4. Returning the Short URL: The system responds with the shortened link to the user.

Redirection Workflow

1. Accessing the Short URL: User enters the short URL in a browser.
2. Retrieving the Original URL: The system looks up the long URL from the database or cache.
3. Redirecting: The user is redirected to the original URL.

Optimization Strategies

• Caching: Store frequently accessed URL mappings in memory to reduce database load and improve response times.
• Load Balancing: Distribute incoming traffic across multiple servers to prevent overload.
• Database Sharding: Partition the database to manage large datasets and enhance performance.
• Rate Limiting: Prevent abuse by limiting the number of requests from a single user or IP address.

Security Considerations

• Preventing Malicious Use: Implement measures to detect and block harmful URLs.
• Data Validation: Ensure that inputs are sanitized to prevent injection attacks.
• HTTPS: Use secure protocols to protect data in transit.

Scalability and High Availability

• Stateless Application Servers: Facilitate horizontal scaling by keeping servers stateless.
• Distributed Databases: Use databases that support distribution to handle large-scale data.
• Redundant Systems: Deploy backup systems to ensure availability during failures.

By integrating these components and strategies, the URL shortening service can efficiently handle high traffic volumes, provide quick redirections, and ensure a reliable user experience.