Building a URL Shortener in Go
A write-up of building a URL shortener using Go, PostgreSQL, Redis, and Render.
1. Overview
The stack uses Go with the net/http library, PostgreSQL for the database, Redis for caching, and Render for deployment. I built it purely as a learning project; nothing here is production-grade.
2. Architecture
3. Code Generation, Custom Aliases, and Collision Handling
I use Base62 to encode the URL. First, I get the next index from the PostgreSQL database using nextval(), encode it into Base62, and then store that code in the database as well. I didn’t use an insert-then-update approach to prevent a null-code scenario.
Custom aliases share the same code column, and the unique constraint ensures that the aliases are unique.
Collisions are handled internally by PostgreSQL via error 23505. I don’t use a pre-check SELECT to avoid race conditions if two users try to enter the same alias at the same time.
4. Caching Strategy
The API first checks Redis for the link. If it is there, it redirects immediately. If not, it performs a database lookup and populates the Redis cache with a TTL. I set the TTL to 1 hour because most of the hits will probably occur around the time the link was created. I could have added a custom TTL as well, but that’s for another time.
The cache is not invalidated if a link is deleted somehow (which can currently only be done by the admin). I know about this gap and it can be fixed, but I’m too lazy to implement it right now.
5. Rate Limiting
type rateLimiter struct {
mu sync.Mutex
bucket map[string]*bucket
capacity float64
rate float64
}
func newLimiter(capacity, rate float64) *rateLimiter {
l := &rateLimiter{
bucket: make(map[string]*bucket),
capacity: capacity,
rate: rate,
}
go l.sweep()
return l
}
func (l *rateLimiter) sweep() {
for {
time.Sleep(5 * time.Minute)
l.mu.Lock()
for ip, b := range l.bucket {
if time.Since(b.lastRefill) > 10*time.Minute {
delete(l.bucket, ip)
}
}
l.mu.Unlock()
}
}
func clientIP(r *http.Request) string {
if xff := r.Header.Get("X-Forwarded-For"); xff != "" {
parts := strings.Split(xff, ",")
return strings.TrimSpace(parts[0])
}
ip, _, err := net.SplitHostPort(r.RemoteAddr)
if err != nil {
return r.RemoteAddr
}
return ip
}
func (l *rateLimiter) allow(ip string) bool {
l.mu.Lock()
defer l.mu.Unlock()
b, exists := l.bucket[ip]
if !exists {
b = &bucket{l.capacity, time.Now()}
l.bucket[ip] = b
}
elapsed := time.Since(b.lastRefill).Seconds()
b.tokens = min(l.capacity, b.tokens+elapsed*l.rate)
b.lastRefill = time.Now()
if b.tokens >= 1 {
b.tokens--
log.Printf("ALLOWED — tokens now %.2f", b.tokens)
return true
}
log.Printf("BLOCKED — tokens now %.2f", b.tokens)
return false
}I made a custom token bucket with a rate of 1 token/sec, which I think is a fairly modest rate limiter. It checks the incoming IP, though I had to extract the IP from X-Forwarded-For (XFF) because of Render’s proxy.
Rate limiting is also temporary, i.e., it lives in RAM as long as the server is alive. I haven’t added persistent storage, but that’s not a problem right now because this isn’t a high-stakes project with significant traffic. It can be fixed later as well.
The rate-limiter state is not shared across multiple instances. If there were multiple instances of the server, each would have its own rate-limiter state. If someone’s request is handled by Server A, their tokens are stored there, and if their next request is served by Server B, they get a separate token bucket there. This effectively allows them to bypass the rate limit across instances. Since there is no real traffic and only one server instance, this isn’t much of a problem right now. A Redis-backed counter would survive restarts and correctly share the rate limit across any number of instances.
There is also a small goroutine called sweep() that runs every 5 minutes. It checks the last refill time for each IP, and if an IP has not been used for more than 10 minutes, we remove it from the map containing the IPs. This prevents the map from growing indefinitely as new IPs make requests.
6. Async Click Analytics
type clickEvent struct {
Code string
Timestamp time.Time
UserAgent string
Referrer string
}
var clickChan = make(chan clickEvent, 1000)
func startClickWorker(pool *pgxpool.Pool) {
for evt := range clickChan {
_, err := pool.Exec(context.Background(),
"INSERT INTO clicks (code, clicked_at, user_agent, referrer) VALUES ($1, $2, $3, $4)",
evt.Code, evt.Timestamp, evt.UserAgent, evt.Referrer)
if err != nil {
log.Printf("click insert failed: %v", err)
}
}
}
go startClickWorker(pool)I use a buffered Go channel with one worker goroutine that takes click events and stores them in the database. I didn’t use Kafka or another message queue, partly just for fun and partly to use goroutines because why not. If the channel is full, one click event is dropped instead of blocking the request.
7. Why 302 Over 301?
A 301 response can be cached by the browser, so subsequent requests may not reach the server and the browser redirects directly to the destination. A 302 response is generally not treated as a permanent redirect, so requests continue to reach the server, allowing us to perform click analytics.
8. Benchmarks
(a) Cache hit vs. miss, local: 0.85 ms vs. 4.25 ms average, which is roughly a 5× improvement.
(b) Cache hit under concurrency, local, c=20: p50 1.5 ms, p90 3.7 ms, p99 23.6 ms, and ~7,473 requests/sec.
What it means: Under 20 concurrent requests, the system handles roughly 7,473 requests per second, while the p99 latency is higher due to mutex contention in the rate limiter.
(c) Production latency, warm instance: ~666 ms average (using curl), dominated by network RTT and Cloudflare rather than application logic. One observed cold-start outlier was 14.5 s, which is a known free-tier limitation of Render.
(d) Rate limiter under flood, production: 186/200 requests (93%) were correctly rejected with 429 under a concurrency of 20.