Modern C++ Systems Project: Building a High-Performance In-Memory Cache

Why College C++ Keeps You Trapped at 3.5 LPA

Most Indian engineering colleges still teach C++ like it is 1998. Professors force students to use Turbo C++ on Windows XP emulators, memorize manual malloc() syntax, and build terminal student management systems using raw pointers and naked new/delete calls.

If you submit a project like that to an engineering team at a systems company, game studio, or high-frequency trading firm, your application is rejected immediately. Production C++ in 2026 does not look anything like college assignments. Modern C++ (C++20 and C++23) relies on Resource Acquisition Is Initialization (RAII), smart pointers, atomic concurrency primitives, and move semantics that prevent memory leaks before code ever compiles.

In this guide, we will build a real systems project: a Thread-Safe In-Memory LRU (Least Recently Used) Cache with TTL Expiration written in modern C++20. This project demonstrates concurrency control, custom data structure composition, and zero memory leaks.

The Architecture of an LRU Cache with TTL

An LRU cache discards the least recently accessed elements when it reaches maximum capacity. To make it production-grade, our cache must support three guarantees:

• O(1) Average Lookup and Insertion: Using a hash map (std::unordered_map) mapping keys to doubly linked list iterators.
• O(1) Eviction: Maintaining access order using a doubly linked list (std::list). When a key is read or written, it moves to the front of the list.
• Thread Safety with Reader-Writer Concurrency: Using std::shared_mutex. Multiple threads can read simultaneously without blocking each other, while writes acquire an exclusive lock.
• TTL Expiration: Items expire after a configurable duration using std::chrono::steady_clock.

Here is the architectural memory layout:

Thread Read/Write Requests
         │
         ▼
[ std::shared_mutex (Shared Read / Exclusive Write) ]
         │
         ├──► std::unordered_map<Key, ListIterator> (O(1) Direct Key Lookup)
         │             │
         │             ▼
         └──► std::list<CacheItem> (O(1) Splice to Front / Evict from Back)
                   │
                   └── CacheItem { key, value, expiry_time }

Step 1: Implementing the Cache Class in C++20

Here is the complete, self-contained implementation. Notice how we use templates for generic keys and values, std::optional for safe lookup returns, and std::shared_mutex for multi-threaded safety.

#include <iostream>
#include <unordered_map>
#include <list>
#include <optional>
#include <shared_mutex>
#include <chrono>
#include <string>
#include <thread>
#include <vector>

template <typename Key, typename Value>
class ThreadSafeLRUCache {
public:
    using Clock = std::chrono::steady_clock;
    using TimePoint = std::chrono::time_point<Clock>;

    struct CacheItem {
        Key key;
        Value value;
        TimePoint expiry;
    };

    explicit ThreadSafeLRUCache(size_t capacity, std::chrono::milliseconds default_ttl)
        : capacity_(capacity), default_ttl_(default_ttl) {}

    // Read operation: Shared lock permits multiple concurrent readers
    std::optional<Value> get(const Key& key) {
        std::unique_lock<std::shared_mutex> lock(mutex_);

        auto map_it = index_.find(key);
        if (map_it == index_.end()) {
            return std::nullopt;
        }

        auto list_it = map_it->second;

        // Check TTL expiration
        if (Clock::now() >= list_it->expiry) {
            items_.erase(list_it);
            index_.erase(map_it);
            return std::nullopt;
        }

        // Move accessed item to front of the LRU list
        items_.splice(items_.begin(), items_, list_it);
        return list_it->value;
    }

    // Write operation: Exclusive lock blocks readers and other writers
    void put(const Key& key, const Value& value, std::optional<std::chrono::milliseconds> custom_ttl = std::nullopt) {
        std::unique_lock<std::shared_mutex> lock(mutex_);

        auto ttl = custom_ttl.value_or(default_ttl_);
        auto expiry = Clock::now() + ttl;

        auto map_it = index_.find(key);
        if (map_it != index_.end()) {
            // Update existing entry
            auto list_it = map_it->second;
            list_it->value = value;
            list_it->expiry = expiry;
            items_.splice(items_.begin(), items_, list_it);
            return;
        }

        // Evict LRU element if capacity is reached
        if (items_.size() >= capacity_) {
            auto last_it = std::prev(items_.end());
            index_.erase(last_it->key);
            items_.pop_back();
        }

        // Insert new entry at the front
        items_.push_front(CacheItem{key, value, expiry});
        index_[key] = items_.begin();
    }

    size_t size() const {
        std::shared_lock<std::shared_mutex> lock(mutex_);
        return items_.size();
    }

private:
    size_t capacity_;
    std::chrono::milliseconds default_ttl_;
    mutable std::shared_mutex mutex_;

    std::list<CacheItem> items_;
    std::unordered_map<Key, typename std::list<CacheItem>::iterator> index_;
};

Step 2: Multi-Threaded Stress Testing

Writing multi-threaded code is useless unless you test it under heavy race conditions. We spawn 10 concurrent reader and writer worker threads to ensure zero data races or segmentation faults.

int main() {
    // Create cache with capacity of 1,000 items and 500ms default TTL
    ThreadSafeLRUCache<std::string, std::string> cache(1000, std::chrono::milliseconds(500));

    std::vector<std::jthread> workers;
    const int num_threads = 8;
    const int ops_per_thread = 5000;

    std::cout << "Starting concurrent read/write stress test...\n";

    for (int t = 0; t < num_threads; ++t) {
        workers.emplace_back([&cache, t, ops_per_thread]() {
            for (int i = 0; i < ops_per_thread; ++i) {
                std::string key = "session_" + std::to_string((i + t * 100) % 1500);
                std::string val = "token_" + std::to_string(i);

                if (i % 3 == 0) {
                    cache.put(key, val);
                } else {
                    auto res = cache.get(key);
                }
            }
        });
    }

    // std::jthread automatically joins upon destruction
    workers.clear();

    std::cout << "Stress test completed cleanly. Final cache size: " << cache.size() << "\n";
    return 0;
}

Step 3: Compiling and Memory Sanitization

Never deliver C++ code without compiling against GCC or Clang sanitizers. Sanitizers inject instrumentation into the binary to detect memory leaks, buffer overflows, and race conditions at runtime.

# Compile with C++20, AddressSanitizer, and UndefinedBehaviorSanitizer
g++ -std=c++20 -O2 -fsanitize=address,undefined -Wall -Wextra main.cpp -o cache_demo
./cache_demo

# Compile with ThreadSanitizer to catch concurrency data races
g++ -std=c++20 -O2 -fsanitize=thread -Wall -Wextra main.cpp -o cache_thread_demo
./cache_thread_demo

If ThreadSanitizer exits with zero warnings, your mutex locking boundaries are solid. If there is a missing lock or a dangling iterator, ThreadSanitizer prints the exact stack trace and conflicting thread IDs.

What to Write on Your Resume

Do not write: "Learned C++ and built an in-memory cache."

Write this instead:

"Architected a thread-safe, templated LRU cache with TTL expiration in C++20 using std::shared_mutex and std::unordered_map iterator splicing. Validated zero data races under 40,000 concurrent operations across 8 worker threads using ThreadSanitizer (TSan)."

When an interviewer sees this bullet point, they know you understand concurrency locks, cache eviction policies, iterator invalidation rules, and modern tooling.

Next Steps to Expand This Project

Once you have the core cache working, here are three ways to take it to the principal engineering level:

• Custom Memory Pool Allocator: Replace the standard heap allocator for std::list nodes with a custom fixed-size memory pool (slab allocator) to eliminate heap fragmentation.
• Non-Blocking Cleaners: Run an asynchronous worker thread that periodically sweeps expired TTL keys instead of checking them lazily on read.
• Network Server Wrapper: Wrap your C++ cache in an asynchronous TCP server using asio or epoll to build a drop-in, lightweight Redis substitute.

To master low-level concepts and system design, explore our breakdowns on C and C++ interview programs, foundational C programming practices, other resume-worthy coding projects, and our end-to-end framework for building real-world projects.