@kido/resilient-cacheCopy Anchor Link

Architecture Paradigm: Two-Tier L1 Memory / L2 Remote Hierarchy + In-Flight SingleFlight Request Deduplication + Fail-Open Circuit Breaker + Atomic Lua Script Stock Reservation + In-Memory Semantic Vector Cache with RRF Fusion.

💥 The Real-World BottleneckCopy Anchor Link

High-traffic applications suffer from severe backend collapse when caching strategies neglect concurrency hazards:

1. The Cache Stampede (Thundering Herd) Disaster: When a popular cache key expires (e.g., product details during a flash sale or live stock ticker), 10,000 concurrent requests experience a cache miss at the exact same millisecond. All 10,000 requests hit the primary PostgreSQL/MySQL database simultaneously, exhausting connection pools and causing cascading 504 Gateway Timeouts.
2. The "Redis Down, Everything Down" Fragility: Standard caching libraries throw fatal unhandled errors when Redis restarts or encounters network partitions. The entire application crashes even though the primary SQL database is perfectly healthy.
3. Flash-Sale Inventory Overselling (The Distributed Race Condition): Checking stock via SELECT stock FROM inventory followed by UPDATE inventory SET stock = stock - 1 fails under concurrent traffic. 50 users can purchase the last 2 remaining items because their read transactions execute before any write locks commit.

🏛️ 3-Tier Production Evolution LadderCopy Anchor Link

📦 Architecture & Core API ExportsCopy Anchor Link

@kido/resilient-cache is split into decoupled, surgical slices:

1. In-Flight SingleFlight & SWR (src/in-flight.ts)Copy Anchor Link

• InFlightManager: Deduplicates concurrent async executions on matching keys. Ensures only 1 downstream query executes while all other callers await the exact same promise.
• SingleFlightSwr: Stale-While-Revalidate micro-cache serving immediate stale values while revalidating cache in the background.
• getGlobalSingleFlight(): Global singleton accessor for application-wide singleflight deduplication.

2. Multi-Tier Cache & Circuit Breaker (src/cache.ts, src/circuit-breaker.ts)Copy Anchor Link

• ResilientCache: Two-tier caching engine orchestrating L1 in-memory LRU and L2 remote Redis/Valkey.
• MemoryL1Cache: High-throughput in-process LRU cache with sub-millisecond TTL eviction.
• CircuitBreaker: Resilient fail-open state machine (CLOSED, OPEN, HALF_OPEN) preventing cascading outages during downstream service downtime.

3. Atomic Lua Flash-Sale Engine (src/flashsale-engine/)Copy Anchor Link

• FlashSaleEngine: High-concurrency inventory reservation engine.
• RESERVE_STOCK_LUA: Atomic Lua script executing inventory bounds check and decrement in a single atomic cycle (HINCRBYBYFLOAT / EVAL).
• ROLLBACK_STOCK_LUA: Atomic stock restoration when payments fail or checkout sessions expire.

4. Semantic Vector Cache (src/semantic-cache/)Copy Anchor Link

• SemanticVectorCache: In-memory semantic similarity cache using cosine distance thresholds ($\ge 0.92$) for AI LLM queries.
• fuseRrfRankings(denseRankings, sparseRankings, k): Reciprocal Rank Fusion (RRF) algorithm combining lexical and semantic search ranks.
• buildPostgresRrfCteSql(): Generates optimized PostgreSQL pgvector CTE queries.

💻 Production Implementation & Code SnippetCopy Anchor Link

SingleFlight Database Protection & Lua Stock ReservationCopy Anchor Link

import { 
  InFlightManager, 
  MemoryL1Cache, 
  FlashSaleEngine 
} from '@kido/resilient-cache';

// 1. Initialize In-Flight SingleFlight Deduplicator
const singleFlight = new InFlightManager();
const l1Cache = new MemoryL1Cache<any>({ maxItems: 10000, defaultTtlMs: 60000 });

// 2. High-Traffic Product Fetcher (Zero Thundering Herd)
export async function getProductDetails(productId: string) {
  const cacheKey = `product:${productId}`;
  
  // A. Check L1 Memory Cache (< 0.05ms)
  const cached = l1Cache.get(cacheKey);
  if (cached) return cached;

  // B. SingleFlight: 1,000 concurrent requests collapse into 1 single DB call
  return await singleFlight.do(cacheKey, async () => {
    // Heavy relational database lookup
    const product = await fetchProductFromDatabase(productId);
    l1Cache.set(cacheKey, product, 30000); // 30s TTL
    return product;
  });
}

// 3. Atomic Flash-Sale Stock Reservation
const flashSale = new FlashSaleEngine({
  async executeLua(script, keys, args) {
    // Bind to your Redis/Valkey client (ioredis / node-redis)
    return await redisClient.eval(script, keys.length, ...keys, ...args);
  },
});

export async function reserveFlashSaleItem(userId: string, campaignId: string, itemId: string) {
  const result = await flashSale.reserveStock({
    campaignId,
    itemId,
    userId,
    quantity: 1,
    lockTtlSeconds: 120, // 2-minute checkout hold
  });

  if (!result.success) {
    return { success: false, reason: result.errorCode }; // 'OUT_OF_STOCK' | 'USER_LIMIT_EXCEEDED'
  }

  return { success: true, reservationId: result.reservationId };
}

⚡ Benchmark & Verified k6 PerformanceCopy Anchor Link

Simulated 20,000 concurrent virtual users querying an uncached endpoint simultaneously:

🚀 Surgical Extraction & CLI InstallationCopy Anchor Link

# Install directly via Kido CLI
npx @kido/cli add @kido/resilient-cache