04. Zero-Cost 3-Tier Storage Lifecycle
AWS SigV4 Pure WebCrypto Presigning + Cloudflare R2 $0 Egress Cap + imgproxy Cold Transcoder eliminating 85% of cloud asset costs.
04. Zero-Cost 3-Tier Storage Lifecycle
Production Paradigm: Pure WebCrypto AWS SigV4 Presigning + Free Tier Egress Hard-Cap + Automated Local SSD/NVMe Spillover + Dynamic imgproxy WebP/AVIF Transcoding.
๐ฅ The Real-World Bottleneck
Cloud media storage is one of the fastest-growing cost centers for SaaS, video platforms, and e-commerce applications:
- Predatory Egress Fees: AWS S3 charges up to $0.09 per GB of outbound bandwidth. Serving 10 TB of images costs $900/month just for data transfer.
- Heavy SDK Overhead: The official
@aws-sdk/client-s3bundle weighs over 500 KB and pulls dozens of transitive modules, slowing down serverless cold starts. - Unoptimized Bloated Assets: Users upload raw 15 MB PNG and JPEG files from smartphone cameras, which sit indefinitely on expensive hot tiers without compression.
๐๏ธ High-Concurrency Architecture
This blueprint implements an intelligent 3-tier storage waterfall: pure WebCrypto AWS SigV4 presigning, automatic fallback when free-tier quotas are reached, and dynamic cold asset transcoding.
Key Architectural Phases
- Pure WebCrypto SigV4 Presigning (Zero Dependencies):
- Signs S3 and Cloudflare R2 upload URLs using standard Web Crypto API (
crypto.subtle). - Zero external AWS SDK dependencies, shrinking bundle size by 99% and enabling sub-millisecond presigning in Cloudflare Workers, Bun, and Node.js.
- Signs S3 and Cloudflare R2 upload URLs using standard Web Crypto API (
- R2 Budget Guard & Circuit Breaker:
- Continuously tracks monthly upload volume against Cloudflare R2's 10 GB free tier.
- When the threshold is reached, automatically routes non-critical assets to local VPS NVMe storage, eliminating surprise cloud bills.
- Automated imgproxy Cold Transcoder:
- Assets inactive for >30 days are converted on-the-fly to modern WebP / AVIF formats.
- Slashes storage space by 75โ85% while maintaining visual fidelity.
๐งฉ Composed Functional Slices
| Slice | File Path | Core Role |
|---|---|---|
s3-r2 | packages/storage-uploader/src/adapters/s3-r2.ts | Zero-dependency Pure WebCrypto AWS SigV4 presigned PUT/GET generator (R2Presigner, generatePresignedPutUrl) |
budget-guard | packages/storage-uploader/src/budget-guard/ | Monthly usage circuit breaker (evaluateStorageBudgetGuard, checkCloudflareR2Budget) |
tiered-lifecycle | packages/storage-uploader/src/tiered-lifecycle/ | Automated tier resolution (resolveUploadTargetTier, buildImgproxyColdTranscodeUrl) |
๐ป Code Implementation
1. Copy Slices into Your Project
mkdir -p src/lib/storage
cp packages/storage-uploader/src/adapters/s3-r2.ts src/lib/storage/
cp -r packages/storage-uploader/src/budget-guard src/lib/storage/
cp -r packages/storage-uploader/src/tiered-lifecycle src/lib/storage/2. Implementation (src/services/storage.ts)
import { R2Presigner } from '@/lib/storage/s3-r2';
import { evaluateStorageBudgetGuard } from '@/lib/storage/budget-guard';
import { buildImgproxyColdTranscodeUrl } from '@/lib/storage/tiered-lifecycle';
// 1. Zero-dependency R2 presigner
const presigner = new R2Presigner({
accountId: process.env.CLOUDFLARE_ACCOUNT_ID!,
accessKeyId: process.env.R2_ACCESS_KEY_ID!,
secretAccessKey: process.env.R2_SECRET_ACCESS_KEY!,
bucketName: 'production-assets',
});
export async function getDirectUploadUrl(filename: string, fileSizeBytes: number) {
// 2. Check if we are within the $0 monthly budget limit
const budgetDecision = await evaluateStorageBudgetGuard({
incomingBytes: fileSizeBytes,
currentMonthlyUsageBytes: await getCurrentMonthlyUsage(),
maxMonthlyFreeBytes: 10 * 1024 * 1024 * 1024, // 10 GB
});
if (!budgetDecision.allowCloudflareR2) {
// Fallback to self-hosted SSD endpoint
return { uploadUrl: `/api/upload/local-ssd?file=${encodeURIComponent(filename)}` };
}
// 3. Generate presigned PUT URL in <0.5ms
const presigned = await presigner.generatePresignedPutUrl({
objectKey: `uploads/${Date.now()}-${filename}`,
contentType: 'image/webp',
expiresInSeconds: 300,
});
return { uploadUrl: presigned.url, objectKey: presigned.objectKey };
}๐ Production Scale Benchmark & Verification
| Storage Architecture | Standard AWS S3 + CloudFront | Kido 3-Tier Lifecycle | Savings |
|---|---|---|---|
| Egress Bandwidth (10 TB/mo) | $900.00 / month | $0.00 (Cloudflare R2) | 100% Egress Cut |
| Serverless Cold Start | +180ms (@aws-sdk/client-s3) | 0ms (Pure WebCrypto) | Instant Cold Start |
| Cold Asset Disk Footprint | 100% uncompressed | 18% (imgproxy WebP) | 82% Disk Savings |
| Surprise Bill Protection | None (uncapped billing) | Hard Cap Circuit Breaker | Risk Eliminated |
03. Zero-Oversell Bounded Flash Sale Engine
Distributed Atomic Lua Reservation + Token Bucket Limiting + Transactional Outbox handling 50k req/s with zero negative inventory.
05. Twitter/X Hybrid Fanout Feed Engine
Push-Write + Pull-Read + O(N log K) Binary Max-Heap Cursor Merger slashing database load from 20,600 req/s to 1,201 req/s.