Simple Caching Layer

🚀 Project Overview

The Advanced Caching Layer is a backend service designed to enhance API performance and reliability through caching. It supports bulk retrieval, automatic cache invalidation, and robust error handling. The service is built with Node.js, TypeScript, and Redis, providing high-speed data access with minimal latency.

✨ Features

• Automatic Cache Invalidation: Set time-to-live (TTL) and clear outdated data.
• Stale-While-Revalidate: Serve stale data while revalidating in the background.
• Bulk Cache Retrieval: Efficiently fetch multiple keys in one go.
• Monitoring and Metrics: Gather insights into cache performance.
• Advanced Error Handling: Gracefully handle errors and log them.
• Configurable via Environment Variables: Fine-tune caching behavior.
• Choose max key amount of max memory allowance: Whichever you prefer.

---

🛠️ Environment Setup

To configure the caching layer, create a .env file with the following variables:

REDIS_HOST=127.0.0.1
REDIS_PORT=6379
CACHE_TTL=3600 # in seconds
LOG_LEVEL=info
PORT=3000
CACHE_THRESHOLD_METHOD=key  # or "memory" for memory-based threshold
MAX_CACHE_SIZE=10000 # amount of keys
MAX_MEMORY_USAGE=1024 # in MB
MAX_RETRIES=3 # Number of allowed retries per fetch
FETCH_NEW_DATA_ON_BULK=false # Set to true to fetch fresh data on bulk requests
DEV_MODE=true # Enable detailed logs for development

---

💻 Installation

1. Clone the repository:

   git clone https://github.com/username/advanced-caching-layer.git
   cd advanced-caching-layer

2. Install dependencies:

   npm install

3. Make sure you have Redis setup and running. This project depends on it.

   // in .env
   REDIS_HOST=your_redis_host
   REDIS_PORT=your_redis_port

4. Start the server:

   npm start

---

🚦 API Endpoints

Health Check

GET /

• Response:

  Everything is properly setup and working. 🚀
  

Fetch Data with Caching

GET /cache?url=<API_URL>

• Query Parameter:
  • url: The API URL to fetch and cache.
• Response: Returns cached or freshly fetched data.

Invalidate Cache

POST /cache/invalidate

• Request Body:

  {
    "key": "<CACHE_KEY>"
  }

• Response:

  Cache invalidated for key: <CACHE_KEY>
  

Fetch All Cache Keys

GET /cache/keys?pattern=<REGEX_PATTERN>

• Response: List of keys matching the pattern.

Bulk Fetch Data

GET /cache/bulk?keys=<key1>&keys=<key2>&keys=<key3>

• Response:

  {
    "result": [
      "data1",
      "data2",
      null
    ]
  }

Metrics

GET /metrics

• Response: Metrics in Prometheus format.

---

📝 Examples

Fetching Data with Caching

curl 'http://localhost:3000/cache?url=https://api.example.com/data'

Bulk Fetch

curl 'http://localhost:3000/cache/bulk?keys=https://api.example.com/data1&keys=https://api.example.com/data2'

Invalidate Cache

curl -X POST 'http://localhost:3000/cache/invalidate' -d '{"key": "https://api.example.com/data1"}' -H 'Content-Type: application/json'

---

📊 Performance and Optimization Tips

• Set CACHE_TTL according to your data volatility.
• Adjust MAX_CACHE_SIZE and MAX_MEMORY_USAGE to balance performance and resource usage.
• Enable FETCH_NEW_DATA_ON_BULK only if real-time data accuracy is crucial.

---

📝 License

This project is licensed under the MIT License.

🤗 Contributing

Any contributions are welcome and very appreciated. 💖 If it's a small change, you can directly open a PR. If it's a massive change, please open an issue first so it can be talked about.