Rearchitecting: Redis to SQLite | Wafris

The Nugget

  • Wafris shifted from Redis to SQLite to simplify deployment and enhance performance for their application firewall, resulting in a 3x speed improvement over Redis in their specific use case.

Make it stick

  • πŸš€ SQLite is faster than Redis due to eliminating network latency, essential for Wafris's middleware.
  • πŸ”„ Wafris v2 achieves improved onboardingβ€”users just need to add a gem, streamlining setup.
  • πŸ—οΈ SQLite's design offers near-infinite horizontal scaling, ideal for high-demand web apps.
  • πŸ“‰ The new architecture decouples reads (fast rule checks) from writes (slower reporting), optimizing performance.

Key insights

The Need for Change

  • The initial v1 client relied on a user-managed Redis datastore, leading to deployment difficulties for users.
  • At RailsWorld 2023, there was notable negativity around the idea of requiring a Redis server alongside applications.
  • Wafris's goal: make web app protection easy, highlighting the burden of having users act as Redis admins.

Performance Considerations

  • Redis, while performant, introduced network latency issues, negatively affecting application speed.
  • SQLite is touted as competing with file access rather than traditional databases, leading to significant speed benefits in specific contexts.
  • Benchmark testing showed SQLite providing better speeds for Wafris's critical IP collection queries.

Architectural Evolution

  • Wafris's model shifted focus to local database instances (SQLite) that eliminate network calls for rule evaluation.
  • The architecture was redesigned to separate read (filters) and write (reporting) processes, significantly improving efficiency.
  • The ability to sync SQLite databases to multiple application instances addresses common load challenges faced under heavy traffic.

Write Operation Management

  • Writes are handled asynchronously, with batch processing designed to keep the SQLite DB lightweight.
  • This method ensures users don’t face the burdens of direct database writes, aligning with Wafris's mission for simplicity and speed.

Key quotes

  • "We shouldn't casually make you a Redis database admin in the process."
  • "SQLite does not compete with client/server databases. SQLite competes with fopen()."
  • "SQLite beat Redis like an overhyped UFC fighter doing a jump off the fence to land a kick punch."
  • "Making Wafris easy to deploy... flows from that."
  • "We're exceedingly happy with our v2 architecture, which uses SQLite."
This summary contains AI-generated information and may be misleading or incorrect.