Implemented Proxy Service with Redis cache and rate limiter

forex-mtl/Dockerfile

@@ -0,0 +1,22 @@
+FROM openjdk:11-jdk
+
+# Set working directory
+WORKDIR /app
+
+# Copy project files
+COPY . .
+
+# Install Scala 2.13.12
+RUN \
+  curl -L https://github.com/sbt/sbt/releases/download/v1.9.0/sbt-1.9.0.tgz | tar xz -C /usr/local && \
+  ln -s /usr/local/sbt/bin/sbt /usr/local/bin/sbt && \
+                                                      sbt assembly
+
+
+# Expose port 9090 for your service
+EXPOSE 9090
+
+COPY src/main/resources/application.conf ./application.conf
+
+# Build your project
+CMD ["java", "-Dconfig.file=/app/application.conf", "-jar", "target/scala-2.13/forex-assembly-1.0.1.jar"]

forex-mtl/README.md

@@ -0,0 +1,175 @@
+# A local proxy for Forex rates
+
+This project is an implementation of the test task from Paidy - https://github.com/paidy/interview/blob/master/Forex.md
+
+## Assumptions and limitations
+
+1. Limit for calling this proxy is 10000 requests per day per token
+2. Limit for calling external service is 1000 requests per day
+3. Rate returned by the proxy shouldn't be older than 5 minutes
+4. Proxy should be able to serve at least 10000 requests per day
+5. It is possible that external service could fail or time out
+6. Rates are not invertible, i.e. AUD-SGD != 1 / SGD-AUD
+
+## Implementation
+
+We have 4 major components in the project:
+
+- `OneFrameService` which is a client for the external service for getting live rates.
+- `CurrencyRateCacheAlgebra` which is a cache injected into OneFrameService for storing cached rates, this can be
+  implemented in different ways, for example using Redis or in-memory cache. We choose **Redis**.
+- `TokenProvider` which is injected into OneFrameService for providing tokens for the external service. This can be
+  implemented in different ways, for example using Redis or config or SQL. We choose **Redis**.
+- `RateLimitterAlgebra` which wraps the Http Calls to the Proxy and checks if the token is valid and if the rate limit
+  is not exceeded. This can be implemented in different ways, for example using Redis or config or SQL. We choose
+  **Redis**.
+
+### Redis as design choice
+
+I choose Redis for 3 purposes:
+
+#### Cache, for caching of the rates for 5 minutes
+
+Pros
+
+1. This cache will be distributed and scalable (if we need to scale the Proxy horizontally)
+2. It will be fast, since it's in-memory
+3. It will be persistent, so we can recover from failures
+
+Cons
+
+1. We need to maintain another service instead of caching in our server.
+2. If we go live we would need to run Redis in cluster mode, otherwise, we will have a single point of failure.
+
+#### Rate limiter, for storing our proxy token and counting requests per token
+
+Pros
+
+1. If we have multiple instances of the Proxy, we need to have a shared storage for the tokens and their limits
+
+Cons
+
+1. If we do not have redis cluster, we will have a single point of failure.
+2. If network latency is high, we will have a high latency for the requests to the Proxy.
+3. We might need to use distributed locks for the rate limiter if we need high accuracy, which will add complexity to
+   the system.
+
+#### Token provider , for providing tokens for the external OneFrame service.
+
+Pros : Same as for the rate limiter
+
+Cons : Same as for the rate limiter. Additionally, if redis is down we will not be able to get tokens for the external
+service
+
+## Build and run
+
+### docker-compose
+
+Execute `docker-compose up --build` , this will build the image for the Proxy and pull the OneFrame image. After this,
+it will start three containers in the same network:
+
+- `forex-service` on **port 9000**
+- `redis` on **port 6379**
+- `one-frame` on **port 8080**
+
+### Accessing API
+
+## API Documentation
+
+### Endpoint: Exchange Rate Lookup
+
+Retrieves the exchange rate between two specified currencies.
+
+**URL:** `http://localhost:9090/rates`
+
+**Method:** `GET`
+
+#### URL Parameters
+
+| Parameter | Type   | Description              | Required |
+|-----------|--------|--------------------------|----------|
+| `from`    | string | The base currency code   | Yes      |
+| `to`      | string | The target currency code | Yes      |
+
+#### Supported Currencies
+
+- AUD (Australian Dollar)
+- CAD (Canadian Dollar)
+- CHF (Swiss Franc)
+- EUR (Euro)
+- GBP (British Pound)
+- NZD (New Zealand Dollar)
+- JPY (Japanese Yen)
+- SGD (Singapore Dollar)
+- USD (United States Dollar)
+
+#### Success Response
+
+**Code:** `200 OK`
+
+**Content example:**
+
+```json
+{
+  "from": "JPY",
+  "to": "EUR",
+  "price": 0.71810472617368925,
+  "timestamp": "2024-01-13T05:39:19.919Z"
+}
+```
+
+```shell
+curl 'http://127.0.0.1:9000/rates?from=USD&to=SGD'
+```
+
+#### Error Responses
+
+**Condition:** If token is invalid or missing.
+
+**Code:** `403 Forbidden`
+
+**Content:**
+
+```
+Invalid Token
+```
+
+**Condition:** If rate limit is exceeded.
+
+**Code:** `429 Too Many Requests`
+
+**Content:**
+
+``` 
+Token 123 exhausted the limit 1
+```
+
+**Condition:** If parameters are invalid or missing.
+
+**Code:** `400 Bad Request`
+
+**Content:**
+
+```
+Invalid currency
+```
+
+**Condition:** If the external service is unavailable or redis is down
+
+**Code:** `503 Service Unavailable`
+
+**Content:**
+
+```
+Service Unavailable
+```
+
+## Possible improvements
+
+- OneFrameService could be implemented as facade for multiple external services, for example, we could have multiple
+  external services for different currencies, we could aggregate them in OneFrameService
+- We could implement a local cache for the rates just as a fallback in case Redis is down
+- If we run multiple instances of the Proxy, the rate limiter will not work correctly, we need to use distributed locks
+  for the rate limiter if we need high accuracy, which will add complexity to the system.
+- More comprehensive tests to cover all scenarios
+- Metrics for checking response times and error rates as we are using Redis and external service

forex-mtl/build.sbt

@@ -1,4 +1,4 @@
-import Dependencies._
+import Dependencies.*
 
 name := "forex"
 version := "1.0.1"
@@ -62,8 +62,15 @@ libraryDependencies ++= Seq(
   Libraries.circeGenericExt,
   Libraries.circeParser,
   Libraries.pureConfig,
+  Libraries.sttp,
+  Libraries.redisClient,
+  Libraries.sttpCirce,
   Libraries.logback,
   Libraries.scalaTest      % Test,
   Libraries.scalaCheck     % Test,
   Libraries.catsScalaCheck % Test
 )
+
+
+mainClass := Some("forex.Main")
+assembly := (assembly dependsOn dependencyUpdates).value

forex-mtl/docker-compose.yml

@@ -0,0 +1,39 @@
+version: "3"
+
+networks:
+  backend:
+
+services:
+
+  redis:
+    image: redis/redis-stack:latest
+    ports:
+      - "6379:6379"  # Default Redis port
+      - "8001:8001"  # Redis UI port
+    networks:
+      - backend
+
+  oneFrame:
+    image: paidyinc/one-frame
+    ports:
+      - "8080:8080"
+    networks:
+      - backend
+
+
+  forex-service:
+    build:
+      context: ./
+      dockerfile: Dockerfile
+    ports:
+      - "9090:9090"
+    environment:
+      REDIS_HOST: redis
+      REDIS_PORT: 6379
+      ONE_FRAME_HOST: oneFrame
+      ONE_FRAME_PORT: 8080
+    depends_on:
+      - redis
+      - oneFrame
+    networks:
+      - backend

forex-mtl/project/Dependencies.scala

@@ -15,6 +15,8 @@ object Dependencies {
     val scalaCheck     = "1.15.3"
     val scalaTest      = "3.2.7"
     val catsScalaCheck = "0.3.2"
+    val sttp           = "3.9.1"
+    val redisClient   = "3.5.2"
   }
 
   object Libraries {
@@ -33,7 +35,9 @@ object Dependencies {
     lazy val circeGenericExt = circe("circe-generic-extras")
     lazy val circeParser     = circe("circe-parser")
     lazy val pureConfig      = "com.github.pureconfig" %% "pureconfig" % Versions.pureConfig
-
+    lazy val sttp            = "com.softwaremill.sttp.client3" %% "core"               % Versions.sttp
+    lazy val sttpCirce       = "com.softwaremill.sttp.client3" %% "circe"               % Versions.sttp
+    lazy val redisClient    =  "redis.clients" % "jedis"       % Versions.redisClient
     // Compiler plugins
     lazy val kindProjector = "org.typelevel" %% "kind-projector" % Versions.kindProjector cross CrossVersion.full
 

forex-mtl/project/plugins.sbt

@@ -1,3 +1,4 @@
 addSbtPlugin("com.lucidchart"    % "sbt-scalafmt-coursier" % "1.16")
 addSbtPlugin("com.timushev.sbt" % "sbt-updates" % "0.5.3")
 addSbtPlugin("io.spray" % "sbt-revolver" % "0.9.1")
+addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.15.0")

forex-mtl/src/main/resources/application.conf

@@ -1,8 +1,43 @@
 app {
   http {
     host = "0.0.0.0"
-    port = 8080
+    port = 9090
     timeout = 40 seconds
+    tokens = ["10dc303535874aeccc86a8251e6992f5", "10dc303535874aeccc86a8251e6992f6"]
+  }
+
+  one-frame {
+    http {
+      host = "localhost"
+      host = ${?ONE_FRAME_HOST}
+      port = 8080
+      port = ${?ONE_FRAME_PORT}
+      timeout = 20 seconds
+      token {
+        values = ["10dc303535874aeccc86a8251e6992f5"]
+        window-size = 86400 seconds
+        limit-per-token = 1000
+      }
+    }
+
+    retry-policy {
+      max-retries = 3
+      delay = 10 seconds
+    }
+    rates-refresh = 5 minutes
+  }
+
+  redis {
+    host = "localhost"
+    host = ${?REDIS_HOST}
+    port = 6379
+    port = ${?REDIS_PORT}
+  }
+
+  rate-limit {
+    limit-per-token = 10000
+    window-size = 86400 seconds
   }
 }
 
+

forex-mtl/src/main/scala/forex/Module.scala

@@ -1,21 +1,44 @@
 package forex
-
-import cats.effect.{ Concurrent, Timer }
+import cats.effect._
+import cats.implicits._
+import forex.cache.RedisCurrencyRateCache
 import forex.config.ApplicationConfig
+import forex.http.ratelimitter.RedisRateLimiter
+import forex.http.ratelimitter.interpreters.RateLimitterAlgebra
 import forex.http.rates.RatesHttpRoutes
-import forex.services._
 import forex.programs._
+import forex.services._
+import forex.services.rates.token.{RedisTokenCache, TokenCacheAlgebra, TokenProvider}
 import org.http4s._
 import org.http4s.implicits._
-import org.http4s.server.middleware.{ AutoSlash, Timeout }
+import org.http4s.server.middleware.{AutoSlash, Timeout}
+import org.slf4j.LoggerFactory
+import redis.clients.jedis.Jedis
+import sttp.client3.{HttpURLConnectionBackend, Identity, SttpBackend}
 
 class Module[F[_]: Concurrent: Timer](config: ApplicationConfig) {
-
-  private val ratesService: RatesService[F] = RatesServices.dummy[F]
+  private val logger                      = LoggerFactory.getLogger(getClass)
+  val backend: SttpBackend[Identity, Any] = HttpURLConnectionBackend()
+  private val jedisResource: Resource[F, Jedis] = Resource.make(
+    Async[F].delay(new Jedis(config.redis.host, config.redis.port)) // Acquire the resource
+  ) { jedis =>
+    Async[F].delay(jedis.close()).handleErrorWith { e =>
+      logger.error("Failed to close jedis resource", e)
+      Async[F].unit
+    }
+  }
+  private val currencyCache: RedisCurrencyRateCache[F] = new RedisCurrencyRateCache[F](jedisResource)
+  private val oneFrameTokensCache: F[TokenCacheAlgebra[F]] =
+    RedisTokenCache[F](
+      jedisResource,
+      config.oneFrame.http.token)
+  private val tokenProvider: TokenProvider[F] = new TokenProvider[F](oneFrameTokensCache)
+  private val ratesService: RatesService[F]   = RatesServices.live(config.oneFrame, backend, currencyCache, tokenProvider)
 
   private val ratesProgram: RatesProgram[F] = RatesProgram[F](ratesService)
 
-  private val ratesHttpRoutes: HttpRoutes[F] = new RatesHttpRoutes[F](ratesProgram).routes
+  val ratesLimiter: F[RateLimitterAlgebra[F]] = RedisRateLimiter[F](jedisResource, config.rateLimit, config.http.tokens)
+  private val ratesHttpRoutes: HttpRoutes[F]  = new RatesHttpRoutes[F](ratesProgram, ratesLimiter).routes
 
   type PartialMiddleware = HttpRoutes[F] => HttpRoutes[F]
   type TotalMiddleware   = HttpApp[F] => HttpApp[F]

forex-mtl/src/main/scala/forex/cache/CurrencyRateCacheAlgebra.scala

@@ -0,0 +1,8 @@
+package forex.cache
+
+import forex.domain.Rate
+import forex.cache.errors.Error
+trait CurrencyRateCacheAlgebra[F[_]] {
+  def getRates(key: String): F[Error Either Rate]
+  def updateRates(key: String, rate: Rate, timeoutInSeconds: Int): F[String]
+}