- create a proxy API for https://pokeapi.co/, to "fetch a list of Pokemon from the api as part of a proxy request for a client application"
- TypeScript implementation
- reliability, performance, and scalability
- application should be able scale to a larger project
- for performance the most indicated runtime would be:
Bun.Bun,Deno,Node.jsall were tested and work (bun for fastest, deno for security, node for stability). WithBunthere is no need for transpilation. - for performance the API library used is fastify, other fast libraries are: fast-json-stringify, pino
- for performance
Redisis used as cache (for a proxy it doesn't make sense to store data in an external database) - although a risk (because reliability is unknown), in order to shorten development time, the lib
pokenode-tswas leveraged, as client to the pokeapi - in order to make this as real as possible (and easily understandable) the API documentation is provided via swagger. The API tries its best to be rich in information and simple
- for reliability a rate-limiter was configured (fastify)
- scaling for a larger project: the app was NOT configured as a monorepo because the use case does not indicate that need, for the scaling size requirement a sensible folder structure and config parameters were adopted
- the most scalable approach would be a
K8scluster with multiple instances of the app, an ingress loadbalancer and a node running aRedis
-+ root, docs and environment config
|
+ src + index.ts base app entrypoints
| |
| + config configurations
| + routes endpoint routes
| + controllers controllers
| + models schemas and models
| + utils shared utilities
|
+ tests + tests
| |
| + controllers
|
+ scripts containers creation
+ docs
- for simplicity only model transformation has been tested
- other tests done via cli using
curl(see sample calls)
- a dockerfile for a bun container is provided in the
scriptsfolder
- a local Redis instance is required (
podman run -dt -p 6379:6379 redis) - run:
bun src/index.tsdeno src/index.tsnpx ts-node src/index.ts
- just names, no details
http://localhost:3000/api/pokemon/v1?limit=20&offset=0
{
"success": true,
"data": [
"beedrill",
"blastoise",
"bulbasaur",
"butterfree",
"caterpie",
"charizard",
"charmander",
"charmeleon",
"ivysaur",
"kakuna",
"metapod",
"pidgeot",
"pidgeotto",
"pidgey",
"raticate",
"rattata",
"squirtle",
"venusaur",
"wartortle",
"weedle"
]
}
- with details
http://localhost:3000/api/pokemon/v1?limit=2&offset=100&detailed=true
{
"success": true,
"data": [
{
"name": "electrode",
"types": [
"electric"
],
"height": 12,
"base_experience": 172,
"moves": [
"headbutt",
"tackle",
"take-down",
"double-edge",
"sonic-boom",
"hyper-beam",
"thunder-shock",
"thunderbolt",
"thunder-wave",
"thunder",
"toxic",
"agility",
"rage",
"teleport",
"mimic",
"screech",
"double-team",
"light-screen",
"reflect",
"bide",
"self-destruct",
"swift",
"skull-bash",
"flash",
"explosion",
"rest",
"substitute",
"thief",
"snore",
"curse",
"protect",
"scary-face",
"zap-cannon",
"endure",
"rollout",
"swagger",
"spark",
"sleep-talk",
"return",
"frustration",
"hidden-power",
"rain-dance",
"mirror-coat",
"torment",
"facade",
"charge",
"taunt",
"helping-hand",
"magic-coat",
"secret-power",
"metal-sound",
"signal-beam",
"shock-wave",
"gyro-ball",
"natural-gift",
"sucker-punch",
"magnet-rise",
"giga-impact",
"discharge",
"charge-beam",
"telekinesis",
"electro-ball",
"foul-play",
"round",
"volt-switch",
"electroweb",
"wild-charge",
"confide",
"eerie-impulse",
"magnetic-flux",
"electric-terrain",
"tera-blast",
"supercell-slam"
],
"abilities": [
"soundproof",
"static",
"aftermath"
],
"held_items": []
},
{
"name": "exeggcute",
"types": [
"grass",
"psychic"
],
"height": 4,
"base_experience": 65,
"moves": [
"swords-dance",
"headbutt",
"take-down",
"double-edge",
"psybeam",
"strength",
"absorb",
"mega-drain",
"leech-seed",
"solar-beam",
"poison-powder",
"stun-spore",
"sleep-powder",
"toxic",
"confusion",
"psychic",
"hypnosis",
"rage",
"teleport",
"mimic",
"double-team",
"light-screen",
"reflect",
"bide",
"self-destruct",
"egg-bomb",
"dream-eater",
"barrage",
"flash",
"psywave",
"explosion",
"rest",
"substitute",
"thief",
"nightmare",
"snore",
"curse",
"protect",
"sludge-bomb",
"giga-drain",
"endure",
"rollout",
"swagger",
"attract",
"sleep-talk",
"return",
"frustration",
"synthesis",
"moonlight",
"hidden-power",
"sunny-day",
"psych-up",
"ancient-power",
"uproar",
"facade",
"nature-power",
"helping-hand",
"trick",
"ingrain",
"skill-swap",
"imprison",
"secret-power",
"extrasensory",
"bullet-seed",
"block",
"gravity",
"natural-gift",
"lucky-chant",
"power-swap",
"worry-seed",
"seed-bomb",
"energy-ball",
"zen-headbutt",
"trick-room",
"leaf-storm",
"captivate",
"grass-knot",
"psyshock",
"telekinesis",
"round",
"stored-power",
"bestow",
"grassy-terrain",
"confide",
"infestation",
"grassy-glide",
"tera-blast",
"psychic-noise"
],
"abilities": [
"chlorophyll",
"harvest"
],
"held_items": [
"psychic-seed"
]
}
]
}
- a single Pokémon
http://localhost:3000/api/pokemon/v1/ditto
{
"success": true,
"data": {
"name": "ditto",
"types": [
"normal"
],
"height": 3,
"base_experience": 101,
"moves": [
"transform"
],
"abilities": [
"limber",
"imposter"
],
"held_items": [
"metal-powder",
"quick-powder"
]
}
}
- caching behavior: currently all requests are cached, but the code could be changed to only cache individual Pokémon
Note
note: bun runtime; cpu Core i5 6th Gen (Skylake); memory DDR4, 2400 MT/s (slow machine, multiple apps running, power saving mode)
- three load testing tools were used providing results over 1.1k requests/sec for 200 clients
bombardier -c 200 -n 10000 --print=i,r --http2 "http://localhost:3000/api/pokemon/v1?limit=20&offset=0"
Bombarding http://localhost:3000/api/pokemon/v1?limit=20&offset=0 with 10000 request(s) using 200 connection(s)
Statistics Avg Stdev Max
Reqs/sec 1381.75 2380.00 10407.82
Latency 142.75ms 23.56ms 284.20ms
HTTP codes:
1xx - 0, 2xx - 10000, 3xx - 0, 4xx - 0, 5xx - 0
others - 0
Throughput: 749.90KB/s
ab -q -n10000 -c200 "http://localhost:3000/api/pokemon/v1?limit=20&offset=0"
This is ApacheBench, Version 2.3 <$Revision: 1913912 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking localhost (be patient).....done
Server Software:
Server Hostname: localhost
Server Port: 3000
Document Path: /api/pokemon/v1?limit=20&offset=0
Document Length: 246 bytes
Concurrency Level: 200
Time taken for tests: 8.871 seconds
Complete requests: 10000
Failed requests: 0
Total transferred: 4310000 bytes
HTML transferred: 2460000 bytes
Requests per second: 1127.24 [#/sec] (mean)
Time per request: 177.424 [ms] (mean)
Time per request: 0.887 [ms] (mean, across all concurrent requests)
Transfer rate: 474.45 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 10 6.9 8 43
Processing: 69 164 20.5 161 281
Waiting: 26 161 21.2 157 280
Total: 69 174 20.8 169 284
Percentage of the requests served within a certain time (ms)
50% 169
66% 177
75% 182
80% 187
90% 199
95% 216
98% 238
99% 247
100% 284 (longest request)
h2load -n 10000 -c 200 --warm-up-time=2 --h1 "http://localhost:3000/api/pokemon/v1?limit=20&offset=0"
starting benchmark...
spawning thread #0: 200 total client(s). 10000 total requests
Application protocol: http/1.1
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done
finished in 7.24s, 1380.80 req/s, 581.18KB/s
requests: 10000 total, 10000 started, 10000 done, 10000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 10000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 4.11MB (4310000) total, 1.35MB (1420000) headers (space savings 0.00%), 2.35MB (2460000) data
min max mean sd +/- sd
time for request: 84.35ms 267.93ms 140.87ms 17.35ms 80.90%
time for connect: 1.18ms 42.96ms 21.22ms 12.09ms 59.50%
time to 1st byte: 154.70ms 270.69ms 184.00ms 17.04ms 62.50%
req/s : 6.93 7.19 7.04 0.04 76.50%
- a randomized approach could also be implemented using
random=trueinstead ofoffset=x - more unit tests could be added to improve coverage
- remove hardcoded values from scripts and create
K8sa config