Skip to main content
Version: v1.9

Benchmarks

In this document you will find benchmark results for different endpoints of ORY Hydra. All benchmarks are executed using rakyll/hey. Please note that these benchmarks run against the in-memory storage adapter of ORY Hydra. These benchmarks represent what performance you would get with a zero-overhead database implementation.

We do not include benchmarks against databases (e.g. MySQL, PostgreSQL or CockroachDB) as the performance greatly differs between deployments (e.g. request latency, database configuration) and tweaking individual things may greatly improve performance. We believe, for that reason, that benchmark results for these database adapters are difficult to generalize and potentially deceiving. They are thus not included.

This file is updated on every push to master. It thus represents the benchmark data for the latest version.

All benchmarks run 10.000 requests in total, with 100 concurrent requests. All benchmarks run on Circle-CI with a "2 CPU cores and 4GB RAM" configuration.

BCrypt#

ORY Hydra uses BCrypt to obfuscate secrets of OAuth 2.0 Clients. When using flows such as the OAuth 2.0 Client Credentials Grant, ORY Hydra validates the client credentials using BCrypt which causes (by design) CPU load. CPU load and performance depend on the BCrypt cost which can be set using the environment variable BCRYPT_COST. For these benchmarks, we have set BCRYPT_COST=8.

OAuth 2.0#

This section contains various benchmarks against OAuth 2.0 endpoints

Token Introspection#


Summary:  Total:    1.9573 secs  Slowest:  0.1461 secs  Fastest:  0.0002 secs  Average:  0.0186 secs  Requests/sec: 5108.9654
  Total data:   1550000 bytes  Size/request: 155 bytes
Response time histogram:  0.000 [1] |  0.015 [5258]  |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■  0.029 [2339]  |■■■■■■■■■■■■■■■■■■  0.044 [1325]  |■■■■■■■■■■  0.059 [651]   |■■■■■  0.073 [261]   |■■  0.088 [117]   |■  0.102 [33]    |  0.117 [11]    |  0.131 [2] |  0.146 [2] |

Latency distribution:  10% in 0.0006 secs  25% in 0.0015 secs  50% in 0.0133 secs  75% in 0.0284 secs  90% in 0.0454 secs  95% in 0.0563 secs  99% in 0.0799 secs
Details (average, fastest, slowest):  DNS+dialup:   0.0000 secs, 0.0002 secs, 0.1461 secs  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0049 secs  req write:    0.0000 secs, 0.0000 secs, 0.0052 secs  resp wait:    0.0185 secs, 0.0002 secs, 0.1460 secs  resp read:    0.0000 secs, 0.0000 secs, 0.0048 secs
Status code distribution:  [200] 10000 responses


Client Credentials Grant#

This endpoint uses BCrypt.


Summary:  Total:    19.4831 secs  Slowest:  0.9063 secs  Fastest:  0.0171 secs  Average:  0.1861 secs  Requests/sec: 513.2648
  Total data:   1570000 bytes  Size/request: 157 bytes
Response time histogram:  0.017 [1] |  0.106 [3566]  |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■  0.195 [2358]  |■■■■■■■■■■■■■■■■■■■■■■■■■■  0.284 [1850]  |■■■■■■■■■■■■■■■■■■■■■  0.373 [1146]  |■■■■■■■■■■■■■  0.462 [650]   |■■■■■■■  0.551 [288]   |■■■  0.640 [101]   |■  0.728 [27]    |  0.817 [11]    |  0.906 [2] |

Latency distribution:  10% in 0.0318 secs  25% in 0.0922 secs  50% in 0.1736 secs  75% in 0.2728 secs  90% in 0.3815 secs  95% in 0.4191 secs  99% in 0.5863 secs
Details (average, fastest, slowest):  DNS+dialup:   0.0001 secs, 0.0171 secs, 0.9063 secs  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0432 secs  req write:    0.0001 secs, 0.0000 secs, 0.0615 secs  resp wait:    0.1857 secs, 0.0171 secs, 0.9062 secs  resp read:    0.0001 secs, 0.0000 secs, 0.0302 secs
Status code distribution:  [200] 10000 responses


OAuth 2.0 Client Management#

Creating OAuth 2.0 Clients#

This endpoint uses BCrypt and generates IDs and secrets by reading from which negatively impacts performance. Performance will be better if IDs and secrets are set in the request as opposed to generated by ORY Hydra.

This test is currently disabled due to issues with /dev/urandom being inaccessible in the CI.

Listing OAuth 2.0 Clients#


Summary:  Total:    0.6655 secs  Slowest:  0.0337 secs  Fastest:  0.0001 secs  Average:  0.0062 secs  Requests/sec: 15025.1907
  Total data:   4880000 bytes  Size/request: 488 bytes
Response time histogram:  0.000 [1] |  0.003 [4995]  |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■  0.007 [960]   |■■■■■■■■  0.010 [1250]  |■■■■■■■■■■  0.014 [843]   |■■■■■■■  0.017 [988]   |■■■■■■■■  0.020 [619]   |■■■■■  0.024 [243]   |■■  0.027 [80]    |■  0.030 [18]    |  0.034 [3] |

Latency distribution:  10% in 0.0002 secs  25% in 0.0002 secs  50% in 0.0035 secs  75% in 0.0115 secs  90% in 0.0167 secs  95% in 0.0190 secs  99% in 0.0236 secs
Details (average, fastest, slowest):  DNS+dialup:   0.0000 secs, 0.0001 secs, 0.0337 secs  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0064 secs  req write:    0.0000 secs, 0.0000 secs, 0.0057 secs  resp wait:    0.0061 secs, 0.0001 secs, 0.0306 secs  resp read:    0.0001 secs, 0.0000 secs, 0.0038 secs
Status code distribution:  [200] 10000 responses


Fetching a specific OAuth 2.0 Client#


Summary:  Total:    0.5859 secs  Slowest:  0.0398 secs  Fastest:  0.0001 secs  Average:  0.0055 secs  Requests/sec: 17067.5827
  Total data:   4860000 bytes  Size/request: 486 bytes
Response time histogram:  0.000 [1] |  0.004 [5492]  |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■  0.008 [1579]  |■■■■■■■■■■■■  0.012 [1105]  |■■■■■■■■  0.016 [923]   |■■■■■■■  0.020 [595]   |■■■■  0.024 [178]   |■  0.028 [87]    |■  0.032 [29]    |  0.036 [7] |  0.040 [4] |

Latency distribution:  10% in 0.0002 secs  25% in 0.0002 secs  50% in 0.0018 secs  75% in 0.0093 secs  90% in 0.0155 secs  95% in 0.0184 secs  99% in 0.0250 secs
Details (average, fastest, slowest):  DNS+dialup:   0.0000 secs, 0.0001 secs, 0.0398 secs  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0063 secs  req write:    0.0000 secs, 0.0000 secs, 0.0066 secs  resp wait:    0.0053 secs, 0.0001 secs, 0.0398 secs  resp read:    0.0001 secs, 0.0000 secs, 0.0101 secs
Status code distribution:  [200] 10000 responses