On Scaling Microservices

THOUGHTS ON SCALING MICROSERVICES

Much of this is a rehash of Susan Fowler's excellent book
 Production-Ready Microservices Published by O'Reilly Media, Inc., 2016 http://shop.oreilly.com/product/0636920053675.do

# Qualitative and Quantitative growth scales of a Microservice

## Quantitative

Qualitative growth scales allow the scalability of a service to tie in with higher-level business metrics: a microservice may, for example, scale with the number of users, with the number of people who open a phone application (“eyeballs”), or with the number of orders (for a food delivery service). These metrics, these qualitative growth scales, aren’t tied to an individual microservice but to the overall system or product(s).
Quantitative

•     Business Metrics 
•     Number of Health Care Claims Adjudicated
•     Number of Insurance claims processed

## Qualitative

If the qualitative growth scale of our microservice is measured in “eyeballs”, and each “eyeball” results in two requests to our microservice and one database transaction, then our quantitative growth scale is measured in terms of requests and transactions, resulting in requests per second and transactions per second as the two key quantities determining our scalability. 

•     RequestsPerSecond/QueriesPerSecond/TransactionsPerSecond
•     HTTP Throughput
•     CPU Utilization
•     Memory
•     Latency
•     (negative - scaling) Threadpool saturation
•     (negative - scaling) Number of open database connections _is it near the conn limit_

When dealing with complex dependency chains, making sure that all microservice teams tie the scalability of their services to high-level business metrics can make sure that all services are properly prepared for expected growth, even when cross-team communication becomes difficult. 

## What To Monitor For Each Microservice Availability

### Infrastructure Metrics

• CPU utilized by the microservice across all containers
• RAM utilized by the microservice across all containers,
• The available threads,
• The microservice’s open file descriptors (FD)
• The number of database connections that the microservice has to any databases it uses.

### Monitor the availability of the service

•     Service-level agreement (SLA) of the service,
•     Latency (of both the service as a whole and its API endpoints), -
•     Success of API endpoints, Responses
•    Average response times of API endpoints, the services (clients) from which API requests originate (along with which endpoints they send requests to),
•    Errors and exceptions (both handled and unhandled), and the health and status of dependencies.

# Monitoring ADVICE

1. CUSTOM DASHBOARD FOR EACH MICROSERVICE that ALERT FOR EACH MICROSERVICE FOR KEY METRICS   
2. Normal, Warning and Critical Alerts
3. On call Runbook procedure for remediating all alerts
4. Low level Remediations should be automated

A microservice should never experience the same exact problem twice.