All Posts

System Design Protocols: REST, RPC, and TCP/UDP

How do servers talk to each other? This guide explains the key protocols: REST vs RPC for APIs, TCP vs UDP for transport.

Abstract AlgorithmsAbstract Algorithms
ยทยท17 min read
Cover Image for System Design Protocols: REST, RPC, and TCP/UDP
Share
AI Share on X / Twitter
AI Share on LinkedIn
Copy link

TLDR: ๐ŸŽฏ Use REST (HTTP + JSON) for public, browser-facing APIs where interoperability matters. Choose gRPC (HTTP/2 + Protobuf) for internal microservice communication when latency counts. Under the hood, TCP guarantees reliable ordered delivery; UDP trades reliability for speed. Know which layer you're operating at and pick the right tool for each job.


๐Ÿ“– The Protocol Stack: What Talks to What

gRPC โ€” Google's Remote Procedure Call framework โ€” transfers the same payload as REST in 60โ€“80% fewer bytes by using binary Protobuf encoding instead of verbose JSON text. For a service handling one million RPCs per second, that compression gap is roughly the difference between $12,000 and $20,000 per month in bandwidth costs. Protocol choice is a real engineering and business decision, not a stylistic preference.

To see the gap concretely, consider a simple User object:

// REST/JSON โ€” 56 bytes (field names travel with every message)
{"id": 42, "name": "Alice", "email": "alice@example.com"}
// gRPC/Protobuf โ€” ~21 bytes (only field numbers + binary values travel)
// Field names are defined once in the .proto schema, never sent on the wire

At 1M requests/second, that 35-byte-per-message difference adds up to over 2.5 TB of saved bandwidth every day.

Before choosing a protocol, it helps to know where it lives in the network stack. Every call crosses multiple layers, each with its own trade-offs:

Application layer  โ†’  REST, gRPC, GraphQL, WebSocket
Transport layer    โ†’  TCP, UDP, QUIC
Network layer      โ†’  IP

REST and gRPC are application-layer protocols built on top of TCP. They define how data is structured and interpreted. WebRTC for video calls uses UDP at the transport layer for lower latency. Understanding this layering tells you which problems each protocol can and cannot solve.


๐Ÿ” REST vs. gRPC: The API Protocol Showdown

These are the two dominant choices for service-to-service communication.

FeatureREST (HTTP/1.1 or HTTP/2 + JSON)gRPC (HTTP/2 + Protobuf)
Message formatJSON (human-readable text)Protobuf (binary, compact)
SchemaOptional (OpenAPI)Required (.proto contract)
Browser supportNativeNeeds a proxy (grpc-web)
StreamingLimited (SSE, polling)First-class (server, client, bidirectional)
Payload sizeLarger (verbose JSON)3โ€“10x smaller (binary)
Code generationOptionalAutomatic (stubs from .proto)
DebuggingEasy (curl, browser DevTools)Harder (need specialized tools)
Best forPublic APIs, external integrationsInternal microservices, high-throughput RPC

The rule of thumb: if you control both ends of the connection and throughput matters, use gRPC. If third parties or browsers need to call your service, REST is the safer default.


โš™๏ธ How REST Actually Works: Request โ†’ Response

A REST API treats everything as a resource identified by a URL. You act on resources using HTTP verbs.

GET    /users/42           โ†’ fetch user 42
POST   /users              โ†’ create a new user
PUT    /users/42           โ†’ replace user 42
PATCH  /users/42           โ†’ partially update user 42
DELETE /users/42           โ†’ delete user 42

The HTTP response includes a status code (200 OK, 404 Not Found, 500 Internal Server Error) and usually a JSON body.

What makes a REST API "RESTful":

  • Stateless โ€” the server doesn't remember previous requests
  • Uniform interface โ€” consistent URL patterns and verb semantics
  • Cacheable โ€” GET responses can be cached by intermediaries

A well-designed REST API is self-documenting: the URL tells you the resource, the verb tells you the action.

๐Ÿ“Š REST Request Lifecycle

sequenceDiagram
    participant C as Client
    participant LB as Load Balancer
    participant API as API Server
    participant DB as Database

    C->>LB: GET /users/42 (HTTPS)
    LB->>API: Route to available server
    API->>DB: SELECT * WHERE id=42
    DB-->>API: User row
    API-->>LB: 200 OK {id:42, name:"Alice"}
    LB-->>C: 200 OK {id:42, name:"Alice"}
    Note over C,LB: GET is cacheable (Cache-Control)

    C->>LB: POST /users {name:"Bob"}
    LB->>API: Route to server
    API->>DB: INSERT INTO users ...
    DB-->>API: New id=43
    API-->>LB: 201 Created Location:/users/43
    LB-->>C: 201 Created Location:/users/43

This diagram shows two complete REST request lifecycles โ€” a GET for an existing resource and a POST to create a new one โ€” passing through a load balancer, API server, and database. Notice the 200 response for the GET versus the 201 Created response with a Location header for the POST. The key HTTP semantics to take away: GET is idempotent and cacheable, POST is not idempotent and always creates a new resource; these distinctions determine how clients, load balancers, and CDNs are allowed to cache and retry requests automatically.


๐Ÿง  Deep Dive: How Protobuf Binary Encoding Works

Protobuf serializes each field as a compact (field_number, wire_type) tag followed by the value. Field names are never sent over the wire โ€” only numeric tags defined in the .proto file. A string "hello" in JSON is 7 bytes; in Protobuf it is 5 bytes plus a 1-byte tag. For messages sent millions of times per day, this 3โ€“10ร— size reduction translates directly into bandwidth savings and lower serialization latency.

FormatSends field names?Self-describing?Relative payload size
JSONYes (as strings)YesBaseline
ProtobufNo (field numbers only)No (.proto required)3โ€“10ร— smaller

โšก How gRPC Works: Contracts and Binary Efficiency

gRPC starts with a Protobuf schema (.proto file) that defines services and message types:

service UserService {
  rpc GetUser (GetUserRequest) returns (User);
  rpc ListUsers (ListUsersRequest) returns (stream User);
}

message User {
  int32 id = 1;
  string name = 2;
  string email = 3;
}

From this schema, gRPC auto-generates client and server stubs in your language of choice. The client calls userService.GetUser(id) like a local function โ€” the network call is transparent.

Why Protobuf is faster than JSON:

  • Binary encoding: 42 as a 4-byte int, not the 2-byte ASCII string "42"
  • No key repetition per record (field numbers are 1-byte tags)
  • Schema-aware: no need to re-parse field names on every deserialization

gRPC streaming types:

TypeDirectionUse case
UnaryClient sends one, server replies oneStandard request/response
Server streamingClient sends one, server replies manyReal-time updates, log streaming
Client streamingClient sends many, server replies oneFile upload, batch ingestion
Bidirectional streamingBoth stream simultaneouslyChat, collaborative editing

๐Ÿ”Œ TCP vs. UDP: The Transport Layer Choice

REST and gRPC both run over TCP. But some workloads need to go lower and choose the transport layer directly.

TCP (Transmission Control Protocol):

  • Establishes a connection (3-way handshake before data flows)
  • Guarantees every packet arrives and arrives in order
  • Automatically retransmits lost packets
  • Flow control prevents fast senders from overwhelming slow receivers
  • Higher overhead, but correctness is automatic

UDP (User Datagram Protocol):

  • No connection setup โ€” just fire packets and hope
  • No delivery guarantee, no ordering guarantee
  • No retransmission โ€” dropped packets are gone
  • Minimal overhead: ideal when speed matters more than completeness
Use caseTCPUDP
Web pages, APIsโœ…โŒ
File transferโœ…โŒ
Video streamingSometimesโœ… (RTP over UDP)
Online gaming (position updates)โŒ too slowโœ…
DNS lookupSometimes TCPโœ… (small, fast)
VoIPโŒ retransmit delay is worse than lossโœ…

Why live video uses UDP: if a video frame is late, retransmitting it is pointless โ€” by the time it arrives, it's already behind. A dropped frame is better than a frozen screen from TCP's retransmit-and-wait behavior.

๐Ÿ“Š TCP 3-Way Handshake

sequenceDiagram
    participant C as Client
    participant S as Server

    Note over C,S: Connection establishment
    C->>S: SYN (seq=100)
    S-->>C: SYN-ACK (seq=200, ack=101)
    C->>S: ACK (ack=201)
    Note over C,S: Connection established

    C->>S: GET /api/data (payload)
    S-->>C: 200 OK + response body

    Note over C,S: Connection teardown
    C->>S: FIN (seq=300)
    S-->>C: ACK (ack=301)
    S->>C: FIN (seq=400)
    C-->>S: ACK (ack=401)

This diagram walks through the full TCP session lifecycle: a three-message SYN/SYN-ACK/ACK handshake establishes the connection, data flows in both directions, and a four-message FIN/ACK/FIN/ACK exchange tears it down. For REST and gRPC calls, this handshake cost is amortised when HTTP keep-alive reuses the connection across multiple requests โ€” but for short-lived connections it adds one full round-trip before any application data is sent. This overhead is one of the key motivations for HTTP/2 multiplexing and QUIC's zero-RTT connection establishment.


๐Ÿ” WebSockets: Persistent Bidirectional TCP

REST is request-response: the client asks, the server answers. But what if the server needs to push data to the client without being asked?

WebSockets upgrade an HTTP connection to a persistent, full-duplex channel. Once the handshake completes, both sides can send messages independently at any time.

sequenceDiagram
    Client->>Server: HTTP Upgrade: websocket
    Server-->>Client: 101 Switching Protocols
    Server-->>Client: {"event": "price_update", "value": 142.50}
    Client->>Server: {"action": "subscribe", "symbol": "AAPL"}
    Server-->>Client: {"event": "price_update", "value": 143.10}

Use cases: live stock tickers, chat applications, collaborative editing, real-time dashboards, multiplayer games.


๐Ÿ“Š Protocol Decision Flow

Use this flowchart to select the right protocol for each service boundary in your system.

flowchart TD
    A[New Service Interface] --> B{External or public-facing?}
    B -->|Yes| C[REST + JSON over HTTPS]
    B -->|No| D{Need real-time streaming?}
    D -->|Server push to browser| E[WebSocket]
    D -->|Internal bidirectional| F[gRPC bidirectional streaming]
    D -->|No streaming needed| G{Latency and throughput critical?}
    G -->|Yes| H[gRPC + Protobuf over HTTP/2]
    G -->|No| C

Choose the outermost layer first: if external clients or browsers need access, REST is the safest default. For internal high-throughput paths, gRPC's binary encoding reduces payload size by 3โ€“10ร— compared to JSON.


๐Ÿงช Practical Protocol Selection Checklist

This checklist translates the REST-vs-gRPC-vs-UDP decision framework from the protocol sections into a concrete set of diagnostic questions you answer for each service boundary in your system. It was structured as a per-protocol checklist because the most common mistake is picking a single protocol for an entire application โ€” in practice, a well-designed system uses different protocols at different layers. As you work through each section, answer honestly: if your answers for the gRPC block are all "yes," that boundary is a genuine gRPC candidate; if even one answer is "no," the tooling overhead may not be justified by the throughput gain.

For REST:

  • Is the API public or consumed by third-party developers? If yes, REST wins on discoverability.
  • Do you need HTTP caching through CDN or proxies? REST GET responses are cacheable by default.
  • Will browser clients call this API directly? REST is natively supported without a proxy.

For gRPC:

  • Do you control both the client and server? gRPC requires shared .proto contracts.
  • Is payload size or serialization latency a bottleneck? Protobuf is 3โ€“10ร— more compact than JSON.
  • Do you need streaming (server push, client upload, or bidirectional chat)? gRPC has first-class support.

For UDP/QUIC:

  • Does a dropped packet expire before a retransmit would arrive? Live video and gaming frames do.
  • Do you need minimal connection setup overhead? QUIC establishes connections in one round-trip.

Apply the checklist per service pair, not per application. A single system often uses REST for its public API, gRPC internally, and WebSockets for a real-time dashboard.


๐ŸŒ Real-World Applications: Real-World Protocol Choices at Scale

Twitter/X internal RPC: Uses a combination of HTTP APIs for external clients and internal Thrift (a REST-like binary protocol) for microservice-to-microservice calls.

Netflix internal APIs: Originally REST; migrated critical paths to gRPC for the efficiency gains at hundreds-of-millions-of-requests/day scale.

YouTube video streaming: Uses UDP-based QUIC (a newer transport protocol from Google) which gives them TCP-like reliability guarantees with UDP-like connection setup speed. QUIC eliminates TCP's head-of-line blocking problem for multiplexed streams.


โš–๏ธ Trade-offs & Failure Modes: Protocol Trade-offs

  • REST verbosity vs. simplicity: JSON is easy to debug with curl or browser DevTools, but verbose at scale โ€” large payloads raise bandwidth costs and serialization latency on hot paths.
  • gRPC tooling overhead: .proto schema management and grpc-web proxies for browsers add operational burden. Debugging binary traffic requires specialized reflection tools rather than a quick browser inspection.
  • UDP packet loss is silent: Dropped UDP packets are gone with no error. Applications that silently skip lost frames must handle partial state carefully to avoid subtle data integrity bugs.
  • WebSocket connection storms: WebSockets hold persistent TCP connections. A rolling deploy that restarts servers simultaneously drops all connections, and clients reconnecting concurrently create a thundering-herd spike on the backend.

๐Ÿงญ Decision Guide: Protocol Selection by Use Case

ScenarioProtocol choiceReasoning
Public API for third-party developersREST + JSONUniversally accessible, easy to document
Internal microservices, high-throughputgRPC + ProtobufBinary efficiency, strong typing, streaming
Real-time push from server to browserWebSocketPersistent bidirectional channel
Live video, gaming, VoIPUDP-based (RTP, QUIC)Latency beats reliability for these workloads
IoT devices on lossy networksMQTT over TCPLightweight pub/sub, designed for constrained devices
Cross-service event streamingKafka (TCP-based)Persistent, replay-able, high-fanout

๐Ÿ› ๏ธ Spring Boot REST: HTTP Semantics That Reveal the REST Constraints

Spring MVC's ResponseEntity exposes every HTTP semantic discussed in this post โ€” status codes, Location headers, ETag fingerprints, and Cache-Control directives โ€” so you can see REST's constraints directly in code. The snippet below focuses on why each choice matters rather than wiring up a database.

// dependencies: spring-boot-starter-web, spring-boot-starter-validation

import org.springframework.http.*;
import org.springframework.web.bind.annotation.*;
import jakarta.validation.Valid;
import java.net.URI;
import java.util.concurrent.TimeUnit;

record UserDto(Long id, String name, String email, long version) {}
record CreateUserRequest(@jakarta.validation.constraints.NotBlank String name,
                         @jakarta.validation.constraints.NotBlank String email) {}

@RestController
@RequestMapping("/users")
public class RestSemanticsDemo {

    // โ”€โ”€ GET: SAFE + IDEMPOTENT โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
    // Safe    โ†’ no side effects; calling 1000ร— changes nothing on the server.
    // Idempotent โ†’ same result every time; proxies may retry automatically.
    //
    // ETag = fingerprint of the resource version (e.g. its DB row version).
    // If-None-Match enables a CONDITIONAL GET: "only send the body if changed."
    @GetMapping("/{id}")
    public ResponseEntity<UserDto> getUser(
            @PathVariable Long id,
            @RequestHeader(value = "If-None-Match", required = false) String clientEtag) {

        UserDto user = userRepository.findById(id).orElse(null);
        if (user == null) return ResponseEntity.notFound().build();       // 404: resource absent

        String serverEtag = "\"" + user.version() + "\"";
        if (serverEtag.equals(clientEtag)) {
            // 304 Not Modified โ€” body is omitted entirely; browser/CDN serves its cached copy.
            // This is REST's stateless caching constraint in action: no session needed.
            return ResponseEntity.status(HttpStatus.NOT_MODIFIED).build();
        }

        return ResponseEntity.ok()
                .eTag(serverEtag)                                          // client stores for next request
                .cacheControl(CacheControl.maxAge(30, TimeUnit.SECONDS))   // hint for proxies/CDNs
                .body(user);                                               // 200 OK with full body
    }

    // โ”€โ”€ POST: NOT IDEMPOTENT โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
    // Each call creates a NEW resource with a NEW identity.
    // 201 Created + Location header: tells the client exactly where the resource now lives.
    // Proxies must NOT auto-retry POST on failure (unlike GET/PUT).
    @PostMapping
    public ResponseEntity<UserDto> createUser(@Valid @RequestBody CreateUserRequest req) {
        UserDto created = userService.create(req);
        // Location: /users/42 โ€” client follows this URL to confirm or retrieve the new resource
        return ResponseEntity.created(URI.create("/users/" + created.id())).body(created);  // 201
    }

    // โ”€โ”€ PUT: IDEMPOTENT โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
    // Replaces the ENTIRE resource at a known URL.
    // Idempotent: PUT /users/42 twice leaves the server in exactly the same state both times.
    // Contrast: POST /users twice โ†’ two different users created with two different IDs.
    @PutMapping("/{id}")
    public ResponseEntity<UserDto> replaceUser(@PathVariable Long id,
                                               @Valid @RequestBody UserDto dto) {
        if (!userRepository.existsById(id)) return ResponseEntity.notFound().build(); // 404
        return ResponseEntity.ok(userService.replace(id, dto));  // 200: resource fully replaced
    }
}

Three REST constraints visible in this snippet:

HTTP propertyDemonstrated byWhy it matters
Safe + idempotent (GET)If-None-Match / 304 conditional responseEnables transparent caching at every layer without coordination
Not idempotent (POST)201 + Location header on creationClients and proxies know they must not auto-retry; the response reveals where the resource is
Idempotent (PUT)Full replacement returning 200Networks can safely retry after timeouts โ€” re-sending the same PUT is harmless

For a full deep-dive on Spring Boot REST, content negotiation, and HATEOAS, a dedicated follow-up post is planned.


๐Ÿ› ๏ธ gRPC Java: Binary-Efficient Internal Service Communication

The gRPC Java library (from Google) implements the Protobuf + HTTP/2 protocol described in this post as a first-class Java API โ€” generating type-safe client stubs and server skeletons from .proto files, eliminating the manual JSON parsing and URL routing required by REST.

// user_service.proto โ€” schema-first contract (defines the API before any code)
syntax = "proto3";
option java_package = "com.example.grpc";

service UserService {
  rpc GetUser    (GetUserRequest)  returns (User);
  rpc ListUsers  (ListUsersRequest) returns (stream User);  // server streaming
}

message User          { int64 id = 1; string name = 2; string email = 3; }
message GetUserRequest  { int64 id = 1; }
message ListUsersRequest { int32 limit = 1; }
// Server implementation โ€” generated stub from protoc + grpc-java plugin
// dependencies: grpc-stub, grpc-protobuf, grpc-netty-shaded, protobuf-java

import io.grpc.stub.StreamObserver;
import com.example.grpc.*;

public class UserServiceImpl extends UserServiceGrpc.UserServiceImplBase {

    @Override
    public void getUser(GetUserRequest req, StreamObserver<User> resp) {
        // Binary Protobuf response โ€” ~3ร— smaller than equivalent JSON
        User user = User.newBuilder()
            .setId(req.getId())
            .setName("Alice")
            .setEmail("alice@example.com")
            .build();
        resp.onNext(user);
        resp.onCompleted();
    }

    @Override
    public void listUsers(ListUsersRequest req, StreamObserver<User> resp) {
        // Server streaming โ€” pushes multiple User messages over one HTTP/2 connection
        for (int i = 1; i <= req.getLimit(); i++) {
            resp.onNext(User.newBuilder().setId(i).setName("User" + i).build());
        }
        resp.onCompleted();
    }
}

// Client call โ€” type-safe, no URL strings, no JSON parsing
// UserServiceGrpc.UserServiceBlockingStub stub = UserServiceGrpc.newBlockingStub(channel);
// User alice = stub.getUser(GetUserRequest.newBuilder().setId(1).build());

The .proto file is the contract that both sides compile against โ€” a breaking change to the schema (removing a field, changing a type) causes a compile error in both client and server code, enforcing the versioning discipline described in the Lessons section.

For a full deep-dive on gRPC Java, a dedicated follow-up post is planned.


๐Ÿ“š Lessons from Protocol Choices in Production

Engineering teams that have operated distributed systems at scale share consistent protocol lessons.

Avoid premature gRPC adoption: gRPC's tooling overhead (.proto management, grpc-web proxies for browsers, reflection for debugging) is significant. Start with REST and migrate the hot paths to gRPC once you have measured the latency impact.

Design for protocol versioning from day one: REST APIs should include a version prefix (/v1/). gRPC services should reserve field numbers in .proto files. Retroactive versioning is painful.

UDP is not "faster REST": UDP eliminates TCP's reliability guarantees entirely. Only use it when stale data is genuinely preferable to delayed data โ€” live game positions, sensor telemetry, or video frames.

Instrument the transport layer: track p99 request latency, payload size, and connection error rates per protocol. Teams often discover that HTTP/1.1 keep-alive was disabled, eliminating the assumed benefit of persistent connections.


๐Ÿ“Œ TLDR: Summary & Key Takeaways

  • REST = stateless, resource-oriented, universally compatible โ€” use it for public APIs and external integrations.
  • gRPC = binary, contract-driven, streaming-capable โ€” use it for internal high-throughput RPC.
  • TCP = reliable, ordered, connection-based โ€” the default for anything where correctness matters.
  • UDP = unreliable, low-overhead, connectionless โ€” use it when a dropped packet is better than a delayed one.
  • WebSockets give you a persistent full-duplex channel on top of TCP for real-time bidirectional communication.
  • Protocol choice is a trade-off between developer ergonomics, latency, bandwidth, and ecosystem compatibility.

๐Ÿ“ Practice Quiz

  1. Q1: You're building an internal microservice that calls another service 50,000 times per second with strict schema requirements. Which protocol is the best fit?
    • A) REST + JSON over HTTP/1.1
    • B) WebSocket
    • C) gRPC + Protobuf over HTTP/2

Correct Answer: C

  1. Q2: A real-time multiplayer game needs to send player position updates 60 times per second. Why is UDP preferred over TCP here?
    • A) UDP provides built-in encryption
    • B) If a position update is delayed by TCP retransmit, the stale data is worse than no data โ€” dropped packets are just skipped
    • C) UDP guarantees in-order delivery, which is critical for game state

Correct Answer: B

  1. Q3: Which feature of gRPC makes it natively better than REST for streaming use cases?
    • A) It automatically converts JSON to binary
    • B) Its first-class support for server streaming, client streaming, and bidirectional streaming over HTTP/2
    • C) It eliminates the need for a load balancer

Correct Answer: B



Abstract Algorithms

Written by

Abstract Algorithms

@abstractalgorithms