awesome-copilot/skills/web-coder/references/http-networking.md

HTTP & Networking Reference

Comprehensive reference for HTTP protocol, networking concepts, and web communication.

HTTP (HyperText Transfer Protocol)

Protocol for transferring hypertext between client and server. Foundation of data communication on the web.

HTTP Versions

• HTTP/1.1 (1997): Text-based, persistent connections, pipelining
• HTTP/2 (2015): Binary protocol, multiplexing, server push, header compression
• HTTP/3 (2022): Based on QUIC (UDP), improved performance, better handling of packet loss

Request Methods

Method	Purpose	Idempotent	Safe	Cacheable
GET	Retrieve resource	Yes	Yes	Yes
POST	Create resource	No	No	Rarely
PUT	Update/replace resource	Yes	No	No
PATCH	Partial update	No	No	No
DELETE	Delete resource	Yes	No	No
HEAD	Like GET but no body	Yes	Yes	Yes
OPTIONS	Get allowed methods	Yes	Yes	No
CONNECT	Establish tunnel	No	No	No
TRACE	Echo request	Yes	Yes	No

Safe: Doesn't modify server state
Idempotent: Multiple identical requests have same effect as single request

Status Codes

1xx Informational

Code	Message	Meaning
100	Continue	Client should continue request
101	Switching Protocols	Server switching protocols

2xx Success

Code	Message	Meaning
200	OK	Request succeeded
201	Created	Resource created
202	Accepted	Accepted but not processed
204	No Content	Success but no content to return
206	Partial Content	Partial resource (range request)

3xx Redirection

Code	Message	Meaning
301	Moved Permanently	Resource permanently moved
302	Found	Temporary redirect
303	See Other	Response at different URI
304	Not Modified	Resource not modified (cache)
307	Temporary Redirect	Like 302 but keep method
308	Permanent Redirect	Like 301 but keep method

4xx Client Errors

Code	Message	Meaning
400	Bad Request	Invalid syntax
401	Unauthorized	Authentication required
403	Forbidden	Access denied
404	Not Found	Resource not found
405	Method Not Allowed	Method not supported
408	Request Timeout	Request took too long
409	Conflict	Request conflicts with state
410	Gone	Resource permanently gone
413	Payload Too Large	Request body too large
414	URI Too Long	URI too long
415	Unsupported Media Type	Media type not supported
422	Unprocessable Entity	Semantic errors
429	Too Many Requests	Rate limit exceeded

5xx Server Errors

Code	Message	Meaning
500	Internal Server Error	Generic server error
501	Not Implemented	Method not supported
502	Bad Gateway	Invalid response from upstream
503	Service Unavailable	Server temporarily unavailable
504	Gateway Timeout	Upstream timeout
505	HTTP Version Not Supported	HTTP version not supported

HTTP Headers

Request Headers

GET /api/users HTTP/1.1
Host: example.com
User-Agent: Mozilla/5.0
Accept: application/json, text/plain
Accept-Language: en-US,en;q=0.9
Accept-Encoding: gzip, deflate, br
Authorization: Bearer token123
Cookie: sessionId=abc123
If-None-Match: "etag-value"
If-Modified-Since: Wed, 21 Oct 2015 07:28:00 GMT
Origin: https://example.com
Referer: https://example.com/page

Common Request Headers:

• Accept: Media types client accepts
• Accept-Encoding: Encoding formats (compression)
• Accept-Language: Preferred languages
• Authorization: Authentication credentials
• Cache-Control: Caching directives
• Cookie: Cookies sent to server
• Content-Type: Type of request body
• Host: Target host and port
• If-Modified-Since: Conditional request
• If-None-Match: Conditional request (ETag)
• Origin: Origin of request (CORS)
• Referer: Previous page URL
• User-Agent: Client information

Response Headers

HTTP/1.1 200 OK
Date: Mon, 04 Mar 2026 12:00:00 GMT
Server: nginx/1.18.0
Content-Type: application/json; charset=utf-8
Content-Length: 348
Content-Encoding: gzip
Cache-Control: public, max-age=3600
ETag: "33a64df551425fcc55e4d42a148795d9f25f89d4"
Last-Modified: Mon, 04 Mar 2026 11:00:00 GMT
Access-Control-Allow-Origin: *
Set-Cookie: sessionId=xyz789; HttpOnly; Secure; SameSite=Strict
Strict-Transport-Security: max-age=31536000; includeSubDomains
X-Content-Type-Options: nosniff
X-Frame-Options: DENY

Common Response Headers:

• Access-Control-*: CORS headers
• Cache-Control: Caching directives
• Content-Encoding: Content compression
• Content-Length: Body size in bytes
• Content-Type: Media type of body
• Date: Response date/time
• ETag: Resource version identifier
• Expires: Expiration date
• Last-Modified: Last modification date
• Location: Redirect URL
• Server: Server software
• Set-Cookie: Set cookies
• Strict-Transport-Security: HSTS
• X-Content-Type-Options: MIME type sniffing
• X-Frame-Options: Clickjacking protection

CORS (Cross-Origin Resource Sharing)

Mechanism to allow cross-origin requests.

Simple Requests

Automatically allowed if:

• Method: GET, HEAD, or POST
• Safe headers only
• Content-Type: application/x-www-form-urlencoded, multipart/form-data, or text/plain

Preflight Requests

For complex requests, browser sends OPTIONS request first:

OPTIONS /api/users HTTP/1.1
Origin: https://example.com
Access-Control-Request-Method: POST
Access-Control-Request-Headers: Content-Type

HTTP/1.1 204 No Content
Access-Control-Allow-Origin: https://example.com
Access-Control-Allow-Methods: GET, POST, PUT, DELETE
Access-Control-Allow-Headers: Content-Type, Authorization
Access-Control-Allow-Credentials: true
Access-Control-Max-Age: 86400

CORS Headers

Request:

• Origin: Request origin
• Access-Control-Request-Method: Intended method
• Access-Control-Request-Headers: Intended headers

Response:

• Access-Control-Allow-Origin: Allowed origins (* or specific)
• Access-Control-Allow-Methods: Allowed methods
• Access-Control-Allow-Headers: Allowed headers
• Access-Control-Allow-Credentials: Allow credentials
• Access-Control-Max-Age: Preflight cache duration
• Access-Control-Expose-Headers: Headers accessible to client

Caching

Cache-Control Directives

Request Directives:

• no-cache: Validate with server before using cache
• no-store: Don't cache at all
• max-age=N: Max age in seconds
• max-stale=N: Accept stale response up to N seconds
• min-fresh=N: Fresh for at least N seconds
• only-if-cached: Use only cached response

Response Directives:

• public: Cacheable by any cache
• private: Cacheable by browser only
• no-cache: Must validate before use
• no-store: Don't cache
• max-age=N: Fresh for N seconds
• s-maxage=N: Max age for shared caches
• must-revalidate: Must validate when stale
• immutable: Content won't change

Examples

# Cache for 1 hour
Cache-Control: public, max-age=3600

# Don't cache
Cache-Control: no-store

# Cache in browser only, revalidate after 1 hour
Cache-Control: private, max-age=3600, must-revalidate

# Cache forever (with versioned URLs)
Cache-Control: public, max-age=31536000, immutable

Conditional Requests

Use ETags or Last-Modified for efficient caching:

GET /resource HTTP/1.1
If-None-Match: "etag-value"
If-Modified-Since: Wed, 21 Oct 2015 07:28:00 GMT

If not modified:

HTTP/1.1 304 Not Modified
ETag: "etag-value"

Cookies

# Server sets cookie
Set-Cookie: sessionId=abc123; Path=/; HttpOnly; Secure; SameSite=Strict; Max-Age=3600

# Client sends cookie
Cookie: sessionId=abc123; userId=456

Cookie Attributes

• Path=/: Cookie path scope
• Domain=example.com: Cookie domain scope
• Max-Age=N: Expire after N seconds
• Expires=date: Expire at specific date
• Secure: Only sent over HTTPS
• HttpOnly: Not accessible via JavaScript
• SameSite=Strict|Lax|None: CSRF protection

REST (Representational State Transfer)

Architectural style for web services.

REST Principles

1. Client-Server: Separation of concerns
2. Stateless: Each request contains all needed info
3. Cacheable: Responses must define cacheability
4. Uniform Interface: Standardized communication
5. Layered System: Client doesn't know if connected to end server
6. Code on Demand (optional): Server can send executable code

RESTful API Design

GET    /users           # List users
GET    /users/123       # Get user 123
POST   /users           # Create user
PUT    /users/123       # Update user 123 (full)
PATCH  /users/123       # Update user 123 (partial)
DELETE /users/123       # Delete user 123

GET    /users/123/posts # List posts by user 123
GET    /posts?author=123 # Alternative: filter posts

HTTP Content Negotiation

# Client requests JSON
Accept: application/json

# Server responds with JSON
Content-Type: application/json

# Client can accept multiple formats
Accept: application/json, application/xml;q=0.9, text/plain;q=0.8

Networking Fundamentals

TCP (Transmission Control Protocol)

Connection-oriented protocol ensuring reliable data delivery.

TCP Handshake (3-way):

1. Client → Server: SYN
2. Server → Client: SYN-ACK
3. Client → Server: ACK

Features:

• Reliable delivery (retransmission)
• Ordered data
• Error checking
• Flow control
• Connection-oriented

UDP (User Datagram Protocol)

Connectionless protocol for fast data transmission.

Features:

• Fast (no handshake)
• No guaranteed delivery
• No ordering
• Lower overhead
• Used for streaming, gaming, DNS

DNS (Domain Name System)

Translates domain names to IP addresses.

example.com → 93.184.216.34

DNS Record Types:

• A: IPv4 address
• AAAA: IPv6 address
• CNAME: Canonical name (alias)
• MX: Mail exchange
• TXT: Text record
• NS: Name server

IP Addressing

IPv4: 192.168.1.1 (32-bit)
IPv6: 2001:0db8:85a3:0000:0000:8a2e:0370:7334 (128-bit)

Ports

• Well-known ports (0-1023):
  • 80: HTTP
  • 443: HTTPS
  • 21: FTP
  • 22: SSH
  • 25: SMTP
  • 53: DNS
• Registered ports (1024-49151)
• Dynamic ports (49152-65535)

Bandwidth & Latency

Bandwidth: Amount of data transferred per unit time (Mbps, Gbps)
Latency: Time delay in data transmission (milliseconds)

Round Trip Time (RTT): Time for request to reach server and response to return

WebSockets

Full-duplex communication over single TCP connection.

// Client
const ws = new WebSocket('wss://example.com/socket');

ws.onopen = () => {
  console.log('Connected');
  ws.send('Hello server!');
};

ws.onmessage = (event) => {
  console.log('Received:', event.data);
};

ws.onerror = (error) => {
  console.error('Error:', error);
};

ws.onclose = () => {
  console.log('Disconnected');
};

// Close connection
ws.close();

Use Cases: Chat, real-time updates, gaming, collaborative editing

Server-Sent Events (SSE)

Server pushes updates to client over HTTP.

// Client
const eventSource = new EventSource('/events');

eventSource.onmessage = (event) => {
  console.log('New message:', event.data);
};

eventSource.addEventListener('custom-event', (event) => {
  console.log('Custom event:', event.data);
});

eventSource.onerror = (error) => {
  console.error('Error:', error);
};

// Close connection
eventSource.close();

// Server response
Content-Type: text/event-stream
Cache-Control: no-cache
Connection: keep-alive

data: First message

data: Second message

event: custom-event
data: Custom message data

Best Practices

Do's

• ✅ Use HTTPS everywhere
• ✅ Implement proper caching strategies
• ✅ Use appropriate HTTP methods
• ✅ Return meaningful status codes
• ✅ Implement rate limiting
• ✅ Use compression (gzip, brotli)
• ✅ Set proper CORS headers
• ✅ Implement proper error handling
• ✅ Use connection pooling
• ✅ Monitor network performance

Don'ts

• ❌ Use HTTP for sensitive data
• ❌ Ignore CORS security
• ❌ Return wrong status codes (200 for errors)
• ❌ Cache sensitive data
• ❌ Send large uncompressed responses
• ❌ Skip SSL/TLS certificate validation
• ❌ Store credentials in URLs
• ❌ Expose internal server details in errors
• ❌ Use synchronous requests

Glossary Terms

Key Terms Covered:

• Ajax
• ALPN
• Bandwidth
• Cacheable
• Cookie
• CORS
• CORS-safelisted request header
• CORS-safelisted response header
• Crawler
• Effective connection type
• Fetch directive
• Fetch metadata request header
• Forbidden request header
• Forbidden response header name
• FTP
• General header
• HOL blocking
• HTTP
• HTTP content
• HTTP header
• HTTP/2
• HTTP/3
• HTTPS
• HTTPS RR
• Idempotent
• IMAP
• Latency
• Packet
• POP3
• Proxy server
• QUIC
• Rate limit
• Request header
• Response header
• REST
• Round Trip Time (RTT)
• RTCP
• RTP
• Safe (HTTP Methods)
• SMTP
• TCP
• TCP handshake
• TCP slow start
• UDP
• WebSockets

Additional Resources

• MDN HTTP Guide
• HTTP/2 Spec
• HTTP/3 Explained
• REST API Tutorial