~/learn / web-fundamentals
How the Web Works
Everything that happens between typing a URL and seeing a page, layer by layer, what travels on the wire at each step, the infrastructure in between, and why every layer matters to security.
Type https://example.com/login and press Enter. In well under a second your computer finds a server somewhere in the world, agrees on encryption with it, asks for a page, and renders it, pulling in dozens of further resources as it goes. Every one of those steps is observable, and every one can go wrong. Understanding the layers is what lets you reason about where a bug can live instead of guessing.
Anatomy of a URL
https://shop.example.com:443/account/orders?status=open#recent
└─┬─┘ └──────┬───────┘└┬┘└──────┬──────┘└────┬────┘└──┬──┘
scheme host port path query fragment| Part | Notes |
|---|---|
| Scheme | https is HTTP over TLS. http is no encryption at all |
| Host | Resolved to an IP address by DNS |
| Port | Defaults to 443 for HTTPS, 80 for HTTP, so usually omitted |
| Path and query | Sent to the server. Logged by servers, proxies, and analytics |
| Fragment | Never sent to the server. Stays in the browser |
Two rows carry security weight. Anything in the path or query lands in server logs, proxy logs, browser history, and Referer headers, so secrets must never go there. And the fragment never reaches the server, which is exactly why DOM-based XSS through location.hash leaves no server-side trace. The host also has more parsing nuance than it looks: userinfo, alternate IP encodings, and punycode all let a string look like one host and resolve to another, which the URL inspector makes visible.
The journey of a request
1. DNS. Your computer asks a resolver what IP shop.example.com has. The answer may come from a cache or travel up to the domain’s authoritative name servers. See DNS, and the DNS lookup tool for any domain’s records.
2. TCP connection. Your computer opens a connection to that IP on port 443 with a three-way handshake (SYN, SYN-ACK, ACK). This is exactly what a port scanner probes, and the handshake state is why a half-open scan reveals an open port without completing a connection.
3. TLS handshake. Client and server agree on keys and the server proves its identity with a certificate. After this, everything is encrypted. See TLS.
4. HTTP request. The browser sends the method, path, headers, and cookies, now inside the encrypted channel. On HTTP/1.1 this is plain text framed by blank lines; HTTP/2 and HTTP/3 carry the same information as binary frames multiplexed over one connection, which is faster but keeps the same request semantics. The framing difference is not academic: when a front-end proxy and a back-end server disagree about where one request ends and the next begins, you get request smuggling.
5. Response and rendering. The server returns HTML. The browser parses it and fires off more requests for scripts, styles, images, fonts, and API data, often dozens, each repeating steps 1 through 4 (minus DNS and TLS when the connection is reused).
See the whole thing in one command:
curl -v https://example.com 2>&1 | head -40The verbose output shows the DNS result, TCP connect, TLS negotiation, request, and response headers in order. It is the entire journey made visible.
What sits in between
Real requests rarely go straight to the application. Each intermediary is a layer that can help or hurt.
| Layer | Examples | Security relevance |
|---|---|---|
| CDN | Cloudflare, Fastly, Akamai | Caches responses; a cache keyed without the auth context can serve one user’s data to another (cache deception and poisoning) |
| WAF | Built into CDNs or standalone | Blocks known attack patterns, but is a filter, not a fix, and is bypassable |
| Load balancer and reverse proxy | nginx, HAProxy, cloud LBs | Parser disagreements with the app cause request smuggling |
| Application | Rails, Django, Node | Where business logic and authorization live, and where real bugs concentrate |
Response headers often reveal these layers: server, via, cf-ray, x-cache, x-served-by. Reading them with the header analyzer is a fast way to map an unfamiliar target.
The browser’s security rules
Websites depend on rules the browser enforces:
- Same-origin policy stops one site’s scripts from reading another’s data. See same-origin and CORS.
- Cookies are scoped to domains and paths, and
Secure,HttpOnly, andSameSitecontrol when they are sent. See sessions and cookies. - Security headers like CSP and HSTS let a site ask the browser for extra protection. Check them with the CSP analyzer and header analyzer.
See it yourself
Open developer tools, go to the Network tab, and load any site. Click a request to see its headers, timing, and response. Then watch the same thing from the command line with the curl -v above. The habit of reading the raw exchange, rather than only the rendered page, is the foundation everything else in this section builds on.
Go deeper
- DNS ExplainedHow names become addresses, how resolvers and caching really work, what each record type means, how encrypted DNS changes the picture, and what DNS reveals about an organization during reconnaissance.
- TLS and CertificatesWhat the padlock actually guarantees, how the TLS 1.3 handshake and certificate chain work, what a cipher suite and forward secrecy mean, the 0-RTT replay tradeoff, and the misconfigurations worth checking.
- Same-Origin Policy and CORSThe browser rule that keeps websites from reading each other's data, what it does and does not cover, exactly how CORS and preflights relax it, how postMessage crosses origins safely, and the misconfigurations that break it.